Researchers found that GPT-4’s responses to math and code generation questions in June 2023 were signficantly less likely to be correct than in March 2023.
Zaharia said that GPT-4’s success rate in response to the question “Is this number prime? Think step by step” fell from 97.6% in March 2023 to 2.4% in June 2023.When it came to code generation, both GPT-4 and GPT-3.5 performed substantially worse.
When asked to give the sum of all integers within a range, GPT-4’s success rate for a directly executable response dropped from 52% to 10%, while GPT-3.5’s declined from 22% to 2%. The only area where both models improved was visual reasoning — with around a 2% increase in successful exact-match responses.“It could definitely be that RLHF [reinforcement learning from human feedback] and fine-tuning are hitting a wall, but might also be bugs. Definitely seems tricky to manage quality,” Zaharia said.
Zaharia said the researchers wanted to conduct a longer study on the issue and asked his followers for feedback on which questions they should measure GPT’s performance.have highlighted the limitations and challenges facing AI development at a time when many tech companies are investing heavily into the technology and others have warned about its existential threat to humanity.
That led the researchers to conclude the jokes were likely learned and memorised directly from user input or training data rather than generated.
South Africa Latest News, South Africa Headlines
Similar News:You can also read news stories similar to this one that we have collected from other news sources.
DStv channel changes in 2023We compared DStv’s channel changes in 2023 to previous years.
Read more »
All-new Toyota Fortuner makes first appearance in new leak | The CitizenAll eyes on the all-new ToyotaFortuner🚘💨 leaked images confirm its imminent arrival. The upcoming Fortuner promises enhanced performance with a cutting-edge mild-hybrid 2.8 GD-6 turbodiesel engine 🚀💪 Motoring TheCitizenMotoring Find out more⬇️
Read more »
FREE | Read the July 2023 edition of Business Law & TaxRead about the effect of so-called 'smart' drugs on the workplace; the effect of genericide on a trademark, and the limits of restraint of trade, among other stories
Read more »
News24's 100 Young Mandela's of the Future 2023There are scores of young South Africans who are defying limits and dreaming up new possibilities for our nation. Today, News24 is celebrating 100 young leaders who've been named Young Mandelas of the Future.
Read more »
MANDELA DAY 2023: Private sector joins hands to fix Nelson Mandela Bay’s ‘horror hostel’The 112 teenagers boarding at the Gelvandale High School hostel have been living without running water, lights, sufficient bedding and adequate study facilities. A cold wind blows through bullet holes left by a gang fight and part of the hostel had to be shut down after gangsters began using it as a hideout.
Read more »
Pirates’ 2023/24 Season Squad Confirmed? | Soccer LadumaOrlando Pirates appear to have almost confirmed their squad for the 2023/24 season as the team’s pre-season preparations concluded in Europe. SLSiya Read more:
Read more »