ChatGPT struggles with coding problems past its 2021 cutoff date
The Hindu
According to a new study, ChatGPT is good at coding problems from Leetcode until 2021 but past that year, its abilities drop by a wide margin.
OpenAI’s ChatGPT was good at solving coding problems before its training cutoff date of 2021, but the chatbot struggled to solve problems after that year, according to a study published by IEEE Transactions on Software Engineering.
ChatGPT’s ability to solve coding problems dropped by a wide margin if the problem set was outside its training data as language model was unable to even understand the meaning of basic problems, the study noted.
The AI chatbot’s success rates of 89% (easy), 71% (medium) and 40% (hard) for coding problems dipped after 2021 to 52% (easy). For hard problems, the rate fell to 0.66%.
The study covered 728 coding problems from Leetcode testing website in five programming languages -- C, C++, Java, JavaScript and Python. The chatbot’s prowess for tested on functionality, complexity and security.
(For top technology news of the day, subscribe to our tech newsletter Today’s Cache)
The research also highlighted that within the Leetcode dataset, ChatGPT was able to code better than humans in some cases, but carried security concerns. ChatGPT was also good at fixing compiling errors in a code but not very good at correcting its own mistakes. One of the researchers involved in the study, Yutian Tang from the University of Glasgow stated, “ChatGPT may generate incorrect code because it does not understand the meaning of algorithm problems, thus, this simple error feedback information is not enough.”
Tang said it was important to understand exactly where these AI models were limited so coders using them could find ways to work around it. “For example, when encountering more complex programming problems, developers can provide relevant knowledge as much as possible, and tell ChatGPT in the prompt which potential vulnerabilities to be aware of,” he added.
Telangana to introduce new clean energy policy to meet rising energy demands sustainably, aiming for 20,000 MW renewable capacity by 2030.
The girl, who was admitted to Aster CMI Hospital with alarming breathlessness and significant pallor, was diagnosed with Wegener’s Granulomatosis (now known as Granulomatosis with Polyangiitis or GPA), a rare autoimmune condition that causes spontaneous bleeding in the lungs, leading to acute respiratory failure.
Puzhal prison officials investigate fire accident that destroyed old files and cardboard boxes, possibly caused by prisoners.
ACB files case against IPS officer N. Sanjay in Andhra Pradesh. The official is accused of manipulating the tender processes for awarding contract for development and maintenance of AGNI-NOC portal, and conducting awareness meetings for SC/STs. It is alleged that the total value of properties stolen, or involved in the case is estimated at ₹1,75,86,600.
The holiday season spanning from Christmas to New Year has seen a surge in tourists visiting Mysuru with the hospitality sector announcing 100 per cent occupancy for the entire duration.
BJP leader and former MP, Prathap Simha, has backed the Mysuru City Corporation’s (MCC) proposal to name a stretch of KRS road in the city after Chief Minister Siddaramaiah.
Tourists flock to Visakhapatnam and Alluri Sitharama Raju districts for holidays, leading to full occupancy in hotels and attractions.
Umayalpuram Swaminatha Iyer's legacy in music and teachings remembered through stories and memories of his disciples and students.
Kalaburagi District Police Dog Squad bids emotional farewell to Reeta, a crucial part of operations and investigations.
India's vibrant startup ecosystem, driven by women, innovation, and government support, is shaping the global entrepreneurial landscape.
Mettur dam water level at 119.49 feet, inflow 2,711 cusecs, discharge 500 cusecs, near full capacity.