Montreal researchers put ChatGPT to the test, tell scientists to beware
CTV
Scientists who rely on artificial intelligence when they write their research papers could wind up spreading misinformation, according to a new Montreal study that looked at the quality and accuracy of information specifically from ChatGPT.
Scientists who rely on artificial intelligence when they write their research papers could wind up spreading misinformation, according to a new Montreal study that looked at the quality and accuracy of information specifically from ChatGPT.
The results of the study, published in Mayo Clinic Proceedings: Digital Health, showed that the chatbot provided dubious answers and included factual errors and fabricated references.
"I was shocked by the extent of fabricated references. I wasn't expecting it to be that significant," said co-author Dr.Esli Osmanlliu, an emergency physician at the Montreal Children's Hospital.
To test the AI model, he and two colleagues at CHU Sainte-Justine — Dr. Jocelyn Gravel and Madeleine D’Amours-Gravel — asked ChatGPT 20 medical questions that came from existing studies. Then, they asked the authors of the papers to rate its answers.
Out of 20 authors, 17 participated and concluded that most of the information the chatbot returned was questionable, resulting in a median score of 60 per cent.
"Specifically, in a certain case, it was suggesting a steroid be given by injection whereas, in fact, the treatment is by mouth so that's a pretty significant difference in the way we administer a medication," Osmanlliu said.
Nearly 70 per cent of the study references provided were concocted by the AI tool and just as concerning, they looked at first glance to be authentic, which means scientists can be more easily duped.
Tesla is recalling almost 700,000 vehicles because of an issue with the warning light on the tire pressure monitoring system.
Alaska Gov. Mike Dunleavy’s wish list for the incoming Trump administration includes oil and gas exploration in the pristine Arctic National Wildlife Refuge and reversing restrictions on logging and road-building in a temperate rainforest that provides habitat for wolves, bears and salmon.
The press freedom group Reporters Without Borders is urging Apple to remove its newly introduced artificial intelligence feature that summarizes news stories after it produced a false headline from the BBC.
A yearlong trial of 493 adults, with an average age of 53, found vagus nerve stimulation therapy led to improvement in participants' depressive symptoms, ability to complete daily tasks and quality of life.
Two thousand years ago, the one-time rulers of modern-day Turkiye's southwestern corner had a fully functioning democratic federation.
U.S. auto safety regulators have backed away from seeking a giant recall of nearly 50 million air bag inflators and will investigate further after the auto industry raised questions about whether all of the inflators are defective.
The world's largest hornet, an invasive breed dubbed the “murder hornet” for its dangerous sting and ability to slaughter a honey bee hive in a matter of hours, has been declared eradicated in the U.S., five years after being spotted for the first time in Washington state near the Canadian border.
Scientists have found unprecedented evidence of another type of squirrel exhibiting carnivorous behaviours, including hunting, killing and eating voles, according to a new study.
A patient in Louisiana has been hospitalized with a severe case of H5N1 bird flu, the U.S. Centers for Disease Control and Prevention said, the first such case in the United States.
With the high cost of living increasingly a concern, some seniors are making sacrifices to help their adult children and grandchildren make ends meet. Here are some of their stories.
Japanese automakers Nissan Motor Corp. and Honda Motor Co. confirmed Wednesday that they are discussing closer collaboration but denied reports they have decided on a merger.