Google AI health chatbot passes U.S. medical exam: study

Google's AI-powered Med-PaLM has achieved a passing grade on the U.S. Medical Licensing Exam, but still falls short of human doctors. ChatGPT, developed by OpenAI, has also achieved passing or near passing results. Google has developed a new evaluation benchmark to reduce "hallucinations" and a newer version of Med-PaLM has reached 86.5%. AI experts caution that AI should be used as an assistant, not a decision maker. Med-PaLM is being tested at the Mayo Clinic for administrative tasks.

Google’s artificial intelligence-powered medical chatbot has achieved a passing grade on a tough US medical licensing exam, but it’s answers still fall short of those from human doctors, a peer-reviewed study said on July 12

Last year the release of ChatGPT— whose developer OpenAI is backed by Google’s rival Microsoft— kicked off a race between tech giants in the burgeoning field of AI.

While much has been made about the future possibilities— and dangers— of AI, health is one area where the technology had already shown tangible progress, with algorithms able to read certain medical scans as well as humans.

Google first unveiled its AI tool for answering medical questions, called Med-PaLM, in a preprint study in December. Unlike ChatGPT, it has not been released to the public.

The U.S. tech giant says Med-PaLM is the first large language model, an AI technique trained on vast amounts of human-produced text, to pass the U.S. Medical Licensing Examination (USMLE).

A passing grade for the exam, which is taken by medical students and physicians-in-training in the United States, is around 60%.

In February, a study said that ChatGPT had achieved passing or near passing results.