Google publishes how it tests health AI for accuracy, safety and usefulness, using expert reviewers as well as automated checks

Google says it has published a whitepaper, written with health experts Dr Elizabeth McGlynn and Dr Dana Gelb Safran, that sets out an end-to-end way to judge AI used for health needs. The framework scores models on accuracy, safety and usefulness. It combines closed-question benchmarks, automated graders called autoraters and expert human review, and covers both common and high-risk health scenarios.

• Treat any chatbot's health answer as a starting point, and check anything that affects a medicine, a symptom or a child with a pharmacist or doctor.
• If you run a business that touches health, such as a clinic, gym or care service, ask any AI vendor how they test health answers and who reviews them.

Source: Google blog. This is our short summary of their reporting. The paragraphs marked "Our read" and the things to do are SeedFoundry's opinion. The summary is drafted with AI tools. Image: Google blog, from its own announcement. How we source and credit