A comparative evaluation of ChatGPT 3.5 and ChatGPT 4 in responses to selected genetics questions

OBJECTIVESTo evaluate the efficacy of ChatGPT 4 (GPT-4) in delivering genetic information about BRCA1, HFE, and MLH1, building on previous findings with ChatGPT 3.5 (GPT-3.5). To focus on assessing the utility, limitations, and ethical implications of using ChatGPT in medical settings.MATERIALS AND...

Full description

Saved in:
Bibliographic Details
Published in:Journal of the American Medical Informatics Association : JAMIA 2024-06
Main Authors: McGrath, Scott P, Kozel, Beth A, Gracefo, Sara, Sutherland, Nykole, Danford, Christopher J, Walton, Nephi
Format: Article
Language:English
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:OBJECTIVESTo evaluate the efficacy of ChatGPT 4 (GPT-4) in delivering genetic information about BRCA1, HFE, and MLH1, building on previous findings with ChatGPT 3.5 (GPT-3.5). To focus on assessing the utility, limitations, and ethical implications of using ChatGPT in medical settings.MATERIALS AND METHODSA structured survey was developed to assess GPT-4's clinical value. An expert panel of genetic counselors and clinical geneticists evaluated GPT-4's responses to these questions. We also performed comparative analysis with GPT-3.5, utilizing descriptive statistics and using Prism 9 for data analysis.RESULTSThe findings indicate improved accuracy in GPT-4 over GPT-3.5 (P 
ISSN:1527-974X
DOI:10.1093/jamia/ocae128