{"id":985488,"date":"2023-11-27T09:00:00","date_gmt":"2023-11-27T17:00:00","guid":{"rendered":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/?p=985488"},"modified":"2023-11-29T08:18:22","modified_gmt":"2023-11-29T16:18:22","slug":"gpt-4s-potential-in-shaping-the-future-of-radiology","status":"publish","type":"post","link":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/blog\/gpt-4s-potential-in-shaping-the-future-of-radiology\/","title":{"rendered":"GPT-4’s potential in shaping the future of radiology"},"content":{"rendered":"\n

This research paper is being presented at the <\/em><\/strong>2023 Conference on Empirical Methods in Natural Language Processing<\/em><\/strong> (opens in new tab)<\/span><\/a> (EMNLP 2023), the premier conference on natural language processing and artificial intelligence.<\/em><\/strong><\/p>\n\n\n\n

\"EMNLP<\/figure>\n\n\n\n
\n\t
\n\t\t
\n\t\t\t\t\t\tProject<\/span>\n\t\t\tProject MAIRA<\/span> <\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n\n\n\n

In recent years, AI has been increasingly integrated into healthcare, bringing about new areas of focus and priority, such as diagnostics, treatment planning, patient engagement. While AI\u2019s contribution in certain fields like image analysis and drug interaction is widely recognized, its potential in natural language tasks with these newer areas presents an intriguing research opportunity. <\/p>\n\n\n\n

One notable advancement in this area involves GPT-4’s impressive performance<\/a> on medical competency exams and benchmark datasets. GPT-4 has also demonstrated potential utility (opens in new tab)<\/span><\/a> in medical consultations, providing a promising outlook for healthcare innovation.<\/p>\n\n\n\n

Progressing radiology AI for real problems<\/h2>\n\n\n\n

Our paper, \u201cExploring the Boundaries of GPT-4 in Radiology<\/a>,\u201d which we are presenting at EMNLP 2023 (opens in new tab)<\/span><\/a>, further explores GPT-4\u2019s potential in healthcare, focusing on its abilities and limitations in radiology\u2014a field that is crucial in disease diagnosis and treatment through imaging technologies like x-rays, computed tomography (CT) and magnetic resonance imaging (MRI). We collaborated with our colleagues at Nuance (opens in new tab)<\/span><\/a>, a Microsoft company, whose solution, PowerScribe, is used by more than 80 percent of US radiologists. Together, we aimed to better understand technology\u2019s impact on radiologists\u2019 workflow.<\/p>\n\n\n\n

Our research included a comprehensive evaluation and error analysis framework to rigorously assess GPT-4\u2019s ability to process radiology reports, including common language understanding and generation tasks in radiology, such as disease classification and findings summarization. This framework was developed in collaboration with a board-certified radiologist to tackle more intricate and challenging real-world scenarios in radiology and move beyond mere metric scores.<\/p>\n\n\n\n

We also explored various effective zero-, few-shot, and chain-of-thought (CoT) prompting techniques for GPT-4 across different radiology tasks and experimented with approaches to improve the reliability of GPT-4 outputs. For each task, GPT-4 performance was benchmarked against prior GPT-3.5 models and respective state-of-the-art radiology models. <\/p>\n\n\n\n

We found that GPT-4 demonstrates new state-of-the-art performance in some tasks, achieving about a 10-percent absolute improvement over existing models, as shown in Table 1. Surprisingly, we found radiology report summaries generated by GPT-4 to be comparable and, in some cases, even preferred over those written by experienced radiologists, with one example illustrated in Table 2.<\/p>\n\n\n\n

\"Table
Table 1: Results overview. GPT-4 either outperforms or is on par with previous state-of-the-art (SOTA) multimodal LLMs.<\/figcaption><\/figure>\n\n\n\n
\"Table
Table 2. Examples where GPT-4 findings summaries are favored over existing manually written ones on the Open-i dataset. In both examples, GPT-4 outputs are more faithful and provide more complete details on the findings.<\/figcaption><\/figure>\n\n\n\n

Another encouraging prospect for GPT-4 is its ability to automatically structure radiology reports, as schematically illustrated in Figure 1. These reports, based on a radiologist\u2019s interpretation of medical images like x-rays and include patients\u2019 clinical history, are often complex and unstructured, making them difficult to interpret. Research shows (opens in new tab)<\/span><\/a> that structuring these reports can improve standardization and consistency in disease descriptions, making them easier to interpret by other healthcare providers and more easily searchable for research and quality improvement initiatives. Additionally, using GPT-4 to structure and standardize radiology reports can further support efforts to augment real-world data (RWD) and its use for real-world evidence<\/a> (RWE). This can complement more robust and comprehensive clinical trials and, in turn, accelerate the application of research findings into clinical practice.<\/p>\n\n\n\n

Beyond radiology, GPT-4\u2019s potential extends to translating medical reports into more empathetic (opens in new tab)<\/span><\/a> and understandable formats for patients and other health professionals. This innovation could revolutionize patient engagement and education, making it easier for them and their carers to actively participate in their healthcare.<\/p>\n\n\n\n\t

\n\t\t\n\n\t\t

\n\t\tSpotlight: AI-POWERED EXPERIENCE<\/span>\n\t<\/p>\n\t\n\t

\n\t\t\t\t\t\t
\n\t\t\t\t\n\t\t\t\t\t\"\"\n\t\t\t\t<\/a>\n\t\t\t<\/div>\n\t\t\t\n\t\t\t
\n\n\t\t\t\t\t\t\t\t\t

Microsoft research copilot experience<\/h2>\n\t\t\t\t\n\t\t\t\t\t\t\t\t

Discover more about research at Microsoft through our AI-powered experience<\/p>\n\t\t\t\t\n\t\t\t\t\t\t\t\t

\n\t\t\t\t\t
\n\t\t\t\t\t\t\n\t\t\t\t\t\t\tStart now\t\t\t\t\t\t<\/a>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t<\/div>\n\t<\/div>\n\t\n\n\n

A promising path toward advancing radiology and beyond<\/h2>\n\n\n\n

When used with human oversight, GPT-4 also has the potential to transform radiology by assisting professionals in their day-to-day tasks. As we continue to explore this cutting-edge technology, there is great promise in improving our evaluation results of GPT-4 by investigating how it can be verified more thoroughly and finding ways to improve its accuracy and reliability. <\/p>\n\n\n\n

Our research highlights GPT-4\u2019s potential in advancing radiology and other medical specialties, and while our results are encouraging, they require further validation through extensive research and clinical trials. Nonetheless, the emergence of GPT-4 heralds an exciting future for radiology. It will take the entire medical community working alongside other stakeholders in technology and policy to determine the appropriate use of these tools and responsibly realize the opportunity to transform healthcare. We eagerly anticipate its transformative impact towards improving patient care and safety.<\/p>\n\n\n\n

Learn more about this work by visiting the Project MAIRA (opens in new tab)<\/span><\/a> (Multimodal AI for Radiology Applications) page.<\/p>\n\n\n\n

Acknowledgements <\/h3>\n\n\n\n

We\u2019d like to thank our coauthors: Qianchu Liu,\u202fStephanie Hyland<\/a>,\u202fShruthi Bannur<\/a>,\u202fKenza Bouzid<\/a>,\u202fDaniel C. Castro<\/a>,\u202fMaria Teodora Wetscherek,\u202fRobert Tinn,\u202fHarshita Sharma<\/a>,\u202fFernando Perez-Garcia<\/a>,\u202fAnton Schwaighofer<\/a>,\u202fPranav Rajpurkar,\u202fSameer Tajdin Khanna,\u202fHoifung Poon<\/a>,\u202fNaoto Usuyama<\/a>,\u202fAnja Thieme<\/a>,\u202fAditya V. Nori<\/a>,\u202fOzan Oktay<\/a> <\/p>\n","protected":false},"excerpt":{"rendered":"

This research paper is being presented at the 2023 Conference on Empirical Methods in Natural Language Processing (opens in new tab) (EMNLP 2023), the premier conference on natural language processing and artificial intelligence. In recent years, AI has been increasingly integrated into healthcare, bringing about new areas of focus and priority, such as diagnostics, treatment […]<\/p>\n","protected":false},"author":42735,"featured_media":984078,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","msr-author-ordering":[{"type":"user_nicename","value":"Javier Alvarez-Valle","user_id":"32137"},{"type":"user_nicename","value":"Matthew Lungren","user_id":"42792"}],"msr_hide_image_in_river":0,"footnotes":""},"categories":[1],"tags":[],"research-area":[13556,13553],"msr-region":[],"msr-event-type":[],"msr-locale":[268875],"msr-post-option":[243984],"msr-impact-theme":[261673],"msr-promo-type":[],"msr-podcast-series":[],"class_list":["post-985488","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-research-blog","msr-research-area-artificial-intelligence","msr-research-area-medical-health-genomics","msr-locale-en_us","msr-post-option-blog-homepage-featured"],"msr_event_details":{"start":"","end":"","location":""},"podcast_url":"","podcast_episode":"","msr_research_lab":[849856],"msr_impact_theme":["Health"],"related-publications":[],"related-downloads":[],"related-videos":[],"related-academic-programs":[],"related-groups":[780706,952050,1143270],"related-projects":[978063],"related-events":[],"related-researchers":[],"msr_type":"Post","featured_image_thumbnail":"\"EMNLP","byline":"Javier Alvarez-Valle and Matthew Lungren","formattedDate":"November 27, 2023","formattedExcerpt":"This research paper is being presented at the 2023 Conference on Empirical Methods in Natural Language Processing (opens in new tab) (EMNLP 2023), the premier conference on natural language processing and artificial intelligence. In recent years, AI has been increasingly integrated into healthcare, bringing about…","locale":{"slug":"en_us","name":"English","native":"","english":"English"},"_links":{"self":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/posts\/985488","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/users\/42735"}],"replies":[{"embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/comments?post=985488"}],"version-history":[{"count":33,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/posts\/985488\/revisions"}],"predecessor-version":[{"id":988098,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/posts\/985488\/revisions\/988098"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/media\/984078"}],"wp:attachment":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/media?parent=985488"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/categories?post=985488"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/tags?post=985488"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=985488"},{"taxonomy":"msr-region","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-region?post=985488"},{"taxonomy":"msr-event-type","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-event-type?post=985488"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=985488"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=985488"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=985488"},{"taxonomy":"msr-promo-type","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-promo-type?post=985488"},{"taxonomy":"msr-podcast-series","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-podcast-series?post=985488"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}