{"id":1106850,"date":"2024-11-27T06:00:00","date_gmt":"2024-11-27T14:00:00","guid":{"rendered":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/?p=1106850"},"modified":"2024-12-18T15:21:47","modified_gmt":"2024-12-18T23:21:47","slug":"advances-in-run-time-strategies-for-next-generation-foundation-models","status":"publish","type":"post","link":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/blog\/advances-in-run-time-strategies-for-next-generation-foundation-models\/","title":{"rendered":"Advances in run-time strategies for next-generation foundation models"},"content":{"rendered":"\n
\"A<\/figure>\n\n\n\n

Groundbreaking advancements in frontier language models are progressing rapidly, paving the way for boosts in accuracy and reliability of generalist models, making them highly effective in specialized domains. As part of our ongoing exploration of foundation model capabilities, we developed Medprompt<\/a> last year\u2014a novel approach to maximize model performance on specialized domain and tasks without fine-tuning. By leveraging multiphase prompting, Medprompt optimizes inference by identifying the most effective chain-of-thought (CoT) examples at run time and <\/em>drawing on multiple calls to refine output. When deployed with GPT-4, Medprompt achieved an impressive 90.2% accuracy on the MedQA benchmark (USMLE-style), outperforming all other methods. <\/p>\n\n\n\n

\"A
Figure 1. Comparative analyses of performance of multiple models on MedQA.<\/figcaption><\/figure>\n\n\n\n

Less than a year later, our tests show the OpenAI o1-preview demonstrated superior performance over Medprompt, reaching 96% on the same benchmark (Figure 1)\u2014without using sophisticated prompt guidance and control. This advancement is driven by the model\u2019s integration of run-time strategies at its core, enabling state-of-the-art results on medical licensing exams in the United States and Japan, medical subsets of the Massive Multitask Language Understanding (MMLU) benchmark, and nursing exams (NCLEX) as shown in Figure 2. <\/p>\n\n\n\n

\"A
Figure 2. Comparisons on a wide range of medical challenge benchmarks.<\/figcaption><\/figure>\n\n\n\n

These results are notable, prompting us to publish our recent study, findings, and analyses, From Medprompt to o1: Exploration of Run-Time Strategies for Medical Challenge Problems and Beyond<\/em> (opens in new tab)<\/span><\/a>. But the numbers are only part of the story. In this blog, we discuss prompting strategies to make the most of o1-preview models and other factors to consider as well as directions forward for run-time strategies.<\/p>\n\n\n\n

Is o1-preview \u201cjust\u201d fancy prompting? <\/h2>\n\n\n\n

The introduction of the OpenAI o1 model series marks a significant shift from prior GPT models. Unlike GPT, o1 models are trained using reinforcement learning (RL) techniques that enable them to \u201cthink\u201d before generating outputs. While Medprompt relies on a cascade of operations with GPT-4 at run time guided by a multistage prompt, the o1 series incorporates this run-time reasoning directly into its RL-based design. The built-in functionality enables the o1 models to significantly outperform even the best results using GPT-4 and Medprompt. The performance gains come with a notable tradeoff: its per-token cost was approximately six times that of GPT-4o at the time of our evaluation. While the results for GPT-4o with Medprompt fall short of o1-preview model performance, the combination offers a more cost-effective alternative. The cost-benefit tradeoffs are highlighted in the following figure, with the x-axis presented on a logarithmic scale.<\/p>\n\n\n\n

\"A
Figure 3. Pareto frontier showing accuracy versus total API cost (log scale) on the MedQA benchmark (1273 questions total). o1-preview (Sep 2024) is compared with GPT-4o (Aug 2024) and GPT-4 Turbo (Nov 2023).<\/figcaption><\/figure>\n\n\n\n

Can we prompt engineer o1-preview?<\/h2>\n\n\n\n

The o1-preview model exhibits distinct run-time behaviors compared to the GPT series. While some of our more dynamic prompting strategies performed better than expected with o1-preview models, our most tried-and-true strategy was anything but consistent throughout our evaluation. Figure 4 captures specific performance results for Tailored Prompt, Ensembling, and Few-Shot Prompting on o1-preview. Here\u2019s a summary of our findings: <\/p>\n\n\n\n

    \n
  1. Tailored Prompt<\/strong>: While minimal prompting\u2014like a brief, one-sentence description followed by a question\u2014offered a strong baseline performance, detailed task descriptions were best for eliciting accurate responses.<\/li>\n\n\n\n
  2. Ensembling<\/strong>: Generating multiple answers per question and using majority voting across different reasoning paths boosted reliability, while shuffling answers in runs produced richer reasoning chains and improved outcomes. Ensembling continues to yield consistent performance improvements.<\/li>\n\n\n\n
  3. Few-Shot Prompting<\/strong>: Guiding the model with a few examples produced inconsistent results and, on average, decreased performance compared with GPT models.<\/li>\n<\/ol>\n\n\n\n
    \"Three
    Figure 4. Tests of different prompting strategies across benchmark datasets.<\/figcaption><\/figure>\n\n\n\n\t
    \n\t\t\n\n\t\t

    \n\t\tPODCAST SERIES<\/span>\n\t<\/p>\n\t\n\t

    \n\t\t\t\t\t\t
    \n\t\t\t\t\n\t\t\t\t\t\"Illustrated\n\t\t\t\t<\/a>\n\t\t\t<\/div>\n\t\t\t\n\t\t\t
    \n\n\t\t\t\t\t\t\t\t\t

    The AI Revolution in Medicine, Revisited<\/h2>\n\t\t\t\t\n\t\t\t\t\t\t\t\t

    Join Microsoft\u2019s Peter Lee on a journey to discover how AI is impacting healthcare and what it means for the future of medicine.<\/p>\n\t\t\t\t\n\t\t\t\t\t\t\t\t

    \n\t\t\t\t\t
    \n\t\t\t\t\t\t\n\t\t\t\t\t\t\tListen now\t\t\t\t\t\t<\/a>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t<\/div>\n\t<\/div>\n\t\n\n\n

    Do results stand in another language? <\/h2>\n\n\n\n
    \"A
    Figure 5. JMLE-2024: National medical licensing exam held in Japan (Feb 2024).<\/figcaption><\/figure>\n\n\n\n

    We expanded our research to include a new multilingual benchmark based on the Japanese national medical licensing exam. The JMLE (Japanese Medical Licensing Examination) is written in Japanese and administered in February 2024, after the o1-preview model\u2019s knowledge cutoff. Even without translation to English, the o1-preview model achieved a remarkable score of 98.2% accuracy (Figure 5), well above the exam\u2019s minimum passing score of approximately 80%. <\/em> <\/p>\n\n\n\n

    Do reasoning tokens improve performance? <\/h2>\n\n\n\n

    For fun, we conducted tests to determine whether increasing the number of reasoning tokens could improve performance. Our findings showed that by adjusting the prompt, we were able to consistently increase the number of reasoning tokens used by o1-preview, and the increase was directly correlated with improved performance as demonstrated in Figure 6.<\/p>\n\n\n\n

    \"A
    Figure 6. The effect of two prompting strategies that elicit variable length reasoning chains across benchmark datasets.<\/figcaption><\/figure>\n\n\n\n

    What\u2019s the takeaway? <\/h2>\n\n\n\n

    Bottom line: There\u2019s a little something for everyone when it comes to run-time strategies. We\u2019re excited by the performance gains from GPT models to o1-preview models. While these improvements are significant, so is the cost. For those needing proven accuracy on a budget, Medprompt leveraging calls to GPT-4 is a viable option for medicine and beyond. We summarize the relative performance of prompting strategies in Figure 7 to determine the best option, or check out the paper for a detailed breakdown of every dataset, experimental configuration, and prompt template (opens in new tab)<\/span><\/a>. <\/p>\n\n\n\n

    \"A
    Figure 7. Heatmap showing absolute accuracy and relative performance over baseline zero-shot prompt (in parenthesis) across all benchmark datasets.<\/figcaption><\/figure>\n\n\n\n

    Anything more to consider?<\/h2>\n\n\n\n

    We highlighted several considerations in the paper that are worth checking out. Here are three opportunities that are top of mind:<\/p>\n\n\n\n