{"id":1015200,"date":"2024-03-20T09:00:00","date_gmt":"2024-03-20T16:00:00","guid":{"rendered":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/blog\/research-focus-week-of-march-18-2024\/"},"modified":"2024-07-19T10:47:14","modified_gmt":"2024-07-19T17:47:14","slug":"research-focus-week-of-march-18-2024","status":"publish","type":"post","link":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/blog\/research-focus-week-of-march-18-2024\/","title":{"rendered":"Research Focus: Week of March 18, 2024"},"content":{"rendered":"\n

Welcome to Research Focus, a series of blog posts that highlights notable publications, events, code\/datasets, new hires and other milestones from across the research community at Microsoft.<\/em><\/p><\/blockquote><\/figure>\n\n\n\n

\"Research<\/figure>\n\n\n\n

NEW RESEARCH<\/h3>\n\n\n\n

Fewer is More: Boosting LLM Reasoning with Reinforced Context Pruning<\/h2>\n\n\n\n

Large language models (LLMs) have shown impressive capabilities, yet they still struggle with math reasoning. In a recent paper: Fewer is More: Boosting LLM Reasoning with Reinforced Context Pruning<\/a>, researchers from Microsoft propose CoT-Influx, a novel approach that pushes the boundary of few-shot chain-of-Thought (CoT) learning to improve LLM mathematical reasoning. <\/p>\n\n\n\n

Given that adding more concise CoT examples in the prompt can improve LLM reasoning performance, CoT-Influx employs a coarse-to-fine pruner to maximize the input of effective and concise CoT examples. The pruner first selects as many crucial CoT examples as possible and then prunes unimportant tokens to fit the context window. A math reasoning dataset with diverse difficulty levels and reasoning steps is used to train the pruner, along with a math-specialized reinforcement learning approach. As a result, by enabling more CoT examples with double the context window size in tokens, CoT-Influx significantly outperforms various prompting baselines across various LLMs (LLaMA2-7B, 13B, 70B) and 5 math datasets, achieving up to 4.55% absolute improvements. Remarkably, without any fine-tuning, LLaMA2-70B with CoT-Influx surpasses GPT-3.5 and a wide range of larger LLMs (PaLM, Minerva 540B, etc.) on the GSM8K. CoT-Influx serves as a plug-and-play module for LLMs and is compatible with most existing reasoning prompting techniques, such as self-consistency and self-verification.<\/p>\n\n\n\n

\n
Read the paper<\/a><\/div>\n<\/div>\n\n\n\n\t
\n\t\t\n\n\t\t

\n\t\tSpotlight: Event Series<\/span>\n\t<\/p>\n\t\n\t

\n\t\t\t\t\t\t
\n\t\t\t\t\n\t\t\t\t\t\"Research\n\t\t\t\t<\/a>\n\t\t\t<\/div>\n\t\t\t\n\t\t\t
\n\n\t\t\t\t\t\t\t\t\t

Microsoft Research Forum<\/h2>\n\t\t\t\t\n\t\t\t\t\t\t\t\t

Join us for a continuous exchange of ideas about research in the era of general AI. Watch the latest episodes on demand.<\/p>\n\t\t\t\t\n\t\t\t\t\t\t\t\t

\n\t\t\t\t\t
\n\t\t\t\t\t\t\n\t\t\t\t\t\t\tWatch on-demand\t\t\t\t\t\t<\/a>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t\t\t<\/div>\n\t<\/div>\n\t<\/div>\n\t\n\n\n

NEW RESEARCH<\/h3>\n\n\n\n

From User Surveys to Telemetry-Driven Agents: Exploring the Potential of Personalized Productivity Solutions<\/h2>\n\n\n\n

Organizations and individuals continuously strive to enhance their efficiency, improve time management, and optimize their work processes. Rapid advancements in AI, natural language processing, and machine learning technologies create new opportunities to develop tools that boost productivity. <\/p>\n\n\n\n

In a recent paper: From User Surveys to Telemetry-Driven Agents: Exploring the Potential of Personalized Productivity Solutions<\/a>, researchers from Microsoft present a comprehensive, user-centric approach to understand preferences in AI-based productivity agents and develop personalized solutions. The research began with a survey of 363 participants, seeking to reveal users\u2019 specific needs and preferences for productivity agents such as relevant productivity challenges of information workers, preferred communication style and approach towards solving problems, and privacy expectations. With the survey insights, the researchers then developed a GPT-4 powered personalized productivity agent that uses telemetry data gathered from information workers via Viva Insights<\/a> to provide tailored assistance. The agent\u2019s performance was compared with alternative productivity-assistive tools, such as the traditional dashboard and AI-enabled summaries, in a study involving 40 participants. The findings highlight the importance of user-centric design, adaptability, and the balance between personalization and privacy in AI-assisted productivity tools. The insights distilled from this study could support future research to further enhance productivity solutions, ultimately leading to optimized efficiency and user experiences for information workers.<\/p>\n\n\n\n

\n
Read the paper<\/a><\/div>\n<\/div>\n\n\n\n
\n\n\n\n

NEW RESEARCH<\/h3>\n\n\n\n

LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens<\/h2>\n\n\n\n

The size of the context window of a large language model (LLM) determines the amount of text that can be entered for processing to generate responses. <\/strong>The window size is specifically measured by a\u202f<\/strong>number of tokens\u2014larger windows are more desirable. However, due to high fine-tuning costs, scarcity of long texts, and catastrophic values introduced by new token positions, current extended context windows are limited to around 128k tokens. <\/p>\n\n\n\n

In a recent paper: LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens<\/a>, researchers from Microsoft introduce a new method that extends the context window of pre-trained LLMs to an impressive 2.048 million tokens, without requiring direct fine-tuning on texts with extremely long lengths, which are scarce, while maintaining performance at the level of the original short context window. Extensive experiments on LLaMA2 and Mistral across various tasks demonstrate the effectiveness of this method. Models extended via LongRoPE retain the original architecture with minor modifications to the positional embedding and can reuse most pre-existing optimizations.<\/p>\n\n\n\n

\n
Read the paper<\/a><\/div>\n<\/div>\n\n\n\n
\n\n\n\n

NEW RESEARCH<\/h3>\n\n\n\n

Exploring Interaction Patterns for Debugging: Enhancing Conversational Capabilities of AI-assistants<\/h2>\n\n\n\n

Conversational interactions with large language models (LLMs) enable programmers to obtain natural language explanations for various software development tasks. However, LLMs often leap to action without sufficient context, giving rise to implicit assumptions and inaccurate responses. Conversations between developers and LLMs are primarily structured as question-answer pairs, where the developer is responsible for asking the right questions and sustaining conversations across multiple turns.  <\/p>\n\n\n\n

In a recent paper: Exploring Interaction Patterns for Debugging: Enhancing Conversational Capabilities of AI-assistants<\/a>, researchers from Microsoft draw inspiration from interaction patterns and conversation analysis to design Robin, an enhanced conversational AI-assistant for debugging. Robin works with the developer collaboratively, creating hypotheses about the bug’s root cause, testing them using IDE debugging features such as breakpoints and watches, and then proposing fixes. A user study with 12 industry professionals shows that equipping the LLM-driven debugging assistant to (1) leverage the insert expansion interaction pattern; (2) facilitate turn-taking; and (3) utilize debugging workflows, leads to lowered conversation barriers, effective fault localization, and 5x improvement in bug resolution rates.<\/p>\n\n\n\n

\n
Read the paper<\/a><\/div>\n<\/div>\n\n\n\n
\n\n\n\n

NEW RESEARCH<\/h3>\n\n\n\n

Ironies of Generative AI: Understanding and mitigating productivity loss in human-AI interactions<\/h2>\n\n\n\n

Generative AI (GenAI) systems, which can produce new content based on input like code, images, speech, video, and more, offer opportunities to increase user productivity in many tasks, such as programming and writing. However, while they boost productivity in some studies, many others show that users are working ineffectively with GenAI systems and actually losing productivity. These \u2018ironies of automation\u2019 have been observed for over three decades in human factors research on automation in aviation, automated driving, and intelligence.  <\/p>\n\n\n\n

In a recent paper: Ironies of Generative AI: Understanding and mitigating productivity loss in human-AI interactions<\/a>, researchers from Microsoft draw on this extensive research alongside recent GenAI user studies to outline four key reasons for why productivity loss can occur with GenAI systems: 1) a shift in users\u2019 roles from production to evaluation; 2) unhelpful restructuring of workflows; 3) interruptions; and 4) a tendency for automation to make easy tasks easier and hard tasks harder. We then suggest how human factors research can also inform GenAI system design to mitigate productivity loss by using approaches such as continuous feedback, system personalization, ecological interface design, task stabilization, and clear task allocation. Grounding developments in GenAI system usability in decades of research aims to ensure that the design of human-AI interactions in this rapidly moving field learns from history instead of repeating it. <\/p>\n\n\n\n

\n
Read the paper<\/a><\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"

Welcome to Research Focus, a series of blog posts that highlights notable publications, events, code\/datasets, new hires and other milestones from across the research community at Microsoft. Large language models (LLMs) have shown impressive capabilities, yet they still struggle with math reasoning. In a recent paper: Fewer is More: Boosting LLM Reasoning with Reinforced Context […]<\/p>\n","protected":false},"author":42735,"featured_media":1015620,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","msr-author-ordering":[{"type":"user_nicename","value":"Li Lyna Zhang","user_id":"38121"},{"type":"user_nicename","value":"Fan Yang","user_id":"31782"},{"type":"user_nicename","value":"Mao Yang","user_id":"32798"},{"type":"user_nicename","value":"Javier Hernandez","user_id":"38413"},{"type":"user_nicename","value":"Kael Rowan","user_id":"32466"},{"type":"user_nicename","value":"Judith Amores","user_id":"42003"},{"type":"user_nicename","value":"Jina Suh","user_id":"32311"},{"type":"user_nicename","value":"Gonzalo Ramos","user_id":"36419"},{"type":"user_nicename","value":"Brian Houck","user_id":"43092"},{"type":"user_nicename","value":"Shamsi Iqbal","user_id":"33592"},{"type":"user_nicename","value":"Mary Czerwinski","user_id":"32824"},{"type":"user_nicename","value":"Chengruidong Zhang","user_id":"42018"},{"type":"user_nicename","value":"Jiahang Xu","user_id":"41569"},{"type":"user_nicename","value":"Yasharth Bajpai","user_id":"42228"},{"type":"user_nicename","value":"Gustavo Soares","user_id":"39183"},{"type":"user_nicename","value":"Arjun Radhakrishna","user_id":"39405"},{"type":"user_nicename","value":"Chris Parnin","user_id":"41985"},{"type":"user_nicename","value":"Sumit Gulwani","user_id":"33755"},{"type":"user_nicename","value":"Lev Tankelevitch","user_id":"43209"},{"type":"user_nicename","value":"Abigail Sellen","user_id":"31112"},{"type":"user_nicename","value":"Sean Rintel","user_id":"33579"},{"type":"user_nicename","value":"Bhavya Chopra","user_id":"43434"}],"msr_hide_image_in_river":0,"footnotes":""},"categories":[1],"tags":[],"research-area":[13556,13554,13560],"msr-region":[],"msr-event-type":[],"msr-locale":[268875],"msr-post-option":[243984],"msr-impact-theme":[264846],"msr-promo-type":[],"msr-podcast-series":[],"class_list":["post-1015200","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-research-blog","msr-research-area-artificial-intelligence","msr-research-area-human-computer-interaction","msr-research-area-programming-languages-software-engineering","msr-locale-en_us","msr-post-option-blog-homepage-featured"],"msr_event_details":{"start":"","end":"","location":""},"podcast_url":"","podcast_episode":"","msr_research_lab":[199560,199561,199565],"msr_impact_theme":["Computing foundations"],"related-publications":[],"related-downloads":[],"related-videos":[],"related-academic-programs":[],"related-groups":[510017,578422,663303],"related-projects":[483294],"related-events":[],"related-researchers":[{"type":"user_nicename","value":"Fan Yang","user_id":31782,"display_name":"Fan Yang","author_link":"Fan Yang<\/a>","is_active":false,"last_first":"Yang, Fan","people_section":0,"alias":"fanyang"},{"type":"user_nicename","value":"Javier Hernandez","user_id":38413,"display_name":"Javier Hernandez","author_link":"Javier Hernandez<\/a>","is_active":false,"last_first":"Hernandez, Javier","people_section":0,"alias":"javierh"},{"type":"user_nicename","value":"Judith Amores","user_id":42003,"display_name":"Judith Amores","author_link":"Judith Amores<\/a>","is_active":false,"last_first":"Amores, Judith","people_section":0,"alias":"judithamores"},{"type":"user_nicename","value":"Brian Houck","user_id":43092,"display_name":"Brian Houck","author_link":"Brian Houck<\/a>","is_active":false,"last_first":"Houck, Brian","people_section":0,"alias":"bhouck"},{"type":"user_nicename","value":"Shamsi Iqbal","user_id":33592,"display_name":"Shamsi Iqbal","author_link":"Shamsi Iqbal<\/a>","is_active":false,"last_first":"Iqbal, Shamsi","people_section":0,"alias":"shamsi"},{"type":"user_nicename","value":"Jiahang Xu","user_id":41569,"display_name":"Jiahang Xu","author_link":"Jiahang Xu<\/a>","is_active":false,"last_first":"Xu, Jiahang","people_section":0,"alias":"jiahangxu"},{"type":"user_nicename","value":"Yasharth Bajpai","user_id":42228,"display_name":"Yasharth Bajpai","author_link":"Yasharth Bajpai<\/a>","is_active":false,"last_first":"Bajpai, Yasharth","people_section":0,"alias":"ybajpai"},{"type":"user_nicename","value":"Gustavo Soares","user_id":39183,"display_name":"Gustavo Soares","author_link":"Gustavo Soares<\/a>","is_active":false,"last_first":"Soares, Gustavo","people_section":0,"alias":"gsoares"},{"type":"user_nicename","value":"Arjun Radhakrishna","user_id":39405,"display_name":"Arjun Radhakrishna","author_link":"Arjun Radhakrishna<\/a>","is_active":false,"last_first":"Radhakrishna, Arjun","people_section":0,"alias":"arradha"},{"type":"user_nicename","value":"Chris Parnin","user_id":41985,"display_name":"Chris Parnin","author_link":"Chris Parnin<\/a>","is_active":false,"last_first":"Parnin, Chris","people_section":0,"alias":"chrisparnin"},{"type":"user_nicename","value":"Sumit Gulwani","user_id":33755,"display_name":"Sumit Gulwani","author_link":"Sumit Gulwani<\/a>","is_active":false,"last_first":"Gulwani, Sumit","people_section":0,"alias":"sumitg"},{"type":"user_nicename","value":"Lev Tankelevitch","user_id":43209,"display_name":"Lev Tankelevitch","author_link":"Lev Tankelevitch<\/a>","is_active":false,"last_first":"Tankelevitch, Lev","people_section":0,"alias":"levt"},{"type":"user_nicename","value":"Abigail Sellen","user_id":31112,"display_name":"Abigail Sellen","author_link":"Abigail Sellen<\/a>","is_active":false,"last_first":"Sellen, Abigail","people_section":0,"alias":"asellen"},{"type":"user_nicename","value":"Sean Rintel","user_id":33579,"display_name":"Sean Rintel","author_link":"Sean Rintel<\/a>","is_active":false,"last_first":"Rintel, Sean","people_section":0,"alias":"serintel"}],"msr_type":"Post","featured_image_thumbnail":"\"Research","byline":"","formattedDate":"March 20, 2024","formattedExcerpt":"Welcome to Research Focus, a series of blog posts that highlights notable publications, events, code\/datasets, new hires and other milestones from across the research community at Microsoft. Large language models (LLMs) have shown impressive capabilities, yet they still struggle with math reasoning. In a recent…","locale":{"slug":"en_us","name":"English","native":"","english":"English"},"_links":{"self":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/posts\/1015200","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/users\/42735"}],"replies":[{"embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/comments?post=1015200"}],"version-history":[{"count":14,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/posts\/1015200\/revisions"}],"predecessor-version":[{"id":1058682,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/posts\/1015200\/revisions\/1058682"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/media\/1015620"}],"wp:attachment":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/media?parent=1015200"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/categories?post=1015200"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/tags?post=1015200"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=1015200"},{"taxonomy":"msr-region","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-region?post=1015200"},{"taxonomy":"msr-event-type","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-event-type?post=1015200"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=1015200"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=1015200"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=1015200"},{"taxonomy":"msr-promo-type","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-promo-type?post=1015200"},{"taxonomy":"msr-podcast-series","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-podcast-series?post=1015200"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}