{"id":755974,"date":"2021-06-24T10:54:10","date_gmt":"2021-06-24T17:54:10","guid":{"rendered":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/?p=755974"},"modified":"2021-06-24T10:54:12","modified_gmt":"2021-06-24T17:54:12","slug":"microsoft-lreasoner-leads-the-reclor-challenge-on-logical-reasoning","status":"publish","type":"post","link":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/blog\/microsoft-lreasoner-leads-the-reclor-challenge-on-logical-reasoning\/","title":{"rendered":"Microsoft LReasoner leads the ReClor challenge on logical reasoning"},"content":{"rendered":"\n
\"\"<\/figure>\n\n\n\n

For many years AI researchers have sought to build upon traditional machine learning, which trains technology to process facts and learn from them, and develop machine reasoning, in which programs apply logic to data and solve problems \u2013 comparable to the way humans think. For a system to analyze multiple sets of logical arguments, it requires both critical thinking and combinatorial reasoning abilities.<\/p>\n\n\n\n

\n\t
\n\t\t
\n\t\t\t\t\t\tPublication <\/span>\n\t\t\tReClor: A Reading Comprehension Dataset Requiring Logical Reasoning<\/span> <\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n\n\n\n

One of the current benchmarks for evaluating a system\u2019s logical reasoning ability is ReClor, a Reading Comprehension Dataset Requiring Logical Reasoning<\/a>. ReClor is a dataset built from logical reasoning problems used in standardized admission tests, including the Law School Admission Test (LSAT) and Graduate Management Admission Test (GMAT). <\/p>\n\n\n\n

Today, we are excited to announce that Microsoft\u2019s LReasoner system is the top-rated performer on the official ReCLor leaderboard. LReasoner also significantly exceeded human performance, as measured by the average accuracy of 10 college students who each answered 10 randomly selected test questions and reported in the ReClor paper. <\/p>\n\n\n\n

\n
\n
\n\t
\n\t\t
\n\t\t\t\t\t\tArticle <\/span>\n\t\t\tMicrosoft LReasoner leads the ReClor challenge on logical reasoning<\/span> <\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n<\/div>\n\n\n\n
\n
\n\t
\n\t\t
\n\t\t\t\t\t\tPublication <\/span>\n\t\t\tLogic-Driven Context Extension and Data Augmentation for Logical Reasoning of Text<\/span> <\/span><\/a>\t\t\t\t\t<\/div>\n\t<\/article>\n<\/div>\n<\/div>\n<\/div>\n\n\n\n
\"Figure<\/a>
Figure1\uff1aLReasoner achieves the state-of-the-art performance on the official ReClor leaderboard<\/a><\/figcaption><\/figure><\/div>\n\n\n\n

The LSAT is a standardized exam established in 1947 that has become an essential benchmark in the law school admissions process. An example of the LSAT\u2019s challenging logical reasoning questions is shown in Figure 2. To answer such a question, a system needs to understand the logical symbols like \u201chave keyboarding skills\u201d, make complex inferences to extend the existing logical expressions according to logical rules and then match the logical expressions with the answer options.<\/p>\n\n\n\n

\"Figure
Figure 2: example of a logical reasoning test question<\/figcaption><\/figure><\/div>\n\n\n\n

To address this logical reasoning challenge in a realistic scenario, the Natural Language Computing Group<\/a> of Microsoft Research Asia proposed the LReasoner system, which helps the model find the answer to a problem by recognizing logical symbols and logical expressions in the text.<\/p>\n\n\n\n

LReasoner improves the reasoning ability of the previous pre-trained language models by using two novel techniques (illustrated in Figure 2): (1) logic-driven context extension framework, which aims to first identify logic symbols from the context and then infer extended logical expressions through logical equivalence law, and (2) logic-driven data augmentation algorithm, which applies contrastive learning to find logically different context to help the model better capture the logical information, especially the logical negation and conditional relationship.<\/p>\n\n\n\n

\"Three
Figure 3: Logic-driven context extension framework<\/figcaption><\/figure><\/div>\n\n\n\n

In surpassing human performance as demonstrated in Figure 1, the LReasoner has taken an essential step towards deeper logical reasoning by AI. The LReasoner system is also one of the first attempts by researchers to apply machine reasoning to real scenarios. In the future, the Natural Language Computing Group of Microsoft Research Asia will continue to explore new tasks and new methods in the field of machine reasoning and promote the research of knowledgeable and interpretable artificial intelligence.<\/p>\n\n\n\n

<\/p>\n","protected":false},"excerpt":{"rendered":"

For many years AI researchers have sought to build upon traditional machine learning, which trains technology to process facts and learn from them, and develop machine reasoning, in which programs apply logic to data and solve problems \u2013 comparable to the way humans think. For a system to analyze multiple sets of logical arguments, it […]<\/p>\n","protected":false},"author":38838,"featured_media":755998,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","msr-author-ordering":null,"msr_hide_image_in_river":0,"footnotes":""},"categories":[1],"tags":[],"research-area":[13556,13545],"msr-region":[],"msr-event-type":[],"msr-locale":[268875],"msr-post-option":[],"msr-impact-theme":[],"msr-promo-type":[],"msr-podcast-series":[],"class_list":["post-755974","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-research-blog","msr-research-area-artificial-intelligence","msr-research-area-human-language-technologies","msr-locale-en_us"],"msr_event_details":{"start":"","end":"","location":""},"podcast_url":"","podcast_episode":"","msr_research_lab":[],"msr_impact_theme":[],"related-publications":[],"related-downloads":[],"related-videos":[],"related-academic-programs":[],"related-groups":[144735],"related-projects":[],"related-events":[],"related-researchers":[{"type":"guest","value":"natural-language-computing-group","user_id":"756685","display_name":"Natural Language Computing Group ","author_link":"Natural Language Computing Group <\/a>","is_active":true,"last_first":"Natural Language Computing Group ","people_section":0,"alias":"natural-language-computing-group"}],"msr_type":"Post","featured_image_thumbnail":"\"Graphic","byline":"Natural Language Computing Group <\/a>","formattedDate":"June 24, 2021","formattedExcerpt":"For many years AI researchers have sought to build upon traditional machine learning, which trains technology to process facts and learn from them, and develop machine reasoning, in which programs apply logic to data and solve problems \u2013 comparable to the way humans think. For…","locale":{"slug":"en_us","name":"English","native":"","english":"English"},"_links":{"self":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/posts\/755974","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/users\/38838"}],"replies":[{"embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/comments?post=755974"}],"version-history":[{"count":13,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/posts\/755974\/revisions"}],"predecessor-version":[{"id":756751,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/posts\/755974\/revisions\/756751"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/media\/755998"}],"wp:attachment":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/media?parent=755974"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/categories?post=755974"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/tags?post=755974"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=755974"},{"taxonomy":"msr-region","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-region?post=755974"},{"taxonomy":"msr-event-type","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-event-type?post=755974"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=755974"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=755974"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=755974"},{"taxonomy":"msr-promo-type","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-promo-type?post=755974"},{"taxonomy":"msr-podcast-series","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-podcast-series?post=755974"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}