{"id":2813,"date":"2011-03-08T09:25:00","date_gmt":"2011-03-08T09:25:00","guid":{"rendered":"https:\/\/blogs.msdn.microsoft.com\/msr_er\/2011\/03\/08\/innereye-visual-recognition-in-the-hospital\/"},"modified":"2016-07-20T07:34:05","modified_gmt":"2016-07-20T14:34:05","slug":"innereye-visual-recognition-in-the-hospital","status":"publish","type":"post","link":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/blog\/innereye-visual-recognition-in-the-hospital\/","title":{"rendered":"InnerEye: Visual Recognition in the Hospital"},"content":{"rendered":"<p><span style=\"font-family: verdana,geneva;\"><span style=\"font-size: medium;\">The neurosurgeon hovers over the patient, preparing to excise a life-threatening brain tumor. In this delicate operation, there is no margin for error: the tumor needs to be cut out with minimal damage to the surrounding healthy tissue. By using simple hand gestures, the surgeon signals a computer to display high-resolution scans of the patient&rsquo;s brain, showing the physician where to place her scalpel, detailing the boundaries between diseased and healthy tissue. No longer must the neurosurgeon stop to refer to the patient&rsquo;s image data during the operation, removing her gloves and potentially compromising the sterile surgical field. The upshot for the patient: reduced time under anesthesia and a lower risk of introduced infection. <\/span><\/span><\/p>\n<p><span style=\"font-family: verdana,geneva;\"><span style=\"font-size: medium;\"><img decoding=\"async\" src=\"https:\/\/msdnshared.blob.core.windows.net\/media\/MSDNBlogsFS\/prod.evol.blogs.msdn.com\/CommunityServer.Blogs.Components.WeblogFiles\/00\/00\/01\/32\/81\/2642.InnerEye_scans_496px.jpg\" original-url=\"http:\/\/blogs.msdn.com\/resized-image.ashx\/__size\/496x394\/__key\/CommunityServer-Blogs-Components-WeblogFiles\/00-00-01-32-81\/2642.InnerEye_5F00_scans_5F00_496px.jpg\" alt=\"Interactive Segmentation of CT and MR Scans\" title=\"TechFest Shows off Advances in Human-Computer Interactions\" style=\"border: 0px currentColor;\" \/><\/span><\/span><\/p>\n<p><span style=\"font-family: verdana,geneva;\"><span style=\"font-size: medium;\">Science fiction? Far from it. This scenario and others like it are on the verge of realization thanks to ground-breaking <\/span><\/span><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"http:\/\/research.microsoft.com\/en-us\/projects\/medicalimageanalysis\/default.aspx\"><span style=\"font-family: verdana,geneva;\"><span style=\"font-size: medium;\">InnerEye<\/span><\/span><span class=\"sr-only\"> (opens in new tab)<\/span><\/a><span style=\"font-family: verdana,geneva;\"><span style=\"font-size: medium;\"> project being conducted by <\/span><\/span><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"http:\/\/research.microsoft.com\/en-us\/default.aspx\"><span style=\"font-family: verdana,geneva;\"><span style=\"font-size: medium;\">Microsoft Research<\/span><\/span><span class=\"sr-only\"> (opens in new tab)<\/span><\/a><span style=\"font-family: verdana,geneva;\"><span style=\"font-size: medium;\"> and a host of collaborators, including <\/span><\/span><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"http:\/\/www.hopkinsmedicine.org\/\"><span style=\"font-family: verdana,geneva;\"><span style=\"font-size: medium;\">Johns Hopkins Medical Institute<\/span><\/span><span class=\"sr-only\"> (opens in new tab)<\/span><\/a><span style=\"font-family: verdana,geneva;\"><span style=\"font-size: medium;\">, <\/span><\/span><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"http:\/\/www.ox.ac.uk\/\"><span style=\"font-family: verdana,geneva;\"><span style=\"font-size: medium;\">The University of Oxford<\/span><\/span><span class=\"sr-only\"> (opens in new tab)<\/span><\/a><span style=\"font-family: verdana,geneva;\"><span style=\"font-size: medium;\">, <\/span><\/span><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"http:\/\/www.med.cornell.edu\/#id=29\"><span style=\"font-family: verdana,geneva;\"><span style=\"font-size: medium;\">Cornell Medical School<\/span><\/span><span class=\"sr-only\"> (opens in new tab)<\/span><\/a><span style=\"font-family: verdana,geneva;\"><span style=\"font-size: medium;\">, <\/span><\/span><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"http:\/\/www.massgeneral.org\/\"><span style=\"font-family: verdana,geneva;\"><span style=\"font-size: medium;\">Massachusetts General Hospital<\/span><\/span><span class=\"sr-only\"> (opens in new tab)<\/span><\/a><span style=\"font-family: verdana,geneva;\"><span style=\"font-size: medium;\">, the&nbsp;<\/span><\/span><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"http:\/\/www.washington.edu\/\"><span style=\"font-family: verdana,geneva;\"><span style=\"font-size: medium;\">University of Washington<\/span><\/span><span class=\"sr-only\"> (opens in new tab)<\/span><\/a><span style=\"font-family: verdana,geneva;\"><span style=\"font-size: medium;\">, <\/span><\/span><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"http:\/\/www.kcl.ac.uk\/index.aspx\"><span style=\"font-family: verdana,geneva;\"><span style=\"font-size: medium;\">Kings College London<\/span><\/span><span class=\"sr-only\"> (opens in new tab)<\/span><\/a><span style=\"font-family: verdana,geneva;\"><span style=\"font-size: medium;\">, and <\/span><\/span><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"http:\/\/www.cuh.org.uk\/\"><span style=\"font-family: verdana,geneva;\"><span style=\"font-size: medium;\">Cambridge University Hospitals<\/span><\/span><span class=\"sr-only\"> (opens in new tab)<\/span><\/a><span style=\"font-family: verdana,geneva;\"><span style=\"font-size: medium;\">.&nbsp; <\/span><\/span><\/p>\n<p><span style=\"font-family: verdana,geneva;\"><span style=\"font-size: medium;\">The analysis of medical images is essential in modern medicine. As images have achieved higher and higher resolutions, the increasing amount of patient data has presented new challenges and opportunities, from diagnosis to therapy. The InnerEye research shows how a single, underlying image-recognition algorithm can enable a multitude of clinical applications, such as semantic image navigation, multimodal image registration, quality control, content-based image search, and natural user interfaces for surgery.<\/span><\/span><\/p>\n<p><span style=\"font-family: verdana,geneva;\"><span style=\"font-size: medium;\">InnerEye takes advantage of advances in computer-human interactions that have put computers on a path to work for us and collaborate with us. The development of a natural user interface (NUI) enables computers to adapt to you and be more integrated into your environment via speech, touch, and gesture. As NUI systems become more powerful and are imbued with more situational awareness, they can provide beneficial, real-time interactions that will be seamless and naturally suited to your context&mdash;in short, systems will understand where you are and what you&rsquo;re doing. <\/span><\/span><\/p>\n<p><span style=\"font-family: verdana,geneva;\"><span style=\"font-size: medium;\">At this year&rsquo;s <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"http:\/\/research.microsoft.com\/en-us\/events\/techfest2011\/default.aspx\">TechFest<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>&mdash;the annual event that showcases the latest work from Microsoft Research&rsquo;s labs around the world&mdash;InnerEye is one of several projects that show where Microsoft is headed with NUI technologies, and how &ldquo;futuristic&rdquo; computing experiences are quickly becoming a reality. Building on the success of Kinect&mdash;a prime example of NUI technology reaching consumer scale&mdash;Microsoft Research continues to explore technologies that will enable the coming shift in how humans will communicate with machines, and vice versa. The possibilities are seemingly endless in how we approach the integration of computing into our lives and can enable a new era of creativity, social interaction, and technological scenarios.<\/span><\/span><\/p>\n<p><i><span style=\"font-family: verdana,geneva;\"><span style=\"font-size: medium;\">&mdash;<a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"http:\/\/research.microsoft.com\/en-us\/people\/antcrim\/default.aspx\">Antonio Criminisi<span class=\"sr-only\"> (opens in new tab)<\/span><\/a><i><span style=\"font-family: verdana,geneva;\"><span style=\"font-size: medium;\">, Researcher, Microsoft Research and <a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"http:\/\/research.microsoft.com\/en-us\/people\/ktolle\/default.aspx\">Kristin Tolle<span class=\"sr-only\"> (opens in new tab)<\/span><\/a>, Director, Natural User Interface Team, Microsoft Research Connections division of Microsoft Research<\/span><\/span><\/i><\/span><\/span><\/i><\/p>\n<p><strong><span style=\"font-family: verdana,geneva;\"><span style=\"font-size: medium;\">Learn More<\/span><\/span><\/strong><\/p>\n<ul>\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"http:\/\/www.msnbc.msn.com\/id\/21134540\/vp\/42017100#4201710\"><span style=\"font-family: verdana,geneva;\"><span style=\"font-size: small;\">How New Digital Body Scanning Will Save Lives<\/span><\/span><span class=\"sr-only\"> (opens in new tab)<\/span><\/a> <span style=\"font-family: verdana,geneva;\"><span style=\"font-size: small;\">(MSNBC Video)<\/span><\/span><\/li>\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"http:\/\/research.microsoft.com\/en-us\/news\/headlines\/tf_nui_zdnet_030911.aspx\"><span style=\"font-family: verdana,geneva;\"><span style=\"font-size: small;\">At Microsoft, NUI Goes Beyond Fun and Games<\/span><\/span><span class=\"sr-only\"> (opens in new tab)<\/span><\/a><span style=\"font-family: verdana,geneva;\"><span style=\"font-size: small;\">&nbsp;(ZDNet)<\/span><\/span><\/li>\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"http:\/\/research.microsoft.com\/en-us\/um\/people\/antcrim\/msrchighresmedicalimages.zip\"><span style=\"font-family: verdana,geneva;\"><span style=\"font-size: small;\"><\/span><\/span><span class=\"sr-only\"> (opens in new tab)<\/span><\/a><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"http:\/\/www.facebook.com\/?ref=home#!\/video\/video.php?v=1622157434683&oid=33575580843&comments\"><span style=\"font-family: verdana,geneva;\"><span style=\"font-size: small;\">TechFest 2011: InnerEye: Visual Recognition in the Hospital Demo Video<\/span><\/span><span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/li>\n<li><span style=\"font-family: verdana,geneva;\"><span style=\"font-size: small;\"><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"http:\/\/research.microsoft.com\/en-us\/um\/people\/antcrim\/msrchighresmedicalimages.zip\"><span style=\"font-family: verdana,geneva;\">Download Samples of High-Resolution Medical Images<\/span><span class=\"sr-only\"> (opens in new tab)<\/span><\/a><span style=\"font-family: verdana,geneva;\"><span style=\"font-size: small;\"> <\/span><\/span><\/span><\/span><\/li>\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"http:\/\/research.microsoft.com\/en-us\/um\/people\/antcrim\/ACriminisi_MICCAI_PMMIAI_2009.wmv\"><span style=\"font-family: verdana,geneva;\"><span style=\"font-size: small;\"><span style=\"font-family: verdana,geneva;\"><span style=\"font-size: small;\">Automatic Localization of Anatomical Structures Demo<\/span><\/span><\/span><\/span><span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/li>\n<li><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"http:\/\/research.microsoft.com\/en-us\/um\/people\/antcrim\/ACriminisi_Medical_Segmentation1.wmv\"><span style=\"font-family: verdana,geneva;\"><span style=\"font-size: small;\"><span style=\"font-family: verdana,geneva;\"><span style=\"font-size: small;\">Interactive Segmentation of CT and MR Scans Demo<\/span><\/span><\/span><\/span><span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/li>\n<li><span style=\"font-family: verdana,geneva;\"><span style=\"font-size: small;\"><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"http:\/\/research.microsoft.com\/en-us\/collaboration\/focus\/nui\/default.aspx\"><span style=\"font-family: verdana,geneva;\"><span style=\"font-size: small;\">Natural User Interface<\/span><\/span><span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/span><\/span><\/li>\n<li><span style=\"font-family: verdana,geneva;\"><span style=\"font-size: small;\"><a class=\"msr-external-link glyph-append glyph-append-open-in-new-tab glyph-append-xsmall\" rel=\"noopener noreferrer\" target=\"_blank\" href=\"http:\/\/research.microsoft.com\/en-us\/collaboration\/default.aspx\"><span style=\"font-family: verdana,geneva;\"><span style=\"font-size: small;\">Microsoft Research Connections<\/span><\/span><span class=\"sr-only\"> (opens in new tab)<\/span><\/a><\/span><\/span><\/li>\n<li><span style=\"font-family: verdana,geneva;\"><span style=\"font-size: small;\"><a target=\"_blank\" href=\"http:\/\/cm-edgetun.pages.dev\/en-us\/researchconnections\/\"><span style=\"font-family: verdana,geneva;\"><span style=\"font-size: small;\">Microsoft Research Connections on Microsoft.com<\/span><\/span><\/a><\/span><\/span><\/li>\n<\/ul>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>The neurosurgeon hovers over the patient, preparing to excise a life-threatening brain tumor. In this delicate operation, there is no margin for error: the tumor needs to be cut out with minimal damage to the surrounding healthy tissue. By using simple hand gestures, the surgeon signals a computer to display high-resolution scans of the patient&rsquo;s [&hellip;]<\/p>\n","protected":false},"author":32627,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","msr-author-ordering":[],"msr_hide_image_in_river":0,"footnotes":""},"categories":[1],"tags":[194684,194894,195172,195181,195884,195914,196056,187348,196150,196171,196311,196338,196347,193504,196439,196576,196625,186910,197147,197348,193514,197443,197444],"research-area":[],"msr-region":[],"msr-event-type":[],"msr-locale":[268875],"msr-post-option":[],"msr-impact-theme":[],"msr-promo-type":[],"msr-podcast-series":[],"class_list":["post-2813","post","type-post","status-publish","format-standard","hentry","category-research-blog","tag-antonio-criminisi","tag-cambridge-university-hospitals","tag-content-based-image-search","tag-cornell-medical-school","tag-image-recognition-algorithm","tag-innereye","tag-johns-hopkins-medical-institute","tag-kinect","tag-kings-college-london","tag-kristin-tolle","tag-massachusetts-general-hospital","tag-medical-images","tag-medicine","tag-microsoft-research","tag-microsoft-research-connections","tag-multimodal-image-registration","tag-natural-user-interface","tag-nui","tag-semantic-image-navigation","tag-surgery","tag-techfest","tag-the-university-of-oxford","tag-the-university-of-washington","msr-locale-en_us"],"msr_event_details":{"start":"","end":"","location":""},"podcast_url":"","podcast_episode":"","msr_research_lab":[],"msr_impact_theme":[],"related-publications":[],"related-downloads":[],"related-videos":[],"related-academic-programs":[],"related-groups":[780706],"related-projects":[740356,169659],"related-events":[],"related-researchers":[],"msr_type":"Post","byline":"","formattedDate":"March 8, 2011","formattedExcerpt":"The neurosurgeon hovers over the patient, preparing to excise a life-threatening brain tumor. In this delicate operation, there is no margin for error: the tumor needs to be cut out with minimal damage to the surrounding healthy tissue. By using simple hand gestures, the surgeon&hellip;","locale":{"slug":"en_us","name":"English","native":"","english":"English"},"_links":{"self":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/posts\/2813","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/users\/32627"}],"replies":[{"embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/comments?post=2813"}],"version-history":[{"count":1,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/posts\/2813\/revisions"}],"predecessor-version":[{"id":262425,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/posts\/2813\/revisions\/262425"}],"wp:attachment":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/media?parent=2813"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/categories?post=2813"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/tags?post=2813"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=2813"},{"taxonomy":"msr-region","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-region?post=2813"},{"taxonomy":"msr-event-type","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-event-type?post=2813"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=2813"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=2813"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=2813"},{"taxonomy":"msr-promo-type","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-promo-type?post=2813"},{"taxonomy":"msr-podcast-series","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-podcast-series?post=2813"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}