{"id":616143,"date":"2019-10-21T05:01:02","date_gmt":"2019-10-21T12:01:02","guid":{"rendered":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/?p=616143"},"modified":"2019-10-21T05:23:49","modified_gmt":"2019-10-21T12:23:49","slug":"a-new-era-of-spatial-computing-brings-fresh-challenges-and-solutions-to-vr","status":"publish","type":"post","link":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/blog\/a-new-era-of-spatial-computing-brings-fresh-challenges-and-solutions-to-vr\/","title":{"rendered":"A new era of spatial computing brings\u00a0fresh\u00a0challenges\u2014and solutions\u2014to VR"},"content":{"rendered":"

\"\"<\/a><\/p>\n

Virtual reality\u00a0(VR)\u00a0has\u00a0continually\u00a0pushed\u00a0the boundaries of how we perceive, from its early days of Ivan Sutherland\u2019s\u00a0Sword of Damocles\u00a0to today. With the technology emerging from\u00a0its early stages of bulky equipment tethered to one place out of necessity, researchers now are working with increased possibilities derived from hardware and\u00a0new input sensors.\u00a0The result is a unique set of challenges requiring innovative approaches to answering some of the most prevalent questions as far as what VR is capable of, where VR can happen, and how humans are able to experience it.<\/p>\n

There have been major advances recently in creating new, immersive environments that enable anything from\u00a0exploring new places<\/a> outside the limitations of a single room to increasing accessibility<\/a> for all, and Microsoft researchers continue to make technologies that expand and enhance the world of\u00a0spatial computing. This week,\u00a0they\u2019re presenting their\u00a0latest and fanciest in New Orleans at the\u00a0ACM Symposium on User Interface Software and Technology (UIST) 2019 (opens in new tab)<\/span><\/a>.<\/p>\n

Researchers will be showing\u00a0off\u00a0a\u00a0method for allowing people to safely\u00a0navigate\u00a0a given route\u00a0in real-world environments, such as a daily walk to work, while seeing themselves strolling a different VR world, such as a city of their choosing.\u00a0They\u2019ve also developed a\u00a0technology that leverages newly introduced\u00a0eye-tracking of users so that visual details in VR can be changed in real time without those changes being detected. Finally, researchers have invented a new haptic feedback controller that uniquely emulates the human sense of touch by meshing centuries-old and cutting-edge technologies.<\/p>\n

\"DreamWalker

DreamWalker is\u00a0a\u00a0VR system that allows users to walk from one place to another in the real world\u00a0(A)\u00a0while experiencing a completely different place, such as\u00a0a crowded city,\u00a0in VR\u00a0(B).\u00a0The technology uses path planning (C)\u00a0to identify\u00a0a VR path that best aligns with the real-world path,\u00a0accounting for any\u00a0known obstacles\u00a0in the real-world with corrections in the VR world.\u00a0Obstacles discovered en route are managed by\u00a0real-time\u00a0sensing technologies.<\/p><\/div>\n

DreamWalker: From your door to workplace by way of Manhattan<\/h3>\n

What if you could explore the streets of a faraway place you\u2019ve always wanted to go while walking to work? What if stepping into a place you\u2019ve always wanted to see was as simple as stepping out your front door? In \u201cDreamWalker: Substituting Real-World Walking Experiences with a Virtual Reality<\/a>,\u201d\u00a0Stanford University PhD student Jackie Yang (opens in new tab)<\/span><\/a>, a\u00a0Microsoft\u00a0Research\u00a0intern\u00a0at the time of the work, and Microsoft researchers\u00a0Eyal\u00a0Ofek<\/a>, Andy Wilson<\/a>, and Christian\u00a0Holz (opens in new tab)<\/span><\/a>, who is now an ETH Z\u00fcrich professor,\u00a0have created a VR system that allows you to walk from one place to another in the real world while experiencing a completely different place in VR.\u00a0The system adapts to a chosen route, making it possible to transform a walk to the grocery store or to a bus stop, say, into a walk through Times Square in VR.<\/p>\n

A precursor project,\u00a0VRoamer<\/a>, enabled playing a VR game inside an uncontrolled indoor environment. DreamWalker takes\u00a0the technology another step, merging the constraints of the virtual city tour and the actual path the user will travel in the real world, which may differ, while handling the complex difficulties of an uncontrolled outdoor<\/i> environment. To accomplish this task, the technology first plans users\u2019 paths in the virtual world and then uses real-time environment detection, walking redirection, and virtual world updates once users have embarked on their journeys.<\/p>\n

Prior to the start of the walk, path planning finds a path in the virtual world that will minimize differences with the real-world path. It will also identify any potential corrections needed in the virtual world so that the user doesn\u2019t collide with any known objects in the real world. Differences between the real path and the chosen virtual tour will be corrected by gradual redirection while users walk and also by\u00a0introducing\u00a0virtual scenarios that will force users to correct their own paths,\u00a0such as road blocks.<\/p>\n

During the walk, DreamWalker monitors users\u2019 environments using a variety of sensing technology\u2014\u00a0inside-out tracking via a Windows Mixed Reality-provided relative position trace, a dual band GPS sensor, and two RGB depth cameras\u2014to detect obstacles discovered along the way.<\/p>\n

Discovered\u00a0obstacles that may move or appear in users\u2019 paths are managed by introducing moving virtual obstacles, or characters, such as pedestrians walking near users, to block them from any potential danger.\u00a0Other options\u00a0for controlling users\u2019 paths\u00a0may include pets and dynamic events such as vehicles\u00a0being parked, moving carts and more, limited only by the imagination of the experience creator.\u00a0The\u00a0real-time environment detection\u00a0technology\u00a0provides\u00a0guidance\u00a0for the creation of this additional virtual content with\u00a0great care taken to generate these virtual obstacles outside users\u2019 field of view to avoid any unnatural \u201cpopping up\u201d of objects in the virtual world.\u00a0Thanks to recent advances, the technology can actually\u00a0do so with the help of users\u2019 gaze. Enter\u00a0Mise-Unseen.<\/p>\n

\"Mise-Unseen<\/a>

Mise-Unseen uses eye-tracking in VR headsets to hide changes that occur inside the user\u2019s field of view by applying them\u00a0when\u00a0the user\u2019s\u00a0attention is\u00a0diverted\u00a0elsewhere. In this example, it unnoticeably changes the scene as the user is focusing\u00a0on\u00a0a movement from\u00a0the cat, facing pieces together to help the user solve this puzzle.<\/p><\/div>\n

Mise-Unseen: Taking advantage of users\u2019 gaze and attention to enhance the VR experience<\/h3>\n

In \u201cMise-Unseen: Using Eye-Tracking to Hide Virtual Reality Scene Changes in Plain Sight<\/a>,\u201d University of Potsdam PhD student Sebastian\u00a0Marwecki (opens in new tab)<\/span><\/a>, a Microsoft Research intern at the time of the work, and researchers Andy Wilson<\/a>, Eyal\u00a0Ofek<\/a>, Mar Gonzalez Franco<\/a>, and Christian\u00a0Holz (opens in new tab)<\/span><\/a> introduce technology that allows covert changes to be made to a VR environment while a user\u2019s gaze and attention are fixed on something else.\u00a0They call this technology Mise-Unseen in homage to the French term\u00a0mise en sc\u00e8ne<\/i>,\u00a0which\u00a0is used\u00a0in film and theater\u00a0to\u00a0describe the scenery and staging of what an audience sees. In film and theater, the field of view is controlled by a director, so the audience sees what the director wants them to see. The audience cannot see the camera or lighting riggings, for instance, and that is by design.<\/p>\n

In VR, however, users are controlling their own field of view, so some changes to the scene need to be done discreetly within their peripheral vision. Mise-Unseen uses eye-tracking to identify where users are focusing their attention and then employs a perception model to recommend where changes to the environment can be made without users noticing those changes. For example, a painting in a virtual art gallery can be changed to better reflect users\u2019 tastes while they\u2019re looking elsewhere in the room. \u201cThis [technology] is all based on the idea that gaze and attention are linked; wherever you are gazing goes your attention, and that means you are not paying attention to other parts,\u201d says Senior Researcher\u00a0Mar\u00a0Gonzalez Franco<\/a>.<\/p>\n

Another application is the ability to assess user understanding. While playing a game, a person might need to solve a puzzle to obtain a code that allows them to move forward. However, there is a chance that the person could guess the code without having solved, or even having looked at, the puzzle. With this technology, if a player hasn\u2019t looked at certain areas of the puzzle before entering the code, the code and puzzle could be visually changed in real time so that the player needs to go back and attempt to solve the puzzle.<\/p>\n

Other applications of Mise-Unseen include support for passive haptics, individualized content\u00a0(like the painting example above), and adaptive difficulty. Another exciting aspect of this technology is its ability to reduce the field of view in VR in a way that alleviates motion sickness for those who experience it. Lastly, this technology allows valuable computing resources to be conserved when rendering visual objects\u2014knowing where a user is looking allows the system to better disperse its resources.<\/p>\n

\"The\u00a0haptic<\/a>

The\u00a0haptic controller\u00a0CapstanCrunch\u00a0supports human-scale forces during touch and grasp and can resist the user\u2019s grasp force up to 20\u00a0newtons. The friction-based capstan brake mechanism magnifies the\u00a0force of the controller\u2019s\u00a0small motor, resulting in an integrated, palm-grounded controller design for interaction. The controller can\u00a0replicate\u00a0complex haptic events exhibiting variable stiffness and compliance.<\/p><\/div>\n

CapstanCrunch: A palm-grounded haptic controller\u00a0that, much like judo, enlists user-supplied force<\/h3>\n

Imagine grasping for a virtual ball. You reach out to grab the ball in the\u00a0VR\u00a0system, but how do you know you\u2019ve got it if you can\u2019t feel it against your hand? That\u2019s where haptic controllers come into play\u2014they\u00a0provide the sensation associated with touching an object\u00a0in real life when you come into contact with it in a virtual world.\u00a0A major challenge for such controllers, though, has been\u00a0producing\u00a0realistic feelings consistent with the strong forces applied by the human hand while keeping the handheld devices small and light.<\/p>\n

In their paper \u201cCapstanCrunch: A Haptic VR Controller with User-supplied Force Feedback<\/a>,\u201d researchers\u00a0Mike Sinclair<\/a>,\u00a0Eyal\u00a0Ofek<\/a>,\u00a0Mar Gonzalez\u00a0Franco<\/a>, and Christian\u00a0Holz (opens in new tab)<\/span><\/a>\u00a0introduce a new haptic controller called CapstanCrunch. While CapstanCrunch may appear outwardly similar to past prototypes such as the CLAW<\/a>, the magic of the new technology lies inside the controller:\u00a0a linear\u00a0and directional\u00a0brake that can sustain human-scale forces\u00a0and\u00a0is\u00a0light, energy efficient,\u00a0and robust.<\/p>\n

A\u00a0key component\u00a0of the\u00a0brake\u2014and the inspiration behind the controller\u2019s\u00a0name\u2014is\u00a0the capstan,\u00a0a centuries-old mechanical device originally used to control ropes on sailing ships (opens in new tab)<\/span><\/a>. \u201cIt\u2019s an old technology that\u00a0people used to tie boats,\u00a0but\u00a0we use it in a different way\u00a0that right now enables us to multiply the force of\u00a0a small internal motor,\u201d says Eyal\u00a0Ofek, a Senior Researcher<\/a>.<\/p>\n

Past\u00a0efforts to design haptic\u00a0controllers used large motors to resist the strong forces applied by the human hand, yet these expensive, heavy,\u00a0and energy-hungry motors can\u00a0break when too\u00a0great\u00a0a force is applied, making the device too expensive and not robust enough to be used in consumer scenarios. The capstan magnifies an input force by around a factor of 40, which allows\u00a0the use\u00a0of small, cheap,\u00a0and energy-efficient motors while insulating them from external world forces.<\/p>\n

CapstanCrunch\u00a0leverages this technology to give fine-grain haptic feedback. Similar to an important tenet in the martial arts practice judo, the\u00a0CapstanCrunch\u00a0mechanism uniquely feeds off users\u2019 own human force and friction to maximize efficiency and minimize effort needed from the motor to provide feedback. In addition to consuming low amounts of energy,\u00a0CapstanCrunch\u00a0is low cost, robust, safe, quiet, and fast when compared with its predecessors.<\/p>\n

Check out the work<\/h3>\n

If you\u2019re attending UIST 2019,\u00a0Mise-Unseen\u00a0will be presented\u00a0on Tuesday, Oct.\u00a022, at 2 p.m. Eastern\u00a0time\u00a0as will\u00a0CapstanCrunch.\u00a0DreamWalker\u00a0will be presented\u00a0at 2 p.m.\u00a0the next day.\u00a0The papers are among several from Microsoft researchers. For information on those works, visit\u00a0Microsoft at UIST 2019<\/a>.<\/p>\n","protected":false},"excerpt":{"rendered":"

Virtual reality\u00a0(VR)\u00a0has\u00a0continually\u00a0pushed\u00a0the boundaries of how we perceive, from\u00a0its\u00a0early\u00a0days\u00a0of\u00a0Ivan Sutherland\u2019s\u00a0Sword of Damocles\u00a0to\u00a0today. With the technology\u00a0emerging\u00a0from\u00a0its early stages of bulky equipment tethered to one place out of necessity, researchers now are working with increased possibilities derived from\u00a0hardware and\u00a0new input sensors.\u00a0The result is\u00a0a\u00a0unique\u00a0set of challenges\u00a0requiring\u00a0innovative approaches to\u00a0answering\u00a0some of the most prevalent questions as far as what VR is capable of, where VR can happen, and how humans are able to experience it.<\/p>\n","protected":false},"author":39507,"featured_media":616302,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"msr-url-field":"","msr-podcast-episode":"","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","msr-author-ordering":[],"msr_hide_image_in_river":0,"footnotes":""},"categories":[243747],"tags":[],"research-area":[13554],"msr-region":[],"msr-event-type":[],"msr-locale":[268875],"msr-post-option":[],"msr-impact-theme":[],"msr-promo-type":[],"msr-podcast-series":[],"class_list":["post-616143","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-human-computing-interaction","msr-research-area-human-computer-interaction","msr-locale-en_us"],"msr_event_details":{"start":"","end":"","location":""},"podcast_url":"","podcast_episode":"","msr_research_lab":[199565],"msr_impact_theme":[],"related-publications":[],"related-downloads":[],"related-videos":[],"related-academic-programs":[],"related-groups":[379814,396845],"related-projects":[617844,432870],"related-events":[613080],"related-researchers":[],"msr_type":"Post","featured_image_thumbnail":"\"Mixed","byline":"","formattedDate":"October 21, 2019","formattedExcerpt":"Virtual reality\u00a0(VR)\u00a0has\u00a0continually\u00a0pushed\u00a0the boundaries of how we perceive, from\u00a0its\u00a0early\u00a0days\u00a0of\u00a0Ivan Sutherland\u2019s\u00a0Sword of Damocles\u00a0to\u00a0today. With the technology\u00a0emerging\u00a0from\u00a0its early stages of bulky equipment tethered to one place out of necessity, researchers now are working with increased possibilities derived from\u00a0hardware and\u00a0new input sensors.\u00a0The result is\u00a0a\u00a0unique\u00a0set of challenges\u00a0requiring\u00a0innovative approaches to\u00a0answering\u00a0some of…","locale":{"slug":"en_us","name":"English","native":"","english":"English"},"_links":{"self":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/posts\/616143","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/users\/39507"}],"replies":[{"embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/comments?post=616143"}],"version-history":[{"count":23,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/posts\/616143\/revisions"}],"predecessor-version":[{"id":616524,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/posts\/616143\/revisions\/616524"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/media\/616302"}],"wp:attachment":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/media?parent=616143"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/categories?post=616143"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/tags?post=616143"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=616143"},{"taxonomy":"msr-region","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-region?post=616143"},{"taxonomy":"msr-event-type","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-event-type?post=616143"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=616143"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=616143"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=616143"},{"taxonomy":"msr-promo-type","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-promo-type?post=616143"},{"taxonomy":"msr-podcast-series","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-podcast-series?post=616143"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}