As Microsoft Research Asia – Tokyo‘s first research intern, Lilika Makabe (opens in new tab) brought her computer vision expertise into embodied AI and robotics, exploring new research questions through hands-on experimentation and interdisciplinary collaboration.
A young lab, and one of its first research interns
When Lilika Makabe joined Microsoft Research Asia – Tokyo, she was stepping into a young and evolving lab, where the research community and directions were still taking shape. As one of its first research interns, Makabe became part of a community shaping how computer vision, robotics, and AI that acts in the physical world could come together. Her experience shows how a research internship can become more than a short stop between university projects. It can be a chance for a researcher to cross into a new field, contribute an established area of expertise, and help build the research community around them.
For Makabe, joining the Tokyo lab was also an opportunity to explore a question that had increasingly captured her interest: how could her background in computer vision contribute to robots that perceive and act in the physical world?
Asked to describe the Tokyo lab in three words, Lilika chose: “Diligent, Curious, and Ambitious.”
Starting from computer vision
Makabe’s background is in computer vision. Before joining Microsoft Research, she worked on digitizing real-world objects from photographs, recovering both three-dimensional shape and surface material so their appearance could be reproduced faithfully in a digital environment, including under different lighting conditions. She also noticed a shift in the computer vision community. Robotics research was appearing more frequently at vision conferences, and vision researchers were exploring how their expertise could help robots understand and operate in the physical world. Lilika wanted to explore that connection herself. Most of her previous work had been in vision and alongside researchers working on related topics.
Joining the Tokyo lab offered an opportunity to enter a new field and research environment, and to learn directly from researchers with robotics expertise. She arrived expecting to apply her computer vision expertise to robot perception, helping robots make sense of their surroundings. The opportunity to work directly on vision-language-action models took that interest further than she had expected. During the internship, she experienced a connected research process: studying existing work, conducting experiments with real robots, collecting data through teleoperation, and using that data in vision-language-action research. This allowed her to see how perception, language, data, and action connect in a physical system. A robot continuously observes the current situation, selects an action, and then observes the result. Makabe’s computer vision background helped her reason about what perception can realistically provide, which aspects are difficult, and what forms of visual information a model can use.
“I study how robots can turn visual observations and language instructions into actions quickly, while preserving the diversity of valid actions robots can take to complete a task in the physical world,” Makabe said.
Makabe’s computer vision perspective also brought greater attention within the team to visual factors that can easily be overlooked when designing robot experiments, including shadows, reflections, and surface albedo.
“Through discussions with Lilika, I learned a lot and gained a deeper understanding of how these factors influence robot perception and what we should consider when designing experiments. Her expertise brought a valuable perspective to our research, helping us recognize that handling variations in visual appearance is essential for developing robot models that can generalize across diverse real-world environments.”
— Namiko Saito, Senior Researcher, Microsoft Research Asia – Tokyo
What changes when research meets a real robot
Working with physical systems introduced challenges that are easy to underestimate in simulation. The same method may behave differently because the environment has changed or because physical effects are not fully represented in a simulated setting. Data also must be collected through real operation, often with human demonstrations or intervention, before it can be used in learning and evaluation. Makabe recalled one moment in particular: after working through the gap between simulation and the real environment with researchers in the lab, the robot finally behaved as the team had hoped. “Seeing the robot finally working with our method was exciting!”
The team’s experience also highlighted a broader challenge in sim-to-real research: understanding why a method behaves differently in the physical world can be as important as improving the method itself.
“Even small discrepancies in visual observations, robot calibration, or contact dynamics can lead to unexpected behavior. This taught us the importance of identifying the source of failures rather than simply collecting more data or retraining the model. Working with Lilika brought together perspectives from computer vision and robotics, helping us better understand these challenges and develop more reliable robotic systems.”
— Namiko Saito, Senior Researcher, Microsoft Research Asia – Tokyo
The experience broadened Lilika’s research perspective from asking how a system can reconstruct and understand the world to asking how intelligence can use visual observations and language instructions to generate effective actions in that world.
Research grows through dialogue
The transition from computer vision to robotics was not a solitary one. Makabe worked with researchers in the Tokyo lab whose expertise and experiences differed from her own. Technical discussions helped her understand unfamiliar concepts, refine questions, and connect her existing knowledge to new problems. The collaboration also showed how expertise from one field can reshape research in another. When reflections and other visual properties affected the quality of Gaussian Splatting-based object models used to transfer real-world objects into simulation, Makabe drew on her computer vision background to diagnose the problem and clarify the approach’s practical capabilities and limitations.
“[Makabe’s] theoretical and systematic approach brought a complementary perspective to our robotics research, which was often driven by experimentation, iteration, and engineering. Her ability to bridge computer vision and robotics was a particularly valuable contribution to the team and enabled productive interdisciplinary research.”
— Heecheol Kim, Senior Researcher, Microsoft Research Asia – Tokyo
She also found that some of the most transferable research skills were not specific to any one field: expressing an idea clearly enough that someone outside the topic can follow it, and reasoning through a problem step by step. Those conversations extended beyond formal research meetings. Everyday exchanges with researchers and fellow interns, including discussions about different cultures and educational backgrounds, became part of the value of the experience.
Growing alongside Microsoft Research Asia – Tokyo
Makabe’s research journey unfolded alongside the growth of the Tokyo lab itself. The lab’s work on embodied AI and physical intelligence connects perception, learning, reasoning, and action to the realities of physical systems. In this environment, interns are not separate from the research community. They join discussions, bring technical depth from their own fields, learn from complementary expertise, and help test ideas against real research problems. As one of the lab’s early research interns, Makabe was not joining a fully established community. Her experience developed alongside the lab’s growing research directions and expanding connections among researchers and interns.
For Makabe, entering a field in which she did not begin as a robotics specialist became a source of confidence rather than a limitation. She found that a different background could itself offer a useful perspective, and she began thinking more broadly about where she could contribute, including problems in robotics. A valuable internship was therefore not only about access to computing resources, tools, or hardware. It was also about having people with whom she could think through ideas, ask questions, and connect different research traditions.
For the next research interns
Makabe’s advice to future intern applicants is to make full use of the environment around them, especially the people and conversations that may sit outside their immediate area of expertise. “I would encourage students to make time for those conversations. They can stay with you long after the internship. And if you are interested but unsure about your chances, I hope you will give yourself the opportunity to apply,” Makabe said.
As the Tokyo lab continues to grow, its research community will continue to be shaped by people who cross boundaries between disciplines. Makabe’s journey shows what that can look like: a computer vision researcher expanding into embodied AI and physical intelligence, while contributing knowledge, questions, and new perspectives to a growing lab.
Story contributors: Mawo Kamakura, Jane Ma, Shuyi Zhang, Lindsay Kalter, Amanda Melfi Black, Brenda Potts, Amber Tingle
Microsoft establishes a new lab, Microsoft Research Asia – Tokyo
This strategic expansion marks a significant step in the company’s mission to advance science and technology for the benefit of humanity.
The Tokyo lab focuses on critical areas that reflect Japan’s socioeconomic priorities, including embodied AI, well-being and neuroscience, societal AI, and industry innovation.
Yasuyuki Matsushita rejoins Microsoft to lead the new Microsoft Research Asia – Tokyo lab. Learn more about his journey and his perspective on the Tokyo lab’s role in evolution of AI.
Microsoft Research and Nissan Motor Corporation have collaborated to develop a machine learning model that improves the accuracy of predicting EV battery degradation by 80%. Learn how this collaboration supports long-term sustainability goals.
