![\"image](\"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-content\/uploads\/2020\/02\/MSR_Podcast_DebadeeptaDey_Site_1400x788-1024x576.png\")
<\/h3>\n<\/h3>\nDr. Debadeepta Dey<\/a> is a Principal Researcher in the Adaptive Systems and Interaction group<\/a> at MSR and he\u2019s currently exploring several lines of research that may help bridge the gap between perception and planning for autonomous agents, teaching them to make decisions under uncertainty and even to stop and ask for directions when they get lost!<\/p>\n On today\u2019s podcast, Dr. Dey talks about how his latest work in meta-reasoning helps improve modular system pipelines and how imitation learning hits the ML sweet spot between supervised and reinforcement learning. He also explains how neural architecture search helps enlighten the \u201cdark arts\u201d of neural network training and reveals how boredom, an old robot and several \u201cbook runs\u201d between India and the US led to a rewarding career in research.<\/p>\n Debadeepta\u00a0Dey:\u00a0We said\u00a0like, you know what? Agents should just train themselves on when to ask during training time. Like when they make mistakes, they should just ask and learn to use their budget of asking questions back to the human at training time itself.\u00a0When you are in the simulation environments\u00a0we\u00a0used imitation learning as opposed to reinforcement learning.\u00a0Because you are in simulation, you have this nice programmatic expert. An expert need not be just a human being, right? Or a human teacher. It can also be an algorithm.<\/p>\n Host:\u00a0<\/b>You\u2019re listening to the Microsoft Research Podcast, a show that brings you closer to the cutting-edge of technology research and the scientists behind it. I\u2019m your host, Gretchen Huizinga.<\/b><\/p>\n Host: Dr.\u00a0<\/b>Debadeepta<\/b>\u00a0Dey is a Principal Researcher in the Adaptive Systems and Interaction group at MSR and he\u2019s currently exploring several lines of research that may help bridge the gap between perception and planning for autonomous agents, teaching them to make decisions under uncertainty<\/b>,<\/b>\u00a0and even to stop and ask for directions when they get lost!<\/b><\/p>\n On today\u2019s podcast, Dr. Dey talks about how his latest work in meta-reasoning helps improve modular system pipelines<\/b>,<\/b>\u00a0and how imitation learning hits the ML sweet spot between supervised and reinforcement learning. He also explains how neural architecture search<\/b>\u00a0<\/b>helps enlighten the\u00a0<\/b>\u201c<\/b>dark arts<\/b>\u201d<\/b>\u00a0of neural network training and reveals how boredom, an old robot and several \u201cbook runs\u201d between India and the US led to a rewarding career in research.\u00a0<\/b>That and much more on this episode of the Microsoft Research Podcast.<\/b><\/p>\n (music plays)<\/i><\/b><\/p>\n Host:\u00a0<\/b>Debadeepta<\/b>\u00a0Dey, welcome to the podcast<\/b>!<\/b><\/p>\n Debadeepta\u00a0Dey: Thank you.<\/p>\n Host: It\u2019s really great to have you here. I talked to one of your colleagues early on because I loved your name. You have one of the most lyrical names on the planet, I think.<\/b><\/p>\n Debadeepta\u00a0Dey: Thank you.<\/p>\n Host: And he said, we call him 3D.<\/b><\/p>\n Debadeepta\u00a0Dey: That\u2019s right. That\u2019s right, yeah!<\/p>\n Host: And then you got your PhD and they said, now we have to call him 4D!<\/b><\/p>\n Debadeepta\u00a0Dey: That\u2019s right. Oh, yes. Yes, so the joke amongst my friends is like,\u00a0well,\u00a0I became a dad,\u00a0so that\u2019s 5D, but they\u2019re like well, we\u2019ll have to wait until you become,\u00a0like twenty, thirty years,\u00a0if you became the director of some institute,\u00a0that will be a sixth D and whatnot. Whereas\u00a0like, the Ds are getting harder to accumulate.<\/p>\n Host:\u00a0<\/b>Right!\u00a0<\/b>And it\u2019s also like the phones\u2026 3G, 4G, 5G.<\/b><\/p>\n Debadeepta\u00a0Dey: Exactly.<\/p>\n Host: When does it end? Well I\u2019m so glad you\u2019re here. You\u2019re a principal researcher in the Adaptive Systems and Interaction, or ASI group, at Microsoft Research and you situate your work at the intersection of robotics and machine learning, yeah?<\/b><\/p>\n Debadeepta\u00a0Dey: That\u2019s right.<\/p>\n Host: So before I go deep on you, I\u2019d like you to situate the work of your group. What\u2019s the big goal of the Adaptive Systems team and what do you hope to accomplish as a group\u00a0<\/b>\u2013\u00a0<\/b>or collectively?<\/b><\/p>\n Debadeepta\u00a0Dey: ASI is one of the earliest groups at MSR right? Like, you know, because it was founded by Eric and if you dig into the history of how MSR groups have been, many groups have spun off from ASI, right? So ASI is more, I would say, instead of a thematic group, it\u2019s more like a family. ASI is a different group than most groups because it has people who have very diverse interests, but there\u2019s some certain common themes which tie the group together and I would say it is decision-making under uncertainty. There\u2019s people doing work on interpretability for machine learning, there\u2019s people doing work on human-robot interaction, social robotics, there\u2019s people doing work in reinforcement learning, planning, decision-making under uncertainty, but all of these things have in common is like you have to do decision-making under bounded constraints. What do we know? How do we get agents to be adaptive? How do we endow agents, be it robots or\u00a0virtual\u00a0agents,\u00a0with the ability to know what they don\u2019t know and act how we would expect intelligent beings to act.<\/p>\n Host: All right, well let\u2019s zoom in a little bit and talk about you<\/b>,<\/b>\u00a0and what gets you up in the morning. What\u2019s your big goal, as a scientist, and if I\u00a0<\/b>could<\/b>\u00a0put a finer point on it, what do you want to be known for at the end of your career?<\/b><\/p>\n Debadeepta\u00a0Dey: You know, I was thinking about it yesterday and one of the things I think which leaped out to me is like, you know, I want to be known for fundamental contributions to decision theory. And by that I don\u2019t mean just coming up with new theory, but also principles of how to apply them, principles of how to practice good decision science in the world.<\/p>\n Host: Well, let\u2019s talk about your work,\u00a0<\/b>Debadeepta<\/b>. Our big arena here is machine learning and on the podcast I\u2019ve had many of your colleagues who\u2019ve talked about the different kinds of machine learning in their work and each flavor has its own unique strengths and weaknesses, but you\u2019re doing some really interesting work in an area of ML that you call\u00a0<\/b>l<\/b>earning from\u00a0<\/b>d<\/b>emonstration<\/b>,<\/b>\u00a0and more specifically,\u00a0<\/b>i<\/b>mitation learning. So I\u2019d like you to unpack those terms for us and tell us how they\u2019re different from the other methods and what they\u2019re good for and why we need them?<\/b><\/p>\n Debadeepta\u00a0Dey: First of all, the big chunk of\u00a0machine\u00a0learning that we well understand today is supervised learning, right? You get a data set of labeled data and then you train some,\u00a0basically,\u00a0a curve-fitting algorithm, right? Like, you are fitting a function approximator to say that if you get new data samples,\u00a0as long as they are under that same distribution that produce the training data,\u00a0you should be able to predict what their label should be.<\/p>\n Host: Right.<\/b><\/p>\n Debadeepta\u00a0Dey:\u00a0Right?\u00a0And same\u00a0holds\u00a0even for\u00a0regression tasks. So supervised learning theory and practice is very well understood.\u00a0I think the challenge\u00a0that\u00a0the world has been focusing\u2026 or has a renewed focus on in the last five, ten years has been reinforcement learning, right? And reinforcement learning algorithms try to explore from scratch, right? You are doing learning tabula\u00a0rasa,\u00a0you assume that the agent just was born and now has to interact with the world and acquire knowledge. Imitation learning is more middle-ground, where it says hey, I\u2019m going to learn a policy,\u00a0or a good way of acting in the world,\u00a0based on what experts are showing me, right?<\/p>\n Host: Okay.<\/b><\/p>\n Debadeepta\u00a0Dey: And the reason this is powerful is because you can bootstrap learning. It\u2019s assuming\u00a0more things,\u00a0that you need access to an expert,\u00a0or teacher, but if the teacher is available,\u00a0and is good, then you can very quickly learn a policy which will do reasonable things.<\/p>\n Host: Okay.<\/b><\/p>\n Debadeepta\u00a0Dey:\u00a0Because all you need to do is mimic the teacher.<\/p>\n Host: So that\u2019s the learning from demonstration? The teacher demonstrates to the agent<\/b>,<\/b>\u00a0and then the agent learns from that and it\u2019s somewhere between just having this data poured down from the heavens, and knowing nothing.<\/b><\/p>\n Debadeepta\u00a0Dey: And knowing nothing, right?<\/p>\n Host: Okay.<\/b><\/p>\n Debadeepta\u00a0Dey: And mostly,\u00a0in the world, especially in domains like robotics, you don\u2019t want your robot to learn from nothing.<\/p>\n Host: Right.<\/b><\/p>\n Debadeepta\u00a0Dey:\u00a0Like, you know, to begin tabula rasa,\u00a0because now you have this random policy that you would start with, right? Because in the beginning you\u2019re just going to try things at random, right?<\/p>\n Host: Right.<\/b><\/p>\n Debadeepta\u00a0Dey: And robots are expensive. Robots can hurt people,\u00a0and also the amount of data needed is immense, right? Like the sample complexity, even\u00a0theoretically,\u00a0of reinforcement learning algorithms is really high and so it means that it will be a long, long time before you do interesting things.<\/p>\n Host:\u00a0<\/b>Right.\u00a0<\/b>Well, I want to talk a bit about automation. You\u2019ve done some\u00a0<\/b>interesting exploration in what you call neural architecture search or NAS, we\u2019ll\u00a0<\/b>call it for short. What is NAS, what\u2019s the motivation for it<\/b>,<\/b>\u00a0and how is it impacting other areas in the machine learning world?<\/b><\/p>\n Debadeepta\u00a0Dey: So NAS is this sub-field of this other sub-field in machine learning colloquially called Auto ML right now, right? Like where Auto ML\u2019s aim is to let algorithms search for the right algorithm for a given data set. Let\u2019s say this is a vision data set or an NLP data set. And it\u2019s labeled, right? So let\u2019s assume in the simpler setting instead of RL.\u00a0And\u00a0you are going to like, okay, I\u2019m going to like\u00a0you know\u00a0try my favorite algorithms that I have in this tool kit, but you are not really sure, is this the best algorithm? Is this the best way to pre-process data? Whatnot, right?\u00a0So the question then becomes, what is the right architecture, right?\u00a0And what are the right hyper-parameters for that architecture? What\u2019s the learning rate schedule? These are all things which are,\u00a0um,\u00a0we call it the\u00a0\u201cdark arts\u201d\u00a0of training and finding a good neural network for,\u00a0let\u2019s say,\u00a0a new data set, right? So this is more art than science, right?\u00a0And,\u00a0as a field, that\u2019s very unsatisfying.\u00a0Like, it\u2019s all great, the progress that deep learning has\u00a0made\u00a0is fantastic. Everybody is very excited, but there\u2019s this dark art part, which is there, and people are like well, you just need to build up a lot of practitioner intuition once you get there, right? And this is an answer which is deeply unsatisfying to the community as a whole, right? Like we refuse to accept this as\u00a0status quo.<\/p>\n Host: Well, when you\u2019re telling a scientist that it\u2019s art and you can\u2019t codify it<\/b>\u2026<\/b><\/p>\n Debadeepta\u00a0Dey: Yes.<\/p>\n Host:\u00a0<\/b>\u2026t<\/b>hat\u2019s just terrible.<\/b><\/p>\n Debadeepta\u00a0Dey: That\u2019s just terrible and it also shows that like, you know, we have given up,\u00a0or we have like lost the battle here so\u2026\u00a0and our understanding of deep learning is so shallow that we don\u2019t know how to codify things.<\/p>\n Host:\u00a0<\/b>All r<\/b>ight, so you\u2019re working on that with NAS, yeah?<\/b><\/p>\n Debadeepta\u00a0Dey: Yes,\u00a0so the goal in neural architecture search is,\u00a0let algorithms search for architectures. Let\u2019s remove the human from this tedious\u00a0\u201cdark arts\u201d\u00a0world of trying to figure things out from experience.\u00a0And it\u2019s also very expensive, right, like, you know, most companies and organizations cannot afford armies of PhDs just sitting around trying things and it\u2019s also not a very good usage of your best scientists\u2019 time, right?\u00a0And we want this,\u00a0ideally,\u00a0that you bring\u00a0a\u00a0data set, let the machine figure out what it should run,\u00a0and spit back out the model.<\/p>\n Host: Right. Well, the first time we met,\u00a0<\/b>Debadeepta<\/b>, you were on a panel talking about how researchers were using ML to troubleshoot and improve real time systems on<\/b>–<\/b>the<\/b>–<\/b>fly\u2026<\/b><\/p>\n Debadeepta\u00a0Dey: Yeah.<\/p>\n Host: \u2026and you published a paper just recently on the concept of meta-reasoning to monitor and adjust software modules on<\/b>–<\/b>the<\/b>–<\/b>fly using reinforcement learning to optimize the pipeline.<\/b><\/p>\n Debadeepta\u00a0Dey: Yeah.<\/p>\n Host: This is fascinating and I really loved how you framed the trade-offs for modular software and its impact on other parts of the systems, right?<\/b><\/p>\n Debadeepta\u00a0Dey: Right.<\/p>\n Host: So I\u2019d like you to kind of give us a review of what the trade-offs are in modular software systems in general and then tell us why you believe meta-reasoning is critical to improving those pipelines.<\/b><\/p>\n Debadeepta\u00a0Dey: So this project,\u00a0so\u00a0just a little bit of fun background, like actually started because of a discussion with the Platform for Situated Interaction team, and Dan\u00a0Bohus,\u00a0who\u2019s in\u00a0the\u00a0ASI group and like, you know, sits a few doors down from me, right?<\/p>\n Host: Yeah.<\/b><\/p>\n Debadeepta\u00a0Dey:\u00a0And so the problem statement actually comes from Dan and Eric.\u00a0I immediately jumped on the problem because I believed reinforcement learning, contextual bandits, provide feasible lines of attack right now.<\/p>\n Host: So why don\u2019t you articulate the problem<\/b>\u2026<\/b><\/p>\n Debadeepta\u00a0Dey: Okay.<\/p>\n Host: \u2026<\/b>writ large, for us.<\/b><\/p>\n Debadeepta\u00a0Dey: Okay. So let me give you this nice example, which will be easy to follow. Imagine you are a self-driving car team,\u00a0right?\u00a0And you are the software team, right?<\/p>\n Host: Yeah.<\/b><\/p>\n Debadeepta\u00a0Dey: And the software team is divided into many sub-teams, which are building many components of the self-driving car software.\u00a0Right?\u00a0Let\u2019s say somebody is writing the planner, somebody is writing low-level motor controllers, somebody is writing vision system, perception system, and then there is parts of the team where everybody\u2019s integrating all these pieces together\u00a0and\u00a0the end application runs, right? And this is a phenomenon which software teams, not just in robotics, but\u00a0also\u00a0like\u00a0if you\u2019re developing web software or whatnot, you find this all the time. Let\u2019s say you have a team which is developing the computer vision software that detects rocks and if there are rocks,\u00a0it will just say that these parts near the robot right now are rocks. Don\u2019t drive over them.\u00a0And in the beginning, they have\u00a0some machine learned\u00a0model where they collected some data and that model is,\u00a0let\u2019s say,\u00a0sixty, seventy percent\u00a0accurate. It\u2019s not super nice, but they don\u2019t want to hold up the rest of the team, so they push the first version of the module out so that there is no bottleneck, right? And so while they push this out, on the side they\u2019re trying to improve it, right? Because clearly\u00a0sixty, seventy percent\u00a0is not good enough, but that\u2019s okay. Like, you know, we will improve it. Three months go by, they do lots of hard work and say now we have a\u00a0ninety nine percent\u00a0good rock detector, right? So rest of the team,\u00a0you don\u2019t need to do anything. Just pull our latest code.\u00a0Nothing\u00a0will\u00a0change for you.\u00a0You will just get an update and everything should work great, right? So everybody goes and does that,\u00a0and the entire robot just starts breaking down, right? And here you have done three months of super-hard work to improve rock detection\u00a0to\u00a0close to\u00a0a hundred percent\u00a0and the robot is just horrible, right? And then all the teams get together is like, what happened? What happened is,\u00a0because the previous rock detector was only like\u00a0sixty, seventy percent\u00a0accurate,\u00a0the parameters of downstream modules had been adjusted to account for that. They\u2019re like\u00a0oh, we are not going to trust the rock detector most of the time. We are actually going to like, you know, be very conservative. These kinds of decisions have been made downstream, which actually have been dependent upon the quality of the results coming out upstream in order to make the whole system behave reasonably. But now that the quality of this module has drastically shifted, even though it is better, the net system actually has not become globally better. It has become globally worse.\u00a0<\/b><\/p>\n Host: Right.<\/b><\/p>\n Debadeepta\u00a0Dey:\u00a0And this is a phenomenon that large software teams see all the time. This is just a canonical example which is easy to explain, like,\u00a0you know, if you imagine anything from\u00a0like\u00a0Windows software or anything\u00a0else.<\/p>\n Host: Any system<\/b>\u2026<\/b><\/p>\n Debadeepta\u00a0Dey:\u00a0Mmm-hmm.<\/p>\n Host:<\/b>\u00a0<\/b>\u2026<\/b>that has multiple parts.<\/b><\/p>\n Debadeepta\u00a0Dey: Yeah.\u00a0So improving one part doesn\u2019t mean the whole system becomes better.<\/p>\n Host: In fact, it may make it worse.<\/b><\/p>\n Debadeepta\u00a0Dey: In fact, it may make it worse.<\/p>\n Host: Right.<\/b><\/p>\n Debadeepta\u00a0Dey:\u00a0Just like in NAS,\u00a0how we are like, you know,\u00a0using algorithms to search for algorithms, this is another kind of Auto ML,\u00a0where we are saying, hey, we want the machine learned monitor to check the entire pipeline and see what I should do to react to changing conditions, right?<\/p>\n Host: Okay.<\/b><\/p>\n Debadeepta\u00a0Dey:\u00a0So the machine\u2026 this monitor is looking at system-specific details like CPU usage, memory usage, the\u00a0run-time taken by each compute,\u00a0like it\u2019s monitoring everything. The entire pipeline,\u00a0as well as the hardware on which it is running and its conditions, right?<\/p>\n Host: Right.<\/b><\/p>\n Debadeepta\u00a0Dey: And it is learning policies to change the configuration of the entire pipeline on-the-fly to try to do the best it can as the environment changes.<\/p>\n Host:\u00a0<\/b>As the modules change, get better, and impact the whole system.\u00a0<\/b>How\u2019s it working?<\/b><\/p>\n Debadeepta\u00a0Dey: We have\u00a0found\u00a0really good promises, right? And right now we are looking for bigger and bigger pipelines to prove this out on and see where we can showcase this even better than we have already have in the research paper.<\/p>\n Host: Real briefly, tell me about the paper that you just published and what\u2019s going on with that in the meta-reasoning for these pipelines.<\/b><\/p>\n Debadeepta\u00a0Dey: So that paper is at AAAI. It will come out in February, actually at New York next week and there we showed that you can use techniques like contextual bandits as well as stateful reinforcement learning to safely change the configurations of entire pipelines all at once, right?<\/p>\n Host: Wow.<\/b><\/p>\n Debadeepta\u00a0Dey: And let them not degrade very drastically to\u00a0adversarial changes and conditions, right?<\/p>\n Host: You know, just as a side note, my husband had knee replacement surgery.<\/b><\/p>\n Debadeepta\u00a0Dey: Okay.<\/p>\n Host: But for decades he had had a compressed knee because he blew it out playing football<\/b>\u2026<\/b><\/p>\n Debadeepta\u00a0Dey: Okay.<\/p>\n Host: \u2026<\/b>and he had no cartilage.<\/b><\/p>\n Debadeepta\u00a0Dey: I see.<\/p>\n Host: So his body was totally used to working in a particular way.<\/b><\/p>\n Debadeepta\u00a0Dey: Yeah.<\/p>\n Host: When they did the knee surgery, he gained an inch in that leg. Suddenly he has back problems.<\/b><\/p>\n Debadeepta\u00a0Dey: Yeah, because now your back has to, like, you know\u2026\u00a0it\u2019s the entire configuration, right? You can\u2019t just\u2026<\/p>\n Host: No, and it\u2019s true of basically every system, including the human body<\/b>, is,\u00a0<\/b>you push down here it comes out there<\/b>!<\/b><\/p>\n Debadeepta\u00a0Dey: No, that\u2019s true. Cars, like people go and put oh, I\u2019m going to go and put a big tire on my car and then the entire performance of the car is degraded because the suspension is not adapted.<\/p>\n Host: But it\u2019s a cool tire.<\/b><\/p>\n Debadeepta\u00a0Dey: Yeah, it\u2019s a cool tire, the steering is now rock hard and unwieldy and but the tire looks good though.<\/p>\n (music plays)<\/i><\/b><\/p>\n Host: Well, let\u2019s talk a little bit more about robots,\u00a0<\/b>Debadeepta<\/b>, since that\u2019s your roots.<\/b><\/p>\n Debadeepta\u00a0Dey: Yes.<\/p>\n Host: So, most of us are familiar with digital assistan<\/b>ts<\/b>\u00a0like Cortana and Siri and Alexa and some of us even have physical robots like Roomba to do menial tasks like vacuuming, but you\u2019d like us to be able to interact with physical robots via natural language and not only train them to do a broad<\/b>er<\/b>\u00a0variety of tasks for us, but also to ask us for help when they need it<\/b>!<\/b><\/p>\n Debadeepta\u00a0Dey: Yeah.<\/p>\n Host: So tell us about the work that you\u2019re doing here. I know that there\u2019s some really interesting threads of research happening.<\/b><\/p>\n Debadeepta\u00a0Dey: This project actually, the one that you\u2019re referring to, actually started with a hallway conversation with Bill Dolan, who runs the NLP group, after an AI seminar on a Tuesday where we just got talking, right?\u00a0Because of my previous experience with robotics and also\u00a0AirSim, which is a simulation system with Ashish and Shital and Chris Lovett. And we found that, hey,\u00a0simulation is starting to play a big role and the community sees that, right? And already like, you know, for home robotics, not just outdoor\u2026<\/p>\n Host: Sure.<\/b><\/p>\n Debadeepta\u00a0Dey: \u2026things that fly and drive\u00a0all\u00a0by themselves and whatnot,\u00a0people are building rich simulators, right,\u00a0and every day we are getting better and better data sets, very rich data sets of real people\u2019s homes scanned and put into\u00a0AirSim-like environments with Unreal engine as the backend,\u00a0or Unity as the backend\u2026\u00a0which, game engines have become so good, right?\u00a0Like,\u00a0I can\u2019t believe how good game engines are at rendering photorealistic scenes, and we saw this opportunity that hey, maybe we can train agents to not just react reasonably to people\u2019s commands and language instructions in indoor scenarios, but also like,\u00a0ask for help\u2026\u00a0because one of the things we saw was that,\u00a0at the time, we had dismal performance on even the best algorithms. Very complicated algorithms were doing terrible, like\u00a0six percent\u00a0accuracy on doing any task provided by our language, right?\u00a0<\/b>But just like any human being, right,\u00a0like, you know, imagine you ask your family member to hey, can you help me? Can you get me this, right?<\/p>\n Host: Yeah.<\/b><\/p>\n Debadeepta\u00a0Dey:\u00a0Um, while I am working on this,\u00a0can you just go upstairs and get me this? They may not know exactly what you are talking about,\u00a0or they may go upstairs and be like, I don\u2019t know. I don\u2019t see it there. Where else should I look? Human beings ask for help. They know when they have an awareness that, hey, we are lost or I\u2019m being inefficient. I should just ask the domain expert.<\/p>\n Host: Ask for directions.<\/b><\/p>\n Debadeepta\u00a0Dey: Exactly. Ask for directions,\u00a0and especially when we feel that we have become uncertain and are getting lost, right?<\/p>\n Host: Sure.<\/b><\/p>\n Debadeepta\u00a0Dey: So\u00a0that scenario,\u00a0we should have our agents doing that as well, right? So let\u2019s see if we give a budgeted number of tries to an agent,\u00a0and this is almost like,\u00a0if you have seen those\u00a0game shows where you get to call a friend?<\/p>\n Host: Yeah, a lifeline.<\/b><\/p>\n Debadeepta\u00a0Dey: A lifeline, exactly, right? Like, you know.\u00a0Um, you\u2026 and let\u2019s say you have three lifelines, right? And so you have to be strategic about how you play those lifelines\u2026<\/p>\n Host: Don\u2019t call me\u2026<\/b><\/p>\n Debadeepta\u00a0Dey: Or at least don\u2019t use them up on easy questions.<\/p>\n Host: Right.<\/b><\/p>\n Debadeepta\u00a0Dey: Right? Like, you know, something like that. But also there\u2019s this trade-off like hey, if you mess up early in the beginning and you didn\u2019t use the lifeline when you should have, you will be out of the game, right? So you won\u2019t live in the game long enough, right?<\/p>\n Host: Yeah.<\/b><\/p>\n Debadeepta\u00a0Dey: So there\u2019s this strategy.\u00a0So we said, you know what? Agents should just train themselves on when to ask during training time. Like when they make mistakes, they should just ask and learn to use their budget of asking questions back to the human at training time itself, right? When you are in the simulation environments\u00a0we\u00a0used imitation learning as opposed to reinforcement learning,\u00a0and we were just talking about imitation before,\u00a0because you are in simulation, you have this nice programmatic expert. An expert need not be just a human being, right? Or a human teacher. It can also be an algorithm which has access to lots more information at training time\u2026\u00a0you would not have that information at test time, but if,\u00a0at training time,\u00a0you have that information,\u00a0you try to,\u00a0like,\u00a0mimic what that expert would do, right? And in\u00a0simulation,\u00a0you can just run a planning algorithm, which is just like shortest path algorithm, and learn to mimic what the shortest path algorithm would do at test time,\u00a0even though now you don\u2019t have the underlying information to run the planning algorithm. And with that,\u00a0we also like built in the ability for the agent to become self-aware like,\u00a0\u201cI\u2019m very uncertain right now. I should ask for help,\u201d and it greatly improved performance, right?<\/p>\n Host: Yeah<\/b>, yeah, yeah<\/b>.<\/b><\/p>\n Debadeepta\u00a0Dey: Of course, we are asking for more information,\u00a0strategically,\u00a0so I don\u2019t think it\u2019s a fair comparison to just compare it to the agent which doesn\u2019t get to ask.<\/p>\n Host: Right.<\/b><\/p>\n Debadeepta\u00a0Dey: But we showed that,\u00a0like, you know, instead of randomly asking,\u00a0or asking only at the beginning or\u00a0at\u00a0the end,\u00a0at\u00a0various normal baselines that you would think of,\u00a0learning how to ask gives you a huge boost.<\/p>\n Host: Well,\u00a0<\/b>Debadeepta<\/b>, this is the part of the podcast where I always ask my guests what could possibly go wrong? And when we\u2019re talking about robots and autonomous systems and automated machine learning<\/b>,<\/b>\u00a0the answer is, in general, a lot!<\/b><\/p>\n Debadeepta\u00a0Dey: Yeah.<\/p>\n Host: That\u2019s why you\u2019re doing this work.<\/b><\/p>\n Debadeepta\u00a0Dey: Right.<\/p>\n Host: So since the stakes are high in these arenas, I want to know what you\u2019re thinking about, specifically. What keeps you up at night and, more importantly, what are you doing about it to help us all get a better night\u2019s sleep?<\/b><\/p>\n Debadeepta\u00a0Dey: So in robotics\u00a0and\u00a0self-driving cars, drones, even for home robotics, like safety is very critical, right? Like,\u00a0you know,\u00a0you are running robots around humans, close to humans, in the open world,\u00a0and not just in factories, which\u00a0have\u00a0cordoned-off\u00a0spaces, right? So robots can be isolated from humans pretty reasonably, but not inside homes and on the road, right?<\/p>\n Host: Or in the sky.<\/b><\/p>\n Debadeepta\u00a0Dey: Or in the sky, absolutely. The good thing is,\u00a0the regulations bodies are pretty aware of this.\u00a0And even the community as a whole realizes that you can\u2019t just go and\u00a0field\u00a0a robot with any not-well-tested machine learning algorithms or decision-making running, right? So there\u2019s huge research\u00a0efforts right now\u00a0on how to do safe reinforcement learning. I\u2019m not personally involved a lot in safe reinforcement learning, but I work closely with, for example, the\u00a0reinforcement learning group in Redmond, the reinforcement learning group in New York City,\u00a0and there\u2019s huge efforts even within MSR on doing safe reinforcement learning, safe decision-making, safe control\u2026 I sleep better knowing that these efforts are going on and there\u2019s also huge efforts, for example, in ASI\u00a0and\u00a0people working on model interpretability\u2026<\/p>\n Host: Right.<\/b><\/p>\n Debadeepta\u00a0Dey: People working on pipeline de-bugging,\u00a0and ethics and fairness,\u00a0including\u00a0at\u00a0other parts of MSR and Microsoft, and the community in general, so I feel like people are hyper-aware. The community is hyper-aware. Everybody is also very worried that we will get an AI winter if we over-promise and under-deliver again, so we need to make our contributions be very realistic and not just over-hype all the buzzes going around.\u00a0The things\u00a0that I\u2019m looking forward to do is\u00a0like,\u00a0for example, like meta-reasoning. We were thinking about\u00a0like\u00a0how to do safe meta-reasoning, right? Just the fact that the system knows that it\u2019s not very aware and I should not be taking decisions blindly. These are beginning steps. Without doing that, you won\u2019t be able to make decisions\u00a0which will\u00a0evade dangerous situations. You first have to know that,\u00a0I\u2019m in a dangerous spot because I am doing decisions without knowing what I am doing, right? And that\u2019s like the first key step and even there,\u00a0we are a ways away.<\/p>\n Host: Right. Well, interestingly, you talk about Microsoft and Microsoft Research and I know Brad Smith\u2019s book\u00a0<\/b>Tools and Weapons<\/i><\/b>\u00a0addresses some of these big questions in that weird space between regulated and unregulated, especially when we\u2019re talking about AI and machine learning<\/b>,\u00a0<\/b>but there\u2019s other actors out there that have access to \u2013 and brains for \u2013 this kind of technology that might use it for more nefarious purposes or might not just even follow best practices. So how is the community thinking about that? You\u2019re making these tools that are incredibly powerful, um\u2026<\/b><\/p>\n Debadeepta\u00a0Dey: Yeah, so that is a big debate right now in the research community because often times what happens is that,\u00a0we want to attract more VC funding, we want to grow bigger, it\u2019s land grabs, so everybody\u00a0wants to show that they have better technology,\u00a0and racing to production or deployment.<\/p>\n Host: First to deploy<\/b>\u2026<\/b><\/p>\n Debadeepta\u00a0Dey: First to deploy, right? And\u00a0then\u00a0first to convince others, even if it\u2019s not completely ready, means that you maybe get,\u00a0like, you know, the biggest share of the pie, right?\u00a0It is,\u00a0indeed,\u00a0very concerning, right? Like,\u00a0even without robotics,\u00a0right,\u00a0even if you have\u00a0like\u00a0services, machine learning services and whatnot, right?<\/p>\n Host: Right.<\/b><\/p>\n Debadeepta\u00a0Dey: And what do we do about things which are beyond our control, right?\u00a0We can write tooling to verify any model which is out there and do interpretability,\u00a0find where the model\u00a0has blind spots\u2026\u00a0That we can provide, right?\u00a0Personally, what I always want to do is be the anti-hype person. I remember there was this tweet at current\u00a0NeurIPS\u00a0where Lin Xiao, who won the Test of Time award, which is\u00a0a\u00a0very hard award to win, for his paper almost twelve years ago, started his talk saying,\u00a0oh, this is just a minor extension of\u00a0Nesterov\u2019s\u00a0famous theorem,\u00a0right, like you know\u2026\u00a0And\u00a0Subbarao\u00a0Kambhampati\u00a0tweeted\u00a0that, hey, in this world where everybody has pretty much invented,\u00a0or is about to invent,\u00a0AGI, so refreshing to see somebody say, oh, this is just a minor extension\u00a0of\u2026!<\/p>\n Host: It\u2019s an iteration.<\/b><\/p>\n Debadeepta\u00a0Dey: Yeah. And most work is that, right?<\/p>\n Host: Yeah.<\/b><\/p>\n Debadeepta\u00a0Dey:\u00a0Like, irrespective of the fancy articles you see, or\u00a0in\u00a0PopSci\u00a0magazines, robots are not taking over the world right now. There\u2019s lots of problems to be solved, right?<\/p>\n Host: All right, well, I want to know a little more about you,\u00a0<\/b>Debadeepta<\/b>, and I bet our listeners do too. So tell us about your journey, mostly professionally, but where did you start? What got a young\u00a0<\/b>Debadeepta<\/b>\u00a0Dey interested in computer science and robotics and how did you end up here at Microsoft Research?<\/b><\/p>\n Debadeepta\u00a0Dey: Okay, well, I\u2019ll try to keep it short.\u00a0But the story begins in undergrad in engineering college in New Delhi. The Indian system for getting into engineering school is that it is a very tough, all-India entrance exam and then, depending\u00a0upon the rank you get, you either get in or you don\u2019t,\u00a0to good places, right? And that\u2019s pretty much it. It\u2019s that four-hour or six-hour exam and how you do on it matters. And that is so tough that you prepare a lot for that. And often what happens is,\u00a0after\u00a0you get to college, the first year is really boring, okay?\u00a0Because, I remember, because we knew everything that was already in the curriculum\u00a0in\u00a0the first two years\u00a0of college\u2026<\/p>\n Host: Just to get in.<\/b><\/p>\n Debadeepta\u00a0Dey: Yeah, just to get in,\u00a0and so you\u2019re like, okay, we have nothing to do. And so I remember the first summer after the first year of college, we were just,\u00a0a bunch of us friends were just bored,\u00a0so we were like,\u00a0we need to do something, man, because we are going out of our mind. And we were like, hey, how about we do robotics? That seems cool. Okay, first of all, none of us knew anything about robotics, right? But this is like young people\u00a0hubris,\u00a0right like\u00a0you know\u2026<\/p>\n Host: You don\u2019t know what you don\u2019t know.<\/b><\/p>\n Debadeepta\u00a0Dey:\u00a0\u2026yeah, like confidence of the young. I guess that\u2019s needed at some point.<\/p>\n Host: Yeah.<\/b><\/p>\n Debadeepta\u00a0Dey:\u00a0You should not get jaded too early in life, so we were like,\u00a0okay, we are going to do robotics and we are going to build a robot and we are going to take part in this competition in the US in two, three years\u2019 time, but we need to\u00a0just\u00a0learn everything about robotics, right? And, okay, you must understand this is like, you know, pre-\u2026 internet was there, but the kind of online course material you have now, especially in India, we didn\u2019t have anything. There was nobody to teach robotics and this was a top school, right?\u00a0And there was\u00a0like\u00a0one dusty robot in the basement of\u00a0some,\u00a0I think the mechanical engineering department, which had not been used in like ten years. Nobody even knew where the software was and everything. Like,\u00a0we went and found some old, dusty book on robotics\u2026 But luckily what happened is, because we were in Delhi, somebody had returned from CMU.\u00a0Anuj\u00a0Kapuria\u00a0had started this company called Hi-Tech Robotics. So we kind of got a meeting with him and we just started doing unpaid internships there, right?\u00a0We were like, we don\u2019t care\u2026 we don\u2019t\u2026\u00a0Because he actually knew what robotics was, right?<\/p>\n Host: Right.<\/b><\/p>\n Debadeepta\u00a0Dey: Because he had come in right from CMU and finishing his master\u2019s and he was starting this company. He would sometimes go to the US and it was so dire that we would like, will you buy this book for us and bring it back from the US, right? Because there\u2019s nobody here\u2026 We can\u2019t even find that book, right?<\/p>\n Host: Right.<\/b><\/p>\n Debadeepta\u00a0Dey: And so I got my like first taste of modern-day robotics and research there and then,\u00a0in undergrad,\u00a0after the end of my third year,\u00a0I did an internship at the Field Robotics Center at Carnegie Melon. And then after that, I finished my master\u2019s and PhD there. I came back to India, finished and then\u00a0went\u00a0back to the US,\u00a0and that\u2019s how I got started mostly because I think it was, I would say, pure perseverance. I\u2019m well-aware I\u2019m not the smartest person in the room, but,\u00a0as somebody had told me right before I started at Intel Research\u00a0and\u00a0who is now at Google, finishing a PhD is\u00a0ninety nine percent\u00a0perseverance. And research is, as almost all big things in life, it\u2019s all perseverance. You just got to stick at it,\u00a0right, and\u00a0through the ups and the downs.\u00a0And lucky enough, I also had fantastic advisors. CMU was a wonderful place. When I came to MSR it also re-energized me in the middle of my PhD.<\/p>\n Host: Would it be fair to say you\u2019re not bored anymore?<\/b><\/p>\n Debadeepta\u00a0Dey:\u00a0Um no, no! Not at all! Like you know, nowadays, we have the opposite problem! We are like\u2026<\/p>\n Host: Too much.<\/b><\/p>\n Debadeepta\u00a0Dey: Too many cool problems to work on and yeah, not enough time, yeah.<\/p>\n Host: Tell us something we don\u2019t know about you. I often ask this question in terms of how a particular character trait or defining moment led to a career in research, but I\u2019m down for an anecdote even if it doesn\u2019t relate to that.<\/b><\/p>\n Debadeepta\u00a0Dey: So my mother is a history professor in India and,\u00a0growing up with her,\u00a0I was reading a lot\u00a0like\u00a0because she would bring me all kinds of books. Not just history,\u00a0like\u00a0literature and everything, and I was very good at English literature and I wanted always to be an English professor. I never wanted to do anything with CS. In fact, I was actually kind of bad at math. I remember I flunked basic calculus in grade 11, right? Mostly because of not paying attention and whatnot, but all of that was very boring and the way math was predominately taught at the time was in this very imperialistic manner. Here\u2019s a set of rules, go do this set of rules and keep applying them over and over. And I was like, why? This all seems very punitive, right? But, my mother one day sat me down and said, look, you\u2019re a good student, here\u2019s the economic realities, at least in India. I am one in\u00a0a\u00a0thousand who makes a living from the humanities, most people don\u2019t,\u00a0and will not make it,\u00a0and it\u2019s very difficult to get, actually, a living wage out of being an English professor, at least in India. And you are good at science and engineering. Do something there. At least you will make enough money to pay your bills, right? But there\u2019s always this part of me which believes\u00a0that\u00a0if there was a parallel life, if only I\u00a0can\u00a0be an English professor at a small, rural college somewhere, that would work out great as well!<\/p>\n Host: As we close, I want to frame my last question in terms of one of your big research interests and you started off with it: decision-making under uncertainty.<\/b><\/p>\n Debadeepta\u00a0Dey: Yeah.<\/p>\n Host: Many of our listeners are at the beginning of their career decision-trees, but\u00a0<\/b>absent\u00a0<\/b>what we might call big data for life choices<\/b>, t<\/b>hey\u2019re trying to make optimal decisions as to their future\u00a0<\/b>in<\/b>\u00a0high tech research. So what would you say to them? I\u2019ll give you the last word.<\/b><\/p>\n Debadeepta\u00a0Dey: The one thing I have found, no matter what you choose, be\u00a0it\u00a0technology,\u00a0arts\u2026 and this is\u00a0particularly true\u00a0for becoming good at\u00a0what you do,\u00a0is pay attention to the fundamentals, right? Like I have never seen a great researcher who doesn\u2019t have mastery over the fundamentals, right? This is just like going to the gym. You are not going to go bench press four hundred pounds the first day you go to the gym. That\u2019s just not going to happen, right? So a lot of people are like well, I\u2019m in this Calculus 101. It seems boring and whatnot and I don\u2019t know why I\u2019m doing this, but all of that stuff, especially if you are going to be in a tech career, math is super useful. Just try to become very, very good at fundamentals. The rest kind of takes care of itself. And wherever you are, irrespective of the prestige of\u00a0your\u00a0university, even that doesn\u2019t matter. One of the principals that we have found true, especially for recruiting purposes,\u00a0is, always pick the candidate who has really strong fundamentals because it doesn\u2019t matter what the rest of the CV says, really good fundamentals\u2026 we will make something good out of that. So if you just focus on that, wherever you are in the world, you will be good!<\/p>\n Host:\u00a0<\/b>Debadeepta<\/b>\u00a0Dey, this has been so much fun. Thanks for coming on the podcast and sharing all these great stories and your great work.<\/b><\/p>\n Debadeepta\u00a0Dey: Thank you. I had a lot of fun as well!<\/p>\n (music plays)<\/i><\/b><\/p>\n To learn more about Dr.\u00a0<\/i><\/b>Debadeepta<\/i><\/b>\u00a0Dey and how researchers are helping your robot make good decisions, visit Microsoft.com\/research<\/i><\/b><\/p>\n","protected":false},"excerpt":{"rendered":" Dr. Debadeepta Dey is a Principal Researcher in the Adaptive Systems and Interaction group at MSR and he\u2019s currently exploring several lines of research that may help bridge the gap between perception and planning for autonomous agents, teaching them to make decisions under uncertainty and even to stop and ask for directions when they get lost! On the podcast, Dr. Dey talks about how his latest work in meta-reasoning helps improve modular system pipelines and how imitation learning hits the ML sweet spot between supervised and reinforcement learning. He also explains how neural architecture search helps enlighten the \u201cdark arts\u201d of neural network training and reveals how boredom, an old robot and several \u201cbook runs\u201d between India and the US led to a rewarding career in research.<\/p>\n","protected":false},"author":37583,"featured_media":638877,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"msr-url-field":"https:\/\/player.blubrry.com\/id\/56493545\/","msr-podcast-episode":"108","msrModifiedDate":"","msrModifiedDateEnabled":false,"ep_exclude_from_search":false,"_classifai_error":"","msr-author-ordering":[],"msr_hide_image_in_river":0,"footnotes":""},"categories":[240054],"tags":[],"research-area":[13561,13556],"msr-region":[],"msr-event-type":[],"msr-locale":[268875],"msr-post-option":[],"msr-impact-theme":[],"msr-promo-type":[],"msr-podcast-series":[],"class_list":["post-638862","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-msr-podcast","msr-research-area-algorithms","msr-research-area-artificial-intelligence","msr-locale-en_us"],"msr_event_details":{"start":"","end":"","location":""},"podcast_url":"https:\/\/player.blubrry.com\/id\/56493545\/","podcast_episode":"108","msr_research_lab":[199565],"msr_impact_theme":[],"related-publications":[],"related-downloads":[],"related-videos":[],"related-academic-programs":[],"related-groups":[144633,395930],"related-projects":[389792,359810],"related-events":[],"related-researchers":[],"msr_type":"Post","featured_image_thumbnail":"Related:<\/h3>\n
\n
\nTranscript<\/h3>\n
![\"image](\"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-content\/uploads\/2020\/02\/MSR_Podcast_DebadeeptaDey_Site_1400x788-960x540.png\")
","byline":"","formattedDate":"February 26, 2020","formattedExcerpt":"Dr. Debadeepta Dey is a Principal Researcher in the Adaptive Systems and Interaction group at MSR and he\u2019s currently exploring several lines of research that may help bridge the gap between perception and planning for autonomous agents, teaching them to make decisions under uncertainty and…","locale":{"slug":"en_us","name":"English","native":"","english":"English"},"_links":{"self":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/posts\/638862","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/users\/37583"}],"replies":[{"embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/comments?post=638862"}],"version-history":[{"count":9,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/posts\/638862\/revisions"}],"predecessor-version":[{"id":668139,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/posts\/638862\/revisions\/668139"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/media\/638877"}],"wp:attachment":[{"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/media?parent=638862"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/categories?post=638862"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/tags?post=638862"},{"taxonomy":"msr-research-area","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/research-area?post=638862"},{"taxonomy":"msr-region","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-region?post=638862"},{"taxonomy":"msr-event-type","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-event-type?post=638862"},{"taxonomy":"msr-locale","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-locale?post=638862"},{"taxonomy":"msr-post-option","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-post-option?post=638862"},{"taxonomy":"msr-impact-theme","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-impact-theme?post=638862"},{"taxonomy":"msr-promo-type","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-promo-type?post=638862"},{"taxonomy":"msr-podcast-series","embeddable":true,"href":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/wp-json\/wp\/v2\/msr-podcast-series?post=638862"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}