{"id":1189333,"date":"2026-10-14T07:34:44","date_gmt":"2026-10-14T14:34:44","guid":{"rendered":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/?post_type=msr-project&p=1189333"},"modified":"2026-10-14T07:49:38","modified_gmt":"2026-10-14T14:49:38","slug":"debug-gym-a-text-based-environment-for-interactive-debugging","status":"publish","type":"msr-project","link":"https:\/\/cm-edgetun.pages.dev\/en-us\/research\/project\/debug-gym-a-text-based-environment-for-interactive-debugging\/","title":{"rendered":"debug-gym: A Text-Based Environment for Interactive Debugging"},"content":{"rendered":"
\n\t
\n\t\t
\n\t\t\t\t\t<\/div>\n\t\t\n\t\t
\n\t\t\t\n\t\t\t
\n\t\t\t\t\n\t\t\t\t
\n\t\t\t\t\t\n\t\t\t\t\t
\n\t\t\t\t\t\t
\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\t\t<\/span>\n\t\t\t\t\t\t\t\t\tMicrosoft Research Montr\u00e9al\t\t\t\t\t\t\t\t<\/a>\n\t\t\t\t\t\t\t\n\t\t\t\t\t\t\t\n\n

debug-gym<\/h1>\n\n\n\n

A Text-Based Environment for Interactive Debugging<\/p>\n\n\t\t\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t<\/div>\n\t\t<\/div>\n\t<\/div>\n<\/section>\n\n\n\n\n\n

\"debug-gym<\/figure>\n\n\n\n
\n
Paper<\/a><\/div>\n\n\n\n
Code<\/a><\/div>\n<\/div>\n\n\n\n

In most existing approaches (shown in black<\/strong>), an agent rewrites its code conditioned on error messages obtained from executing its code. debug-gym<\/code> equips the agent with additional tools such as pdb<\/code> (shown in red<\/strong>), so it can interactively seek necessary information from the semantic space hidden behind the code, and therefore have better code repairing performance.<\/p>\n\n\n\n

Abstract<\/h2>\n\n\n\n

Large Language Models (LLMs) are increasingly relied upon for coding tasks, yet in most scenarios it is assumed that all relevant information can be either accessed in context or matches their training data. We posit that LLMs may benefit from the ability to interactively explore a codebase to gather the information relevant to their task. To achieve this, we present a textual environment, namely debug-gym<\/code>, that can be used to develop LLM agents in an interactive coding setting, bridging the gap between current LLM capabilities and large-scale real-world code generation and debugging requirements. Our environment is lightweight and provides a preset of useful tools, such as a python debugger (pdb<\/code>), designed to facilitate a LLM-based agent\u2019s interactive debugging. Beyond coding and debugging tasks, this approach can be generalized to other tasks that would benefit from information-seeking behavior by an LLM agent.<\/p>\n\n\n\n

Video Demo<\/h2>\n\n\n\n
\"Debug-Gym<\/figure>\n\n\n\n
<\/div>\n\n\n\n

Experiments: Mini Nightmare<\/h2>\n\n\n\n

Mini-nightmare is a collection of ten minimal buggy Python scripts that human developers would find easier to debug if provided with interactive debugging tools. For instance, the shopping_cart<\/code> problem, which consists of a simple code snippet that aims to implement a shopping cart class. This specific task was created in Python 2, where the round() function adopts an \u201caway from zero\u201d style. However, in Python 3, where we run the debug-gym experiments in, the behavior has been changed to a \u201cties to even\u201d style. We observe smart debugging traces in trajectories from the agent using Claude 3.7 Sonnet below, showing its use of the pdb tool to test its hypothesis and then import an external library to make sure the rounding is working properly.<\/p>\n\n\n\n

\"Mini<\/figure>\n\n\n\n
<\/div>\n\n\n\n\n
\n\t\t
\n\t\t
\n\t\t\t