Alessandro Sordoni

Senior Principal Research Manager

Publications

View by:

- Orchard: An Open-Source Agentic Modeling Framework
  
  Baolin Peng, Wenlin Yao, Qianhui Wu, Hao Cheng, Xiao Yu, Ruiyi Yang, Tao Ge, Alessandro Sordoni, Xingdi Yuan, Yelong Shen, Pengcheng He, Tong Zhang, Zhou Yu, Jianfeng Gao
  
  May 2026
  
  arXiv | May 2026
  
  Publication
- Test-Time Learning with an Evolving Library
  
  Weijia Xu, Alessandro Sordoni, Chandan Singh, Zelalem Gero, Michel Galley, Xingdi Yuan, Jianfeng Gao
  
  May 2026
  
  arXiv | May 2026
  
  Publication
- Trade-offs in Ensembling, Merging and Routing Among Parameter-Efficient Experts
  
  Sanae Lotfi, Lucas Caccia, Alessandro Sordoni, Jordan Ash, Miroslav Dudík
  
  March 2026
  
  arXiv | March 2026
  
  Publication
- Effect of Document Packing on the Latent Multi-Hop Reasoning Capabilities of Large Language Models
  
  Gabriele Prato, Shagun Sodhani, Alessandro Sordoni, Sarath Chandar, Alessandro Sordoni
  
  December 2025
  
  arXiv | December 2025
  
  Publication
- Gistify! Codebase-Level Understanding via Runtime Execution
  
  Hyunji Lee, Minseon Kim, Chinmay Singh, Matheus Pereira, Atharv Sonwane, Isadora White, Elias Stengel-Eskin, Mohit Bansal, Zhengyan Shi, Alessandro Sordoni, Marc-Alexandre Côté, Xingdi Yuan, Lucas Caccia
  
  ICLR 2026 | October 2025
  
  DOI Publication
- The Markovian Thinker
  
  Milad Aghajohari, Kamran Chitsaz, Amirhossein Kazemnejad, Sarath Chandar, Alessandro Sordoni, Aaron C. Courville, Siva Reddy
  
  ICLR 2026 | October 2025
  
  Publication
- Learning to Solve Complex Problems via Dataset Decomposition
  
  Wanru Zhao, Lucas Caccia, Zhengyan Shi, Minseon Kim, Xingdi Yuan, Weijia Xu, Marc-Alexandre Côté, Alessandro Sordoni
  
  NeurIPS 2025 | July 2025
  
  Publication
- Exploring Sparse Adapters for Scalable Merging of Parameter Efficient Experts
  
  Samin Yeasar Arnob, Zhan Su, Minseon Kim, Oleksiy Ostapenko, Riyasat Ohib, Esra'a Saleh, Doina Precup, Lucas Caccia, Alessandro Sordoni
  
  COLM 2025 | July 2025
  
  Publication
- Putting the Value Back in RL: Better Test-Time Scaling by Unifying LLM Reasoners With Verifiers
  
  Kusha Sareen, Morgane M Moss, Alessandro Sordoni, Rishabh Agarwal, Arian Hosseini
  
  COLM 2025 | July 2025
  
  Publication
- Training Plug-and-Play Knowledge Modules with Deep Context Distillation
  
  Lucas Caccia, Alan Ansell, E. Ponti, Ivan Vuli'c, Alessandro Sordoni
  
  COLM 2025 | July 2025
  
  Publication
- debug-gym: A Text-Based Environment for Interactive Debugging
  
  Xingdi Yuan, Morgane M Moss, Charbel Feghali, Chinmay Singh, Darya Moldavskaya, Drew MacPhee, Lucas Caccia, Matheus Pereira, Minseon Kim, Alessandro Sordoni, Marc-Alexandre Côté
  
  March 2025
  
  ArXiv | March 2025
  
  Publication Access
- Energy-efficient Reinforcement Learning by Discovering Neural Pathways
  
  Alessandro Sordoni
  
  October 2024
- A Survey on Model MoErging: Recycling and Routing Among Specialized Experts for Collaborative Learning
  
  Prateek Yadav, Colin Raffel, Mohammed Muqeeth, Lucas Caccia, Haokun Liu, Tian-Xiang Chen, Mohit Bansal, Leshem Choshen, Alessandro Sordoni
  
  ArXiv | August 2024
  
  DOI Publication Publication
- Improving Context-Aware Preference Modeling for Language Models
  
  Silviu Pitis, Ziang Xiao, Nicolas Le Roux, Alessandro Sordoni
  
  NeurIPS 2024 | July 2024
  
  DOI Publication Publication
- Efficient Adversarial Training in LLMs with Continuous Attacks
  
  Sophie Xhonneux, Alessandro Sordoni, Stephan Günnemann, Gauthier Gidel, Leo Schwinn
  
  NeurIPS 2024 | May 2024
  
  DOI Publication Publication
- Towards Modular LLMs by Building and Reusing a Library of LoRAs
  
  O. Ostapenko, Zhan Su, E. Ponti, Laurent Charlin, Nicolas Le Roux, Matheus Pereira, Lucas Caccia, Alessandro Sordoni
  
  ICML 2024 | May 2024
  
  Publication
- Multi-Head Adapter Routing for Data-Efficient Fine-Tuning
  
  Lucas Caccia, E. Ponti, Lu Liu, Matheus Pereira, Nicolas Le Roux, Alessandro Sordoni
  
  ArXiv | November 2023
  
  DOI Publication Publication
- Guiding Language Model Reasoning with Planning Tokens
  
  Xinyi Wang, Lucas Caccia, O. Ostapenko, Xingdi Yuan, Alessandro Sordoni
  
  October 2023
  
  arXiv | October 2023
  
  DOI Publication
- Joint Prompt Optimization of Stacked LLMs using Variational Inference
  
  Alessandro Sordoni, Xingdi Yuan, Marc-Alexandre Côté, Matheus Pereira, Adam Trischler, Ziang Xiao, Arian Hosseini, Friederike Niedtner, Nicolas Le Roux
  
  NeurIPS 2023 | October 2023
  
  Publication
- Multi-Head Adapter Routing for Cross-Task Generalization
  
  Lucas Caccia, Edoardo Ponti, Zhan Su, Matheus Pereira, Nicolas Le Roux, Alessandro Sordoni
  
  NeurIPS 2023 | October 2023
  
  Publication
- Better Language Model with Hypernym Class Prediction
  
  He Bai, Tong Wang, Alessandro Sordoni, Peng Shi
  
  ACL 2022 | May 2022
  
  Publication
- Unsupervised Dependency Graph Network
  
  Yikang Shen, Shawn Tan, Alessandro Sordoni, Peng Li, Jie Zhou, Aaron Courville
  
  ACL 2022 | May 2022
  
  Publication
- Evaluating Distributional Distortion in Neural Language Modeling
  
  Benjamin LeBrun, Alessandro Sordoni, Timothy J. O'Donnell
  
  ICLR 2022 | April 2022
  
  Publication
- Learning to Dequantise with Truncated Flows
  
  Shawn Tan, Chin-Wei Huang, Alessandro Sordoni, Aaron Courville
  
  ICLR 2022 | April 2022
  
  Publication
- Self-training with Few-shot Rationalization: Teacher Explanations Aid Student in Few-shot NLU
  
  Meghana Moorthy Bhat, Alessandro Sordoni, Subhabrata (Subho) Mukherjee
  
  EMNLP 2021 | November 2021
  
  Publication Project Project
- The emergence of the shape bias results from communicative efficiency
  
  Eva Portelance, Michael C. Frank, Dan Jurafsky, Alessandro Sordoni, Romain Laroche
  
  Conference on Computational Natural Language Learning (CoNLL) | November 2021
  
  Publication
- Decomposed Mutual Information Estimation for Contrastive Representation Learning
  
  Alessandro Sordoni, Nouha Dziri, Hannes Schulz, Geoff Gordon, Philip Bachman, Remi Tachet des Combes
  
  2021 International Conference on Machine Learning | July 2021
  
  DOI
- Explicitly Modeling Syntax in Language Models with Incremental Parsing and a Dynamic Oracle
  
  Yikang Shen, Shawn Tan, Alessandro Sordoni, Siva Reddy, Aaron Courville
  
  May 2021
  
  arXiv | May 2021
  
  Publication
- Understanding by Understanding Not: Modeling Negation in Language Models
  
  Arian Hosseini, Siva Reddy, Dzmitry Bahdanau, Devon Hjelm, Alessandro Sordoni, Aaron Courville
  
  May 2021
  
  arXiv | May 2021
  
  PDF
- Linguistic dependencies and statistical dependence
  
  Jacob Louis Hoover, Alessandro Sordoni, Wenyu Du, Timothy J. O'Donnell
  
  Empirical Methods in Natural Language Processing | April 2021
  
  Publication
- Exploring and Predicting Transferability Across NLP Tasks
  
  Tu Vu, Tong Wang, Tsendsuren Munkhdalai, Alessandro Sordoni, Adam Trischler, Andrew Mattarella-Micke, Subhransu Maji, Mohit Iyyer
  
  May 2020
  
  ArXiv | May 2020
  
  Publication
- Fast Domain Adaptation For Goal-Oriented Dialogue Using A Hybrid Generative-Retrieval Transformer
  
  Igor Shalyminov, Alessandro Sordoni, Adam Atkinson, Hannes Schulz
  
  2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) | April 2020
  
  Publication Access Project
- Hybrid Generative-Retrieval Transformers For Dialogue Domain Adaptation
  
  Igor Shalyminov, Alessandro Sordoni, Adam Atkinson, Hannes Schulz
  
  arXiv preprint arXiv:2003.01680 | March 2020
  
  PDF Project
- Report on the First HIPstIR Workshop on the Future of Information Retrieval
  
  Laura Dietz, Bhaskar Mitra, Jeremy Pickens, Hana Anber, Sandeep Avula, Asia J. Biega, Adrian Boteanu, Shubham Chatterjee, Jeff Dalton, Shiri Dori-Hacohen, John Foley, Henry Feild, Ben Gamari, Rosie Jones, Pallika Kanani, Sumanta Kashyapi, Widad Machmouchi, Matthew Mitsui, Steve Nole, Alexandre Tachard Passos, Jordan Ramsdell, Adam Roegiest, David Smith, Alessandro Sordoni
  
  SIGIR Forum | December 2019, Vol 53(2): pp. 62-75
  
  Publication PDF
- Ordered Memory
  
  Yikang Shen, Shawn Tan, Arian Hosseini, Zhouhan Lin, Alessandro Sordoni, Aaron Courville
  
  NeurIPS 2019 | November 2019
  
  ArXiv
  
  Publication
- Robust Natural Language Inference Models with Example Forgetting
  
  Yadollah Yaghoobzadeh, Remi Tachet des Combes, T. J. Hazen, Alessandro Sordoni
  
  November 2019
  
  Arxiv | November 2019
  
  Publication
- Metalearned Neural Memory
  
  Tsendsuren Munkhdalai, Alessandro Sordoni, Tong Wang, Adam Trischler
  
  NeurIPS 2019 | July 2019
  
  Preprint under review
  
  Publication
- Brief Report: Ordered Neurons: Integrating Tree Structures into Recurrent Neural Networks
  
  Yikeng Shen, Shawn Tan, Alessandro Sordoni, Aaron Courville
  
  ICLR 2019 | May 2019
  
  ICLR 2019 Best Paper Award
  
  Publication
- An Empirical Study of Example Forgetting during Deep Neural Network Learning
  
  Mariya Toneva, Alessandro Sordoni, Remi Tachet des Combes, Adam Trischler, Yoshua Bengio, Geoff Gordon
  
  ICLR 2019 | April 2019
  
  Publication
- Towards Text Generation with Adversarially Learned Neural Outlines
  
  Sandeep Subramanian, Sai Rajeswar, Alessandro Sordoni, Adam Trischler, Aaron Courville, Christopher Pal
  
  NeurIPS 2018 | December 2018
  
  Publication Project
- Counting to Explore and Generalize in Text-based Games
  
  Xingdi Yuan, Marc-Alexandre Côté, Alessandro Sordoni, Romain Laroche, Remi Tachet des Combes, Matthew Hausknecht, Adam Trischler
  
  European Workshop on Reinforcement Learning (EWRL) | October 2018
  
  Publication Project
- Straight to the Tree: Constituency Parsing with Neural Syntactic Distance
  
  Yikang Shen, Zhouhan Lin, Athul Paul Jacob, Alessandro Sordoni, Aaron Courville, Yoshua Bengio
  
  Proceedings of ACL | July 2018
  
  PDF
- Counting to Explore and Generalize in Text-based Games
  
  Xingdi Yuan, Marc-Alexandre Côté, Alessandro Sordoni, Romain Laroche, Remi Tachet des Combes, Matthew Hausknecht, Adam Trischler
  
  ICML 2018 | June 2018
  
  Preprint Project
- Augmented cyclegan: Learning many-to-many mappings from unpaired data
  
  Amjad Almahairi, Sai Rajeswar, Alessandro Sordoni, Philip Bachman, Aaron Courville
  
  ICML 2018 | June 2018
  
  ArXiv
  
  Publication
- Focused Hierarchical RNNs for Conditional Sequence Processing
  
  Nan Rosemary Ke, Konrad Zolna, Alessandro Sordoni, Zhouhan Lin, Adam Trischler, Yoshua Bengio, Joelle Pineau, Laurent Charlin, Chris Pal
  
  ICML 2018 | June 2018
  
  Publication Project
- Twin Networks: Matching the Future for Sequence Generation
  
  Dmitriy Serdyuk, Nan Rosemary Ke, Alessandro Sordoni, Chris Pal, Adam Trischler, Yoshua Bengio
  
  ICLR 2018 | January 2018
  
  Publication Project
- Z-Forcing: Training Stochastic Recurrent Networks
  
  Anirudh Goyal, Alessandro Sordoni, Marc-Alexandre Côté, Nan Rosemary Ke, Yoshua Bengio
  
  31st Conference on Neural Information Processing Systems (NIPS 2017 - Advances in Neural Information Processing Systems) | November 2017
  
  PDF Preprint
- Learning Algorithms for Active Learning
  
  Philip Bachman, Alessandro Sordoni, Adam Trischler
  
  34th International Conference on Machine Learning (ICML 2017) | August 2017
  
  Publication

- Test-Time Learning with an Evolving Library
  
  Weijia Xu, Alessandro Sordoni, Chandan Singh, Zelalem Gero, Michel Galley, Xingdi Yuan, Jianfeng Gao
  
  May 2026
  
  arXiv | May 2026
  
  Publication
- Orchard: An Open-Source Agentic Modeling Framework
  
  Baolin Peng, Wenlin Yao, Qianhui Wu, Hao Cheng, Xiao Yu, Ruiyi Yang, Tao Ge, Alessandro Sordoni, Xingdi Yuan, Yelong Shen, Pengcheng He, Tong Zhang, Zhou Yu, Jianfeng Gao
  
  May 2026
  
  arXiv | May 2026
  
  Publication
- Trade-offs in Ensembling, Merging and Routing Among Parameter-Efficient Experts
  
  Sanae Lotfi, Lucas Caccia, Alessandro Sordoni, Jordan Ash, Miroslav Dudík
  
  March 2026
  
  arXiv | March 2026
  
  Publication
- Effect of Document Packing on the Latent Multi-Hop Reasoning Capabilities of Large Language Models
  
  Gabriele Prato, Shagun Sodhani, Alessandro Sordoni, Sarath Chandar, Alessandro Sordoni
  
  December 2025
  
  arXiv | December 2025
  
  Publication
- Gistify! Codebase-Level Understanding via Runtime Execution
  
  Hyunji Lee, Minseon Kim, Chinmay Singh, Matheus Pereira, Atharv Sonwane, Isadora White, Elias Stengel-Eskin, Mohit Bansal, Zhengyan Shi, Alessandro Sordoni, Marc-Alexandre Côté, Xingdi Yuan, Lucas Caccia
  
  ICLR 2026 | October 2025
  
  DOI Publication
- The Markovian Thinker
  
  Milad Aghajohari, Kamran Chitsaz, Amirhossein Kazemnejad, Sarath Chandar, Alessandro Sordoni, Aaron C. Courville, Siva Reddy
  
  ICLR 2026 | October 2025
  
  Publication
- Learning to Solve Complex Problems via Dataset Decomposition
  
  Wanru Zhao, Lucas Caccia, Zhengyan Shi, Minseon Kim, Xingdi Yuan, Weijia Xu, Marc-Alexandre Côté, Alessandro Sordoni
  
  NeurIPS 2025 | July 2025
  
  Publication
- Training Plug-and-Play Knowledge Modules with Deep Context Distillation
  
  Lucas Caccia, Alan Ansell, E. Ponti, Ivan Vuli'c, Alessandro Sordoni
  
  COLM 2025 | July 2025
  
  Publication
- Putting the Value Back in RL: Better Test-Time Scaling by Unifying LLM Reasoners With Verifiers
  
  Kusha Sareen, Morgane M Moss, Alessandro Sordoni, Rishabh Agarwal, Arian Hosseini
  
  COLM 2025 | July 2025
  
  Publication
- Exploring Sparse Adapters for Scalable Merging of Parameter Efficient Experts
  
  Samin Yeasar Arnob, Zhan Su, Minseon Kim, Oleksiy Ostapenko, Riyasat Ohib, Esra'a Saleh, Doina Precup, Lucas Caccia, Alessandro Sordoni
  
  COLM 2025 | July 2025
  
  Publication
- debug-gym: A Text-Based Environment for Interactive Debugging
  
  Xingdi Yuan, Morgane M Moss, Charbel Feghali, Chinmay Singh, Darya Moldavskaya, Drew MacPhee, Lucas Caccia, Matheus Pereira, Minseon Kim, Alessandro Sordoni, Marc-Alexandre Côté
  
  March 2025
  
  ArXiv | March 2025
  
  Publication Access
- Energy-efficient Reinforcement Learning by Discovering Neural Pathways
  
  Alessandro Sordoni
  
  October 2024
- A Survey on Model MoErging: Recycling and Routing Among Specialized Experts for Collaborative Learning
  
  Prateek Yadav, Colin Raffel, Mohammed Muqeeth, Lucas Caccia, Haokun Liu, Tian-Xiang Chen, Mohit Bansal, Leshem Choshen, Alessandro Sordoni
  
  ArXiv | August 2024
  
  DOI Publication Publication
- Improving Context-Aware Preference Modeling for Language Models
  
  Silviu Pitis, Ziang Xiao, Nicolas Le Roux, Alessandro Sordoni
  
  NeurIPS 2024 | July 2024
  
  DOI Publication Publication
- Efficient Adversarial Training in LLMs with Continuous Attacks
  
  Sophie Xhonneux, Alessandro Sordoni, Stephan Günnemann, Gauthier Gidel, Leo Schwinn
  
  NeurIPS 2024 | May 2024
  
  DOI Publication Publication
- Towards Modular LLMs by Building and Reusing a Library of LoRAs
  
  O. Ostapenko, Zhan Su, E. Ponti, Laurent Charlin, Nicolas Le Roux, Matheus Pereira, Lucas Caccia, Alessandro Sordoni
  
  ICML 2024 | May 2024
  
  Publication
- Multi-Head Adapter Routing for Data-Efficient Fine-Tuning
  
  Lucas Caccia, E. Ponti, Lu Liu, Matheus Pereira, Nicolas Le Roux, Alessandro Sordoni
  
  ArXiv | November 2023
  
  DOI Publication Publication
- Guiding Language Model Reasoning with Planning Tokens
  
  Xinyi Wang, Lucas Caccia, O. Ostapenko, Xingdi Yuan, Alessandro Sordoni
  
  October 2023
  
  arXiv | October 2023
  
  DOI Publication
- Multi-Head Adapter Routing for Cross-Task Generalization
  
  Lucas Caccia, Edoardo Ponti, Zhan Su, Matheus Pereira, Nicolas Le Roux, Alessandro Sordoni
  
  NeurIPS 2023 | October 2023
  
  Publication
- Joint Prompt Optimization of Stacked LLMs using Variational Inference
  
  Alessandro Sordoni, Xingdi Yuan, Marc-Alexandre Côté, Matheus Pereira, Adam Trischler, Ziang Xiao, Arian Hosseini, Friederike Niedtner, Nicolas Le Roux
  
  NeurIPS 2023 | October 2023
  
  Publication
- Learning to Dequantise with Truncated Flows
  
  Shawn Tan, Chin-Wei Huang, Alessandro Sordoni, Aaron Courville
  
  ICLR 2022 | April 2022
  
  Publication
- Evaluating Distributional Distortion in Neural Language Modeling
  
  Benjamin LeBrun, Alessandro Sordoni, Timothy J. O'Donnell
  
  ICLR 2022 | April 2022
  
  Publication
- The emergence of the shape bias results from communicative efficiency
  
  Eva Portelance, Michael C. Frank, Dan Jurafsky, Alessandro Sordoni, Romain Laroche
  
  Conference on Computational Natural Language Learning (CoNLL) | November 2021
  
  Publication
- Decomposed Mutual Information Estimation for Contrastive Representation Learning
  
  Alessandro Sordoni, Nouha Dziri, Hannes Schulz, Geoff Gordon, Philip Bachman, Remi Tachet des Combes
  
  2021 International Conference on Machine Learning | July 2021
  
  DOI
- Explicitly Modeling Syntax in Language Models with Incremental Parsing and a Dynamic Oracle
  
  Yikang Shen, Shawn Tan, Alessandro Sordoni, Siva Reddy, Aaron Courville
  
  May 2021
  
  arXiv | May 2021
  
  Publication
- Understanding by Understanding Not: Modeling Negation in Language Models
  
  Arian Hosseini, Siva Reddy, Dzmitry Bahdanau, Devon Hjelm, Alessandro Sordoni, Aaron Courville
  
  May 2021
  
  arXiv | May 2021
  
  PDF
- Hybrid Generative-Retrieval Transformers For Dialogue Domain Adaptation
  
  Igor Shalyminov, Alessandro Sordoni, Adam Atkinson, Hannes Schulz
  
  arXiv preprint arXiv:2003.01680 | March 2020
  
  PDF Project
- Report on the First HIPstIR Workshop on the Future of Information Retrieval
  
  Laura Dietz, Bhaskar Mitra, Jeremy Pickens, Hana Anber, Sandeep Avula, Asia J. Biega, Adrian Boteanu, Shubham Chatterjee, Jeff Dalton, Shiri Dori-Hacohen, John Foley, Henry Feild, Ben Gamari, Rosie Jones, Pallika Kanani, Sumanta Kashyapi, Widad Machmouchi, Matthew Mitsui, Steve Nole, Alexandre Tachard Passos, Jordan Ramsdell, Adam Roegiest, David Smith, Alessandro Sordoni
  
  SIGIR Forum | December 2019, Vol 53(2): pp. 62-75
  
  Publication PDF
- Robust Natural Language Inference Models with Example Forgetting
  
  Yadollah Yaghoobzadeh, Remi Tachet des Combes, T. J. Hazen, Alessandro Sordoni
  
  November 2019
  
  Arxiv | November 2019
  
  Publication
- Ordered Memory
  
  Yikang Shen, Shawn Tan, Arian Hosseini, Zhouhan Lin, Alessandro Sordoni, Aaron Courville
  
  NeurIPS 2019 | November 2019
  
  ArXiv
  
  Publication
- Metalearned Neural Memory
  
  Tsendsuren Munkhdalai, Alessandro Sordoni, Tong Wang, Adam Trischler
  
  NeurIPS 2019 | July 2019
  
  Preprint under review
  
  Publication
- Brief Report: Ordered Neurons: Integrating Tree Structures into Recurrent Neural Networks
  
  Yikeng Shen, Shawn Tan, Alessandro Sordoni, Aaron Courville
  
  ICLR 2019 | May 2019
  
  ICLR 2019 Best Paper Award
  
  Publication
- An Empirical Study of Example Forgetting during Deep Neural Network Learning
  
  Mariya Toneva, Alessandro Sordoni, Remi Tachet des Combes, Adam Trischler, Yoshua Bengio, Geoff Gordon
  
  ICLR 2019 | April 2019
  
  Publication
- Towards Text Generation with Adversarially Learned Neural Outlines
  
  Sandeep Subramanian, Sai Rajeswar, Alessandro Sordoni, Adam Trischler, Aaron Courville, Christopher Pal
  
  NeurIPS 2018 | December 2018
  
  Publication Project
- Counting to Explore and Generalize in Text-based Games
  
  Xingdi Yuan, Marc-Alexandre Côté, Alessandro Sordoni, Romain Laroche, Remi Tachet des Combes, Matthew Hausknecht, Adam Trischler
  
  European Workshop on Reinforcement Learning (EWRL) | October 2018
  
  Publication Project
- Straight to the Tree: Constituency Parsing with Neural Syntactic Distance
  
  Yikang Shen, Zhouhan Lin, Athul Paul Jacob, Alessandro Sordoni, Aaron Courville, Yoshua Bengio
  
  Proceedings of ACL | July 2018
  
  PDF
- Counting to Explore and Generalize in Text-based Games
  
  Xingdi Yuan, Marc-Alexandre Côté, Alessandro Sordoni, Romain Laroche, Remi Tachet des Combes, Matthew Hausknecht, Adam Trischler
  
  ICML 2018 | June 2018
  
  Preprint Project
- Focused Hierarchical RNNs for Conditional Sequence Processing
  
  Nan Rosemary Ke, Konrad Zolna, Alessandro Sordoni, Zhouhan Lin, Adam Trischler, Yoshua Bengio, Joelle Pineau, Laurent Charlin, Chris Pal
  
  ICML 2018 | June 2018
  
  Publication Project
- Augmented cyclegan: Learning many-to-many mappings from unpaired data
  
  Amjad Almahairi, Sai Rajeswar, Alessandro Sordoni, Philip Bachman, Aaron Courville
  
  ICML 2018 | June 2018
  
  ArXiv
  
  Publication
- Twin Networks: Matching the Future for Sequence Generation
  
  Dmitriy Serdyuk, Nan Rosemary Ke, Alessandro Sordoni, Chris Pal, Adam Trischler, Yoshua Bengio
  
  ICLR 2018 | January 2018
  
  Publication Project
- Z-Forcing: Training Stochastic Recurrent Networks
  
  Anirudh Goyal, Alessandro Sordoni, Marc-Alexandre Côté, Nan Rosemary Ke, Yoshua Bengio
  
  31st Conference on Neural Information Processing Systems (NIPS 2017 - Advances in Neural Information Processing Systems) | November 2017
  
  PDF Preprint
- Learning Algorithms for Active Learning
  
  Philip Bachman, Alessandro Sordoni, Adam Trischler
  
  34th International Conference on Machine Learning (ICML 2017) | August 2017
  
  Publication
- Orchard: An Open-Source Agentic Modeling Framework
  
  Baolin Peng, Wenlin Yao, Qianhui Wu, Hao Cheng, Xiao Yu, Ruiyi Yang, Tao Ge, Alessandro Sordoni, Xingdi Yuan, Yelong Shen, Pengcheng He, Tong Zhang, Zhou Yu, Jianfeng Gao
  
  May 2026
  
  arXiv | May 2026
  
  Publication
- debug-gym: A Text-Based Environment for Interactive Debugging
  
  Xingdi Yuan, Morgane M Moss, Charbel Feghali, Chinmay Singh, Darya Moldavskaya, Drew MacPhee, Lucas Caccia, Matheus Pereira, Minseon Kim, Alessandro Sordoni, Marc-Alexandre Côté
  
  March 2025
  
  ArXiv | March 2025
  
  Publication Access
- Unsupervised Dependency Graph Network
  
  Yikang Shen, Shawn Tan, Alessandro Sordoni, Peng Li, Jie Zhou, Aaron Courville
  
  ACL 2022 | May 2022
  
  Publication
- Better Language Model with Hypernym Class Prediction
  
  He Bai, Tong Wang, Alessandro Sordoni, Peng Shi
  
  ACL 2022 | May 2022
  
  Publication
- Self-training with Few-shot Rationalization: Teacher Explanations Aid Student in Few-shot NLU
  
  Meghana Moorthy Bhat, Alessandro Sordoni, Subhabrata (Subho) Mukherjee
  
  EMNLP 2021 | November 2021
  
  Publication Project Project
- Understanding by Understanding Not: Modeling Negation in Language Models
  
  Arian Hosseini, Siva Reddy, Dzmitry Bahdanau, Devon Hjelm, Alessandro Sordoni, Aaron Courville
  
  May 2021
  
  arXiv | May 2021
  
  PDF
- Linguistic dependencies and statistical dependence
  
  Jacob Louis Hoover, Alessandro Sordoni, Wenyu Du, Timothy J. O'Donnell
  
  Empirical Methods in Natural Language Processing | April 2021
  
  Publication
- Exploring and Predicting Transferability Across NLP Tasks
  
  Tu Vu, Tong Wang, Tsendsuren Munkhdalai, Alessandro Sordoni, Adam Trischler, Andrew Mattarella-Micke, Subhransu Maji, Mohit Iyyer
  
  May 2020
  
  ArXiv | May 2020
  
  Publication
- Fast Domain Adaptation For Goal-Oriented Dialogue Using A Hybrid Generative-Retrieval Transformer
  
  Igor Shalyminov, Alessandro Sordoni, Adam Atkinson, Hannes Schulz
  
  2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) | April 2020
  
  Publication Access Project
- Report on the First HIPstIR Workshop on the Future of Information Retrieval
  
  Laura Dietz, Bhaskar Mitra, Jeremy Pickens, Hana Anber, Sandeep Avula, Asia J. Biega, Adrian Boteanu, Shubham Chatterjee, Jeff Dalton, Shiri Dori-Hacohen, John Foley, Henry Feild, Ben Gamari, Rosie Jones, Pallika Kanani, Sumanta Kashyapi, Widad Machmouchi, Matthew Mitsui, Steve Nole, Alexandre Tachard Passos, Jordan Ramsdell, Adam Roegiest, David Smith, Alessandro Sordoni
  
  SIGIR Forum | December 2019, Vol 53(2): pp. 62-75
  
  Publication PDF

- Test-Time Learning with an Evolving Library
  
  Weijia Xu, Alessandro Sordoni, Chandan Singh, Zelalem Gero, Michel Galley, Xingdi Yuan, Jianfeng Gao
  
  May 2026
  
  arXiv | May 2026
  
  Publication
- Orchard: An Open-Source Agentic Modeling Framework
  
  Baolin Peng, Wenlin Yao, Qianhui Wu, Hao Cheng, Xiao Yu, Ruiyi Yang, Tao Ge, Alessandro Sordoni, Xingdi Yuan, Yelong Shen, Pengcheng He, Tong Zhang, Zhou Yu, Jianfeng Gao
  
  May 2026
  
  arXiv | May 2026
  
  Publication
- Trade-offs in Ensembling, Merging and Routing Among Parameter-Efficient Experts
  
  Sanae Lotfi, Lucas Caccia, Alessandro Sordoni, Jordan Ash, Miroslav Dudík
  
  March 2026
  
  arXiv | March 2026
  
  Publication
- Effect of Document Packing on the Latent Multi-Hop Reasoning Capabilities of Large Language Models
  
  Gabriele Prato, Shagun Sodhani, Alessandro Sordoni, Sarath Chandar, Alessandro Sordoni
  
  December 2025
  
  arXiv | December 2025
  
  Publication
- debug-gym: A Text-Based Environment for Interactive Debugging
  
  Xingdi Yuan, Morgane M Moss, Charbel Feghali, Chinmay Singh, Darya Moldavskaya, Drew MacPhee, Lucas Caccia, Matheus Pereira, Minseon Kim, Alessandro Sordoni, Marc-Alexandre Côté
  
  March 2025
  
  ArXiv | March 2025
  
  Publication Access
- Guiding Language Model Reasoning with Planning Tokens
  
  Xinyi Wang, Lucas Caccia, O. Ostapenko, Xingdi Yuan, Alessandro Sordoni
  
  October 2023
  
  arXiv | October 2023
  
  DOI Publication
- Explicitly Modeling Syntax in Language Models with Incremental Parsing and a Dynamic Oracle
  
  Yikang Shen, Shawn Tan, Alessandro Sordoni, Siva Reddy, Aaron Courville
  
  May 2021
  
  arXiv | May 2021
  
  Publication
- Understanding by Understanding Not: Modeling Negation in Language Models
  
  Arian Hosseini, Siva Reddy, Dzmitry Bahdanau, Devon Hjelm, Alessandro Sordoni, Aaron Courville
  
  May 2021
  
  arXiv | May 2021
  
  PDF
- Exploring and Predicting Transferability Across NLP Tasks
  
  Tu Vu, Tong Wang, Tsendsuren Munkhdalai, Alessandro Sordoni, Adam Trischler, Andrew Mattarella-Micke, Subhransu Maji, Mohit Iyyer
  
  May 2020
  
  ArXiv | May 2020
  
  Publication
- Robust Natural Language Inference Models with Example Forgetting
  
  Yadollah Yaghoobzadeh, Remi Tachet des Combes, T. J. Hazen, Alessandro Sordoni
  
  November 2019
  
  Arxiv | November 2019
  
  Publication
- Gistify! Codebase-Level Understanding via Runtime Execution
  
  Hyunji Lee, Minseon Kim, Chinmay Singh, Matheus Pereira, Atharv Sonwane, Isadora White, Elias Stengel-Eskin, Mohit Bansal, Zhengyan Shi, Alessandro Sordoni, Marc-Alexandre Côté, Xingdi Yuan, Lucas Caccia
  
  ICLR 2026 | October 2025
  
  DOI Publication
- The Markovian Thinker
  
  Milad Aghajohari, Kamran Chitsaz, Amirhossein Kazemnejad, Sarath Chandar, Alessandro Sordoni, Aaron C. Courville, Siva Reddy
  
  ICLR 2026 | October 2025
  
  Publication
- Learning to Solve Complex Problems via Dataset Decomposition
  
  Wanru Zhao, Lucas Caccia, Zhengyan Shi, Minseon Kim, Xingdi Yuan, Weijia Xu, Marc-Alexandre Côté, Alessandro Sordoni
  
  NeurIPS 2025 | July 2025
  
  Publication
- Training Plug-and-Play Knowledge Modules with Deep Context Distillation
  
  Lucas Caccia, Alan Ansell, E. Ponti, Ivan Vuli'c, Alessandro Sordoni
  
  COLM 2025 | July 2025
  
  Publication
- Putting the Value Back in RL: Better Test-Time Scaling by Unifying LLM Reasoners With Verifiers
  
  Kusha Sareen, Morgane M Moss, Alessandro Sordoni, Rishabh Agarwal, Arian Hosseini
  
  COLM 2025 | July 2025
  
  Publication
- Exploring Sparse Adapters for Scalable Merging of Parameter Efficient Experts
  
  Samin Yeasar Arnob, Zhan Su, Minseon Kim, Oleksiy Ostapenko, Riyasat Ohib, Esra'a Saleh, Doina Precup, Lucas Caccia, Alessandro Sordoni
  
  COLM 2025 | July 2025
  
  Publication
- Improving Context-Aware Preference Modeling for Language Models
  
  Silviu Pitis, Ziang Xiao, Nicolas Le Roux, Alessandro Sordoni
  
  NeurIPS 2024 | July 2024
  
  DOI Publication Publication
- Efficient Adversarial Training in LLMs with Continuous Attacks
  
  Sophie Xhonneux, Alessandro Sordoni, Stephan Günnemann, Gauthier Gidel, Leo Schwinn
  
  NeurIPS 2024 | May 2024
  
  DOI Publication Publication
- Towards Modular LLMs by Building and Reusing a Library of LoRAs
  
  O. Ostapenko, Zhan Su, E. Ponti, Laurent Charlin, Nicolas Le Roux, Matheus Pereira, Lucas Caccia, Alessandro Sordoni
  
  ICML 2024 | May 2024
  
  Publication
- Multi-Head Adapter Routing for Cross-Task Generalization
  
  Lucas Caccia, Edoardo Ponti, Zhan Su, Matheus Pereira, Nicolas Le Roux, Alessandro Sordoni
  
  NeurIPS 2023 | October 2023
  
  Publication
- Joint Prompt Optimization of Stacked LLMs using Variational Inference
  
  Alessandro Sordoni, Xingdi Yuan, Marc-Alexandre Côté, Matheus Pereira, Adam Trischler, Ziang Xiao, Arian Hosseini, Friederike Niedtner, Nicolas Le Roux
  
  NeurIPS 2023 | October 2023
  
  Publication
- Unsupervised Dependency Graph Network
  
  Yikang Shen, Shawn Tan, Alessandro Sordoni, Peng Li, Jie Zhou, Aaron Courville
  
  ACL 2022 | May 2022
  
  Publication
- Better Language Model with Hypernym Class Prediction
  
  He Bai, Tong Wang, Alessandro Sordoni, Peng Shi
  
  ACL 2022 | May 2022
  
  Publication
- Learning to Dequantise with Truncated Flows
  
  Shawn Tan, Chin-Wei Huang, Alessandro Sordoni, Aaron Courville
  
  ICLR 2022 | April 2022
  
  Publication
- Evaluating Distributional Distortion in Neural Language Modeling
  
  Benjamin LeBrun, Alessandro Sordoni, Timothy J. O'Donnell
  
  ICLR 2022 | April 2022
  
  Publication
- The emergence of the shape bias results from communicative efficiency
  
  Eva Portelance, Michael C. Frank, Dan Jurafsky, Alessandro Sordoni, Romain Laroche
  
  Conference on Computational Natural Language Learning (CoNLL) | November 2021
  
  Publication
- Self-training with Few-shot Rationalization: Teacher Explanations Aid Student in Few-shot NLU
  
  Meghana Moorthy Bhat, Alessandro Sordoni, Subhabrata (Subho) Mukherjee
  
  EMNLP 2021 | November 2021
  
  Publication Project Project
- Decomposed Mutual Information Estimation for Contrastive Representation Learning
  
  Alessandro Sordoni, Nouha Dziri, Hannes Schulz, Geoff Gordon, Philip Bachman, Remi Tachet des Combes
  
  2021 International Conference on Machine Learning | July 2021
  
  DOI
- Linguistic dependencies and statistical dependence
  
  Jacob Louis Hoover, Alessandro Sordoni, Wenyu Du, Timothy J. O'Donnell
  
  Empirical Methods in Natural Language Processing | April 2021
  
  Publication
- Fast Domain Adaptation For Goal-Oriented Dialogue Using A Hybrid Generative-Retrieval Transformer
  
  Igor Shalyminov, Alessandro Sordoni, Adam Atkinson, Hannes Schulz
  
  2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) | April 2020
  
  Publication Access Project
- Ordered Memory
  
  Yikang Shen, Shawn Tan, Arian Hosseini, Zhouhan Lin, Alessandro Sordoni, Aaron Courville
  
  NeurIPS 2019 | November 2019
  
  ArXiv
  
  Publication
- Metalearned Neural Memory
  
  Tsendsuren Munkhdalai, Alessandro Sordoni, Tong Wang, Adam Trischler
  
  NeurIPS 2019 | July 2019
  
  Preprint under review
  
  Publication
- Brief Report: Ordered Neurons: Integrating Tree Structures into Recurrent Neural Networks
  
  Yikeng Shen, Shawn Tan, Alessandro Sordoni, Aaron Courville
  
  ICLR 2019 | May 2019
  
  ICLR 2019 Best Paper Award
  
  Publication
- An Empirical Study of Example Forgetting during Deep Neural Network Learning
  
  Mariya Toneva, Alessandro Sordoni, Remi Tachet des Combes, Adam Trischler, Yoshua Bengio, Geoff Gordon
  
  ICLR 2019 | April 2019
  
  Publication
- Towards Text Generation with Adversarially Learned Neural Outlines
  
  Sandeep Subramanian, Sai Rajeswar, Alessandro Sordoni, Adam Trischler, Aaron Courville, Christopher Pal
  
  NeurIPS 2018 | December 2018
  
  Publication Project
- Counting to Explore and Generalize in Text-based Games
  
  Xingdi Yuan, Marc-Alexandre Côté, Alessandro Sordoni, Romain Laroche, Remi Tachet des Combes, Matthew Hausknecht, Adam Trischler
  
  European Workshop on Reinforcement Learning (EWRL) | October 2018
  
  Publication Project
- Counting to Explore and Generalize in Text-based Games
  
  Xingdi Yuan, Marc-Alexandre Côté, Alessandro Sordoni, Romain Laroche, Remi Tachet des Combes, Matthew Hausknecht, Adam Trischler
  
  ICML 2018 | June 2018
  
  Preprint Project
- Focused Hierarchical RNNs for Conditional Sequence Processing
  
  Nan Rosemary Ke, Konrad Zolna, Alessandro Sordoni, Zhouhan Lin, Adam Trischler, Yoshua Bengio, Joelle Pineau, Laurent Charlin, Chris Pal
  
  ICML 2018 | June 2018
  
  Publication Project
- Augmented cyclegan: Learning many-to-many mappings from unpaired data
  
  Amjad Almahairi, Sai Rajeswar, Alessandro Sordoni, Philip Bachman, Aaron Courville
  
  ICML 2018 | June 2018
  
  ArXiv
  
  Publication
- Twin Networks: Matching the Future for Sequence Generation
  
  Dmitriy Serdyuk, Nan Rosemary Ke, Alessandro Sordoni, Chris Pal, Adam Trischler, Yoshua Bengio
  
  ICLR 2018 | January 2018
  
  Publication Project
- Z-Forcing: Training Stochastic Recurrent Networks
  
  Anirudh Goyal, Alessandro Sordoni, Marc-Alexandre Côté, Nan Rosemary Ke, Yoshua Bengio
  
  31st Conference on Neural Information Processing Systems (NIPS 2017 - Advances in Neural Information Processing Systems) | November 2017
  
  PDF Preprint
- Learning Algorithms for Active Learning
  
  Philip Bachman, Alessandro Sordoni, Adam Trischler
  
  34th International Conference on Machine Learning (ICML 2017) | August 2017
  
  Publication
- A Survey on Model MoErging: Recycling and Routing Among Specialized Experts for Collaborative Learning
  
  Prateek Yadav, Colin Raffel, Mohammed Muqeeth, Lucas Caccia, Haokun Liu, Tian-Xiang Chen, Mohit Bansal, Leshem Choshen, Alessandro Sordoni
  
  ArXiv | August 2024
  
  DOI Publication Publication
- Multi-Head Adapter Routing for Data-Efficient Fine-Tuning
  
  Lucas Caccia, E. Ponti, Lu Liu, Matheus Pereira, Nicolas Le Roux, Alessandro Sordoni
  
  ArXiv | November 2023
  
  DOI Publication Publication
- Hybrid Generative-Retrieval Transformers For Dialogue Domain Adaptation
  
  Igor Shalyminov, Alessandro Sordoni, Adam Atkinson, Hannes Schulz
  
  arXiv preprint arXiv:2003.01680 | March 2020
  
  PDF Project
- Report on the First HIPstIR Workshop on the Future of Information Retrieval
  
  Laura Dietz, Bhaskar Mitra, Jeremy Pickens, Hana Anber, Sandeep Avula, Asia J. Biega, Adrian Boteanu, Shubham Chatterjee, Jeff Dalton, Shiri Dori-Hacohen, John Foley, Henry Feild, Ben Gamari, Rosie Jones, Pallika Kanani, Sumanta Kashyapi, Widad Machmouchi, Matthew Mitsui, Steve Nole, Alexandre Tachard Passos, Jordan Ramsdell, Adam Roegiest, David Smith, Alessandro Sordoni
  
  SIGIR Forum | December 2019, Vol 53(2): pp. 62-75
  
  Publication PDF
- Straight to the Tree: Constituency Parsing with Neural Syntactic Distance
  
  Yikang Shen, Zhouhan Lin, Athul Paul Jacob, Alessandro Sordoni, Aaron Courville, Yoshua Bengio
  
  Proceedings of ACL | July 2018
  
  PDF
- Energy-efficient Reinforcement Learning by Discovering Neural Pathways
  
  Alessandro Sordoni
  
  October 2024

Alessandro Sordoni

Publications

Orchard: An Open-Source Agentic Modeling Framework

Test-Time Learning with an Evolving Library

Trade-offs in Ensembling, Merging and Routing Among Parameter-Efficient Experts

Effect of Document Packing on the Latent Multi-Hop Reasoning Capabilities of Large Language Models

Gistify! Codebase-Level Understanding via Runtime Execution

The Markovian Thinker

Learning to Solve Complex Problems via Dataset Decomposition

Exploring Sparse Adapters for Scalable Merging of Parameter Efficient Experts

Putting the Value Back in RL: Better Test-Time Scaling by Unifying LLM Reasoners With Verifiers

Training Plug-and-Play Knowledge Modules with Deep Context Distillation

debug-gym: A Text-Based Environment for Interactive Debugging

Energy-efficient Reinforcement Learning by Discovering Neural Pathways

A Survey on Model MoErging: Recycling and Routing Among Specialized Experts for Collaborative Learning

Improving Context-Aware Preference Modeling for Language Models

Efficient Adversarial Training in LLMs with Continuous Attacks

Towards Modular LLMs by Building and Reusing a Library of LoRAs

Multi-Head Adapter Routing for Data-Efficient Fine-Tuning

Guiding Language Model Reasoning with Planning Tokens

Joint Prompt Optimization of Stacked LLMs using Variational Inference

Multi-Head Adapter Routing for Cross-Task Generalization

Better Language Model with Hypernym Class Prediction

Unsupervised Dependency Graph Network

Evaluating Distributional Distortion in Neural Language Modeling

Learning to Dequantise with Truncated Flows

Self-training with Few-shot Rationalization: Teacher Explanations Aid Student in Few-shot NLU

The emergence of the shape bias results from communicative efficiency

Decomposed Mutual Information Estimation for Contrastive Representation Learning

Explicitly Modeling Syntax in Language Models with Incremental Parsing and a Dynamic Oracle

Understanding by Understanding Not: Modeling Negation in Language Models

Linguistic dependencies and statistical dependence

Exploring and Predicting Transferability Across NLP Tasks

Fast Domain Adaptation For Goal-Oriented Dialogue Using A Hybrid Generative-Retrieval Transformer

Hybrid Generative-Retrieval Transformers For Dialogue Domain Adaptation

Report on the First HIPstIR Workshop on the Future of Information Retrieval

Ordered Memory

Robust Natural Language Inference Models with Example Forgetting

Metalearned Neural Memory

Brief Report: Ordered Neurons: Integrating Tree Structures into Recurrent Neural Networks

An Empirical Study of Example Forgetting during Deep Neural Network Learning

Towards Text Generation with Adversarially Learned Neural Outlines

Counting to Explore and Generalize in Text-based Games

Straight to the Tree: Constituency Parsing with Neural Syntactic Distance

Counting to Explore and Generalize in Text-based Games

Augmented cyclegan: Learning many-to-many mappings from unpaired data

Focused Hierarchical RNNs for Conditional Sequence Processing

Twin Networks: Matching the Future for Sequence Generation

Z-Forcing: Training Stochastic Recurrent Networks

Learning Algorithms for Active Learning

Test-Time Learning with an Evolving Library

Orchard: An Open-Source Agentic Modeling Framework

Trade-offs in Ensembling, Merging and Routing Among Parameter-Efficient Experts

Effect of Document Packing on the Latent Multi-Hop Reasoning Capabilities of Large Language Models

Gistify! Codebase-Level Understanding via Runtime Execution

The Markovian Thinker

Learning to Solve Complex Problems via Dataset Decomposition

Training Plug-and-Play Knowledge Modules with Deep Context Distillation

Putting the Value Back in RL: Better Test-Time Scaling by Unifying LLM Reasoners With Verifiers

Exploring Sparse Adapters for Scalable Merging of Parameter Efficient Experts

debug-gym: A Text-Based Environment for Interactive Debugging

Energy-efficient Reinforcement Learning by Discovering Neural Pathways

A Survey on Model MoErging: Recycling and Routing Among Specialized Experts for Collaborative Learning

Improving Context-Aware Preference Modeling for Language Models

Efficient Adversarial Training in LLMs with Continuous Attacks

Towards Modular LLMs by Building and Reusing a Library of LoRAs

Multi-Head Adapter Routing for Data-Efficient Fine-Tuning

Guiding Language Model Reasoning with Planning Tokens

Multi-Head Adapter Routing for Cross-Task Generalization

Joint Prompt Optimization of Stacked LLMs using Variational Inference

Learning to Dequantise with Truncated Flows

Evaluating Distributional Distortion in Neural Language Modeling

The emergence of the shape bias results from communicative efficiency

Decomposed Mutual Information Estimation for Contrastive Representation Learning

Explicitly Modeling Syntax in Language Models with Incremental Parsing and a Dynamic Oracle

Understanding by Understanding Not: Modeling Negation in Language Models

Hybrid Generative-Retrieval Transformers For Dialogue Domain Adaptation

Report on the First HIPstIR Workshop on the Future of Information Retrieval

Robust Natural Language Inference Models with Example Forgetting

Ordered Memory