Shuohang Wang

Principal Researcher

论文与出版物

查看方式：

- Decoder-Hybrid-Decoder Architecture for Efficient Reasoning with Long Generation
  
  Liliang Ren, Congcong Chen, Haoran Xu, Young Jin Kim, Adam Atkinson, Zheng Zhan, Jiankai Sun, Baolin Peng, Liyuan Liu, Shuohang Wang, Hao Cheng, Jianfeng Gao, Weizhu Chen, Yelong Shen
  
  NeurIPS 2025 | July 2025
  
  DOI
- Routing Mamba: Scaling State Space Models with Mixture-of-Experts Projection
  
  Zheng Zhan, Liliang Ren, Shuohang Wang, Liyuan Liu, Yang Liu, Yeyun Gong, Yanzhi Wang, Yelong Shen
  
  NeurIPS 2025 | June 2025
  
  DOI Publication Publication
- Reinforcement Learning for Reasoning in Large Language Models with One Training Example
  
  Yiping Wang, Qing Yang, Zhiyuan Zeng, Liliang Ren, Liyuan Liu, Baolin Peng, Hao Cheng, Xuehai He, Kuan Wang, Jianfeng Gao, Weizhu Chen, Shuohang Wang, S. Du, Yelong Shen
  
  NeurIPS 2025 | April 2025
  
  DOI Publication Publication
- SciAgent: Tool-augmented Language Models for Scientific Reasoning
  
  Yubo Ma, Zhibin Gou, Junheng Hao, Ruochen Xu, Shuohang Wang, Liangming Pan, Yujiu Yang, Yixin Cao, Aixin Sun
  
  2024 Empirical Methods in Natural Language Processing | November 2024
  
  DOI Publication
- Sparse Modular Activation for Efficient Sequence Modeling
  
  Liliang Ren, Yang Liu, Shuohang Wang, Yichong Xu, Chenguang Zhu, ChengXiang Zhai
  
  NeurIPS 2023 | July 2023
  
  Publication
- In-Context Demonstration Selection with Cross Entropy Difference
  
  Dan Iter, Reid Pryzant, Ruochen Xu, Shuohang Wang, Yang Liu, Chenguang Zhu
  
  May 2023
  
  Arxiv
  
  Publication
- PEARL: Prompting Large Language Models to Plan and Execute Actions Over Long Documents
  
  Simeng Sun, Yang Liu, Shuohang Wang, Chenguang Zhu, Mohit Iyyer
  
  May 2023
  
  Publication
- Generate rather than Retrieve: Large Language Models are Strong Context Generators
  
  Wenhao Yu, Dan Iter, Shuohang Wang, Yichong Xu, Mingxuan Ju, Soumya Sanyal, Chenguang Zhu, Michael Zeng, Meng Jiang
  
  ICLR 2023 | May 2023
  
  Publication
- InheritSumm: A General, Versatile and Compact Summarizer by Distilling from GPT
  
  Yichong Xu, Ruochen Xu, Dan Iter, Yang Liu, Shuohang Wang, Chenguang Zhu, Michael Zeng
  
  May 2023
  
  PDF
- LMGQS: A Large-scale Dataset for Query-focused Summarization
  
  Ruochen Xu, Song Wang, Yang Liu, Shuohang Wang, Yichong Xu, Dan Iter, Chenguang Zhu, Michael Zeng
  
  May 2023
  
  PDF
- Prompting GPT-3 To Be Reliable
  
  Silei Cheng, Zhe Gan, Zhengyuan Yang, Shuohang Wang, Jianfeng Wang, Jordan Boyd-Graber, Lijuan Wang
  
  International Conference on Learning Representations (ICLR 23) | May 2023
  
  PDF
- G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment
  
  Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, Chenguang Zhu
  
  arXiv 2303.16634 | March 2023
  
  Publication
- AdaPrompt: Adaptive Model Training for Prompt-based NLP
  
  Yulong Chen, Yang Liu, Li Dong, Shuohang Wang, Chenguang Zhu, Michael Zeng, Yue Zhang
  
  Findings of Empirical Methods in Natural Language Processing (EMNLP), Abu Dhabi, the United Arab Emirates, 2022. | December 2022
  
  Publication
- Empowering Language Models with Knowledge Graph Reasoning for Open-Domain Question Answering
  
  Ziniu Hu, Yichong Xu, Wenhao Yu, Shuohang Wang, Ziyi Yang, Chenguang Zhu, Kai-Wei Chang, Yizhou Sun
  
  2022 Empirical Methods in Natural Language Processing | December 2022
  
  Best Paper Award at SoCal NLP Symposium 2022
  
  PDF Publication
- ParaTag: A Dataset of Paraphrase Tagging for Fine-Grained Labels, NLG Evaluation, and Data Augmentation
  
  Shuohang Wang, Ruochen Xu, Yang Liu, Chenguang Zhu, Michael Zeng
  
  Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing | December 2022
  
  PDF
- Task Compass: Scaling Multi-task Pre-training with Task Prefix
  
  Zhuosheng Zhang, Shuohang Wang, Yichong Xu, Yuwei Fang, Wenhao Yu, Yang Liu, Hai Zhao, Chenguang Zhu, Michael Zeng
  
  EMNLP 2022 | December 2022
  
  Publication
- Language Models with Image Descriptors are Strong Few-Shot Video-Language Learners
  
  Zhenhailong Wang, Manling Li, Ruochen Xu, Luowei Zhou, Jie Lei, Xudong Lin, Shuohang Wang, Ziyi Yang, Chenguang Zhu, Derek Hoiem, Shih-Fu Chang, Mohit Bansal, Heng Ji
  
  Conference on Neural Information Processing Systems (NeurIPS) 2022 | November 2022
  
  PDF
- Human Parity on CommonsenseQA: Augmenting Self-Attention with External Attention
  
  Yichong Xu, Chenguang Zhu, Shuohang Wang, Siqi Sun, Hao Cheng, Xiaodong Liu, Jianfeng Gao, Pengcheng He, Michael Zeng, Xuedong Huang
  
  IJCAI 2022 | July 2022
  
  human parity result on CommonsenseQA
  
  Publication
- An Empirical Study of Training End-to-End Vision-and-Language Transformers
  
  Zi-Yi Dou, Yichong Xu, Zhe Gan, Jianfeng Wang, Shuohang Wang (shuowa), Lijuan Wang (lijuanw), Chenguang Zhu, Nanyun (Violet) Peng, Zicheng Liu, Michael Zeng
  
  CVPR | June 2022
  
  Publication Publication 项目
- CLIP-Event: Connecting Text and Images with Event Structures
  
  Manling Li, Ruochen Xu, Shuohang Wang, Luowei Zhou, Xudong Lin, Chenguang Zhu, Michael Zeng, Heng Ji, Shih-Fu Chang
  
  Conference on Computer Vision and Pattern Recognition (CVPR), 2022. | June 2022
  
  Publication
- DICT-BERT: Enhancing Language Model Pre-Training with Dictionary
  
  Wenhao Yu, Chenguang Zhu, Yuwei Fang, Donghan Yu, Shuohang Wang, Yichong Xu, Michael Zeng, Meng Jiang
  
  ACL 2022 | May 2022
  
  Preprint
- KG-FiD: Infusing Knowledge Graph in Fusion-in-Decoder for Open-Domain Question Answering
  
  Donghan Yu, Chenguang Zhu, Yuwei Fang, Wenhao Yu, Shuohang Wang, Yichong Xu, Xiang Ren, Yiming Yang, Michael Zeng
  
  ACL 2022 | May 2022
  
  Publication
- Leveraging Knowledge in Multilingual Commonsense Reasoning
  
  Yuwei Fang, Shuohang Wang, Yichong Xu, Ruochen Xu, Siqi Sun, Chenguang Zhu, Michael Zeng
  
  ACL 2022 | May 2022
  
  Publication
- Training Data is More Valuable than You Think: A Simple and Effective Method by Retrieving from Training Data
  
  Shuohang Wang, Yichong Xu, Yuwei Fang, Yang Liu, Siqi Sun, Ruochen Xu, Chenguang Zhu, Michael Zeng
  
  ACL 2022 | May 2022
  
  Publication
- Playing Lottery Tickets with Vision and Language
  
  Zhe Gan, Yen-Chun Chen, Linjie Li, Tianlong Chen, Yu Cheng, Shuohang Wang, Jingjing Liu, Lijuan Wang, Zicheng Liu
  
  AAAI 2022 | February 2022
  
  Publication 项目
- Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Models
  
  Boxin Wang, Chejian Xu, Shuohang Wang, Zhe Gan, Yu Cheng, Jianfeng Gao, Ahmed Awadallah, Bo Li
  
  35th Annual Conference on Neural Information Processing System (NeurIPS 2021) | December 2021
  
  Publication
- MLP Architectures for Vision-and-Language Modeling: An Empirical Study
  
  Yixin Nie, Linjie Li, Zhe Gan, Shuohang Wang, Chenguang Zhu, Michael Zeng, Zicheng Liu, Mohit Bansal, Lijuan Wang
  
  December 2021
  
  Preprint 项目
- The Elastic Lottery Ticket Hypothesis
  
  Xiaohan Chen, Yu Cheng, Shuohang Wang, Zhe Gan, Jingjing Liu, Zhangyang Wang
  
  35th Annual Conference on Neural Information Processing System (NeurIPS 2021) | December 2021
  
  Publication 项目
- NOAHQA: Numerical Reasoning with Interpretable Graph Question Answering Dataset
  
  Qiyuan Zhang, Lei Wang, Sicheng Yu, Shuohang Wang, Yang Wang, Jing Jiang, Ee-Peng Lim
  
  2021 Empirical Methods in Natural Language Processing | September 2021
  
  Publication
- Accelerating Real-Time Question Answering via Question Generation
  
  Yuwei Fang, Shuohang Wang, Zhe Gan, Siqi Sun, Jingjing Liu, Chenguang Zhu
  
  arXiv: Computation and Language | September 2021
- Cluster-Former: Clustering-based Sparse Transformer for Long-Range Dependency Encoding
  
  Shuohang Wang, Luowei Zhou, Zhe Gan, Yen-Chun Chen, Siqi Sun, Yuwei Fang, Yu Cheng, Jingjing Liu
  
  ACL-IJCNLP 2021 | August 2021
- EarlyBERT: Efficient BERT Training via Early-bird Lottery Tickets
  
  Xiaohan Chen, Yu Cheng, Shuohang Wang, Zhe Gan, Atlas Wang, Jingjing Liu
  
  ACL-IJCNLP 2021 | August 2021
  
  项目
- Want To Reduce Labeling Cost? GPT-3 Can Help
  
  Shuohang Wang, Yang Liu, Yichong Xu, Chenguang Zhu, Michael Zeng
  
  Conference on Empirical Methods in Natural Language Processing | August 2021
  
  Publication
- UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-training
  
  Mingyang Zhou, Luowei Zhou, Shuohang Wang, Yu Cheng, Linjie Li, Zhou Yu, Jingjing Liu
  
  IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2021) | June 2021
  
  Publication
- LightningDOT: Pre-training Visual-Semantic Embeddings for Real-Time Image-Text Retrieval
  
  Siqi Sun, Yen-Chun Chen, Linjie Li, Shuohang Wang, Yuwei Fang, JJ (Jingjing) Liu
  
  March 2021
  
  Publication
- InfoBERT: Improving Robustness of Language Models from An Information Theoretic Perspective
  
  Boxin Wang, Shuohang Wang, Yu Cheng, Zhe Gan, Ruoxi Jia, Bo Li, JJ (Jingjing) Liu
  
  International Conference on Learning Representations (ICLR 2021) | March 2021
  
  Publication
- Contrastive Distillation on Intermediate Representations for Language Model Compression
  
  Siqi Sun, Zhe Gan, Yu Cheng, Yuwei Fang, Shuohang Wang, JJ (Jingjing) Liu
  
  The 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP 2020) | October 2020
  
  Publication
- Cross-Thought for Sentence Encoder Pre-training
  
  Shuohang Wang, Yuwei Fang, Siqi Sun, Zhe Gan, Yu Cheng, Jing Jiang, JJ (Jingjing) Liu
  
  The 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP 2020) | October 2020
  
  Publication
- Multi-Fact Correction in Abstractive Text Summarization
  
  Yue Dong, Shuohang Wang, Zhe Gan, Yu Cheng, Jackie Chi Kit Cheung, JJ (Jingjing) Liu
  
  The 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP 2020) | October 2020
  
  Publication
- Hierarchical Graph Network for Multi-hop Question Answering
  
  Yuwei Fang, Siqi Sun, Zhe Gan, Rohit Pillai, Shuohang Wang, JJ (Jingjing) Liu
  
  2020 Empirical Methods in Natural Language Processing | April 2020
  
  DOI PDF Github

- Decoder-Hybrid-Decoder Architecture for Efficient Reasoning with Long Generation
  
  Liliang Ren, Congcong Chen, Haoran Xu, Young Jin Kim, Adam Atkinson, Zheng Zhan, Jiankai Sun, Baolin Peng, Liyuan Liu, Shuohang Wang, Hao Cheng, Jianfeng Gao, Weizhu Chen, Yelong Shen
  
  NeurIPS 2025 | July 2025
  
  DOI
- Routing Mamba: Scaling State Space Models with Mixture-of-Experts Projection
  
  Zheng Zhan, Liliang Ren, Shuohang Wang, Liyuan Liu, Yang Liu, Yeyun Gong, Yanzhi Wang, Yelong Shen
  
  NeurIPS 2025 | June 2025
  
  DOI Publication Publication
- Reinforcement Learning for Reasoning in Large Language Models with One Training Example
  
  Yiping Wang, Qing Yang, Zhiyuan Zeng, Liliang Ren, Liyuan Liu, Baolin Peng, Hao Cheng, Xuehai He, Kuan Wang, Jianfeng Gao, Weizhu Chen, Shuohang Wang, S. Du, Yelong Shen
  
  NeurIPS 2025 | April 2025
  
  DOI Publication Publication
- SciAgent: Tool-augmented Language Models for Scientific Reasoning
  
  Yubo Ma, Zhibin Gou, Junheng Hao, Ruochen Xu, Shuohang Wang, Liangming Pan, Yujiu Yang, Yixin Cao, Aixin Sun
  
  2024 Empirical Methods in Natural Language Processing | November 2024
  
  DOI Publication
- Sparse Modular Activation for Efficient Sequence Modeling
  
  Liliang Ren, Yang Liu, Shuohang Wang, Yichong Xu, Chenguang Zhu, ChengXiang Zhai
  
  NeurIPS 2023 | July 2023
  
  Publication
- In-Context Demonstration Selection with Cross Entropy Difference
  
  Dan Iter, Reid Pryzant, Ruochen Xu, Shuohang Wang, Yang Liu, Chenguang Zhu
  
  May 2023
  
  Arxiv
  
  Publication
- PEARL: Prompting Large Language Models to Plan and Execute Actions Over Long Documents
  
  Simeng Sun, Yang Liu, Shuohang Wang, Chenguang Zhu, Mohit Iyyer
  
  May 2023
  
  Publication
- LMGQS: A Large-scale Dataset for Query-focused Summarization
  
  Ruochen Xu, Song Wang, Yang Liu, Shuohang Wang, Yichong Xu, Dan Iter, Chenguang Zhu, Michael Zeng
  
  May 2023
  
  PDF
- InheritSumm: A General, Versatile and Compact Summarizer by Distilling from GPT
  
  Yichong Xu, Ruochen Xu, Dan Iter, Yang Liu, Shuohang Wang, Chenguang Zhu, Michael Zeng
  
  May 2023
  
  PDF
- Prompting GPT-3 To Be Reliable
  
  Silei Cheng, Zhe Gan, Zhengyuan Yang, Shuohang Wang, Jianfeng Wang, Jordan Boyd-Graber, Lijuan Wang
  
  International Conference on Learning Representations (ICLR 23) | May 2023
  
  PDF
- ParaTag: A Dataset of Paraphrase Tagging for Fine-Grained Labels, NLG Evaluation, and Data Augmentation
  
  Shuohang Wang, Ruochen Xu, Yang Liu, Chenguang Zhu, Michael Zeng
  
  Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing | December 2022
  
  PDF
- Empowering Language Models with Knowledge Graph Reasoning for Open-Domain Question Answering
  
  Ziniu Hu, Yichong Xu, Wenhao Yu, Shuohang Wang, Ziyi Yang, Chenguang Zhu, Kai-Wei Chang, Yizhou Sun
  
  2022 Empirical Methods in Natural Language Processing | December 2022
  
  Best Paper Award at SoCal NLP Symposium 2022
  
  PDF Publication
- Task Compass: Scaling Multi-task Pre-training with Task Prefix
  
  Zhuosheng Zhang, Shuohang Wang, Yichong Xu, Yuwei Fang, Wenhao Yu, Yang Liu, Hai Zhao, Chenguang Zhu, Michael Zeng
  
  EMNLP 2022 | December 2022
  
  Publication
- Human Parity on CommonsenseQA: Augmenting Self-Attention with External Attention
  
  Yichong Xu, Chenguang Zhu, Shuohang Wang, Siqi Sun, Hao Cheng, Xiaodong Liu, Jianfeng Gao, Pengcheng He, Michael Zeng, Xuedong Huang
  
  IJCAI 2022 | July 2022
  
  human parity result on CommonsenseQA
  
  Publication
- CLIP-Event: Connecting Text and Images with Event Structures
  
  Manling Li, Ruochen Xu, Shuohang Wang, Luowei Zhou, Xudong Lin, Chenguang Zhu, Michael Zeng, Heng Ji, Shih-Fu Chang
  
  Conference on Computer Vision and Pattern Recognition (CVPR), 2022. | June 2022
  
  Publication
- Training Data is More Valuable than You Think: A Simple and Effective Method by Retrieving from Training Data
  
  Shuohang Wang, Yichong Xu, Yuwei Fang, Yang Liu, Siqi Sun, Ruochen Xu, Chenguang Zhu, Michael Zeng
  
  ACL 2022 | May 2022
  
  Publication
- DICT-BERT: Enhancing Language Model Pre-Training with Dictionary
  
  Wenhao Yu, Chenguang Zhu, Yuwei Fang, Donghan Yu, Shuohang Wang, Yichong Xu, Michael Zeng, Meng Jiang
  
  ACL 2022 | May 2022
  
  Preprint
- Playing Lottery Tickets with Vision and Language
  
  Zhe Gan, Yen-Chun Chen, Linjie Li, Tianlong Chen, Yu Cheng, Shuohang Wang, Jingjing Liu, Lijuan Wang, Zicheng Liu
  
  AAAI 2022 | February 2022
  
  Publication 项目
- MLP Architectures for Vision-and-Language Modeling: An Empirical Study
  
  Yixin Nie, Linjie Li, Zhe Gan, Shuohang Wang, Chenguang Zhu, Michael Zeng, Zicheng Liu, Mohit Bansal, Lijuan Wang
  
  December 2021
  
  Preprint 项目
- Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Models
  
  Boxin Wang, Chejian Xu, Shuohang Wang, Zhe Gan, Yu Cheng, Jianfeng Gao, Ahmed Awadallah, Bo Li
  
  35th Annual Conference on Neural Information Processing System (NeurIPS 2021) | December 2021
  
  Publication
- The Elastic Lottery Ticket Hypothesis
  
  Xiaohan Chen, Yu Cheng, Shuohang Wang, Zhe Gan, Jingjing Liu, Zhangyang Wang
  
  35th Annual Conference on Neural Information Processing System (NeurIPS 2021) | December 2021
  
  Publication 项目
- Cluster-Former: Clustering-based Sparse Transformer for Long-Range Dependency Encoding
  
  Shuohang Wang, Luowei Zhou, Zhe Gan, Yen-Chun Chen, Siqi Sun, Yuwei Fang, Yu Cheng, Jingjing Liu
  
  ACL-IJCNLP 2021 | August 2021
- EarlyBERT: Efficient BERT Training via Early-bird Lottery Tickets
  
  Xiaohan Chen, Yu Cheng, Shuohang Wang, Zhe Gan, Atlas Wang, Jingjing Liu
  
  ACL-IJCNLP 2021 | August 2021
  
  项目
- Want To Reduce Labeling Cost? GPT-3 Can Help
  
  Shuohang Wang, Yang Liu, Yichong Xu, Chenguang Zhu, Michael Zeng
  
  Conference on Empirical Methods in Natural Language Processing | August 2021
  
  Publication
- UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-training
  
  Mingyang Zhou, Luowei Zhou, Shuohang Wang, Yu Cheng, Linjie Li, Zhou Yu, Jingjing Liu
  
  IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2021) | June 2021
  
  Publication
- LightningDOT: Pre-training Visual-Semantic Embeddings for Real-Time Image-Text Retrieval
  
  Siqi Sun, Yen-Chun Chen, Linjie Li, Shuohang Wang, Yuwei Fang, JJ (Jingjing) Liu
  
  March 2021
  
  Publication
- InfoBERT: Improving Robustness of Language Models from An Information Theoretic Perspective
  
  Boxin Wang, Shuohang Wang, Yu Cheng, Zhe Gan, Ruoxi Jia, Bo Li, JJ (Jingjing) Liu
  
  International Conference on Learning Representations (ICLR 2021) | March 2021
  
  Publication
- Cross-Thought for Sentence Encoder Pre-training
  
  Shuohang Wang, Yuwei Fang, Siqi Sun, Zhe Gan, Yu Cheng, Jing Jiang, JJ (Jingjing) Liu
  
  The 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP 2020) | October 2020
  
  Publication
- Multi-Fact Correction in Abstractive Text Summarization
  
  Yue Dong, Shuohang Wang, Zhe Gan, Yu Cheng, Jackie Chi Kit Cheung, JJ (Jingjing) Liu
  
  The 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP 2020) | October 2020
  
  Publication
- Contrastive Distillation on Intermediate Representations for Language Model Compression
  
  Siqi Sun, Zhe Gan, Yu Cheng, Yuwei Fang, Shuohang Wang, JJ (Jingjing) Liu
  
  The 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP 2020) | October 2020
  
  Publication
- Hierarchical Graph Network for Multi-hop Question Answering
  
  Yuwei Fang, Siqi Sun, Zhe Gan, Rohit Pillai, Shuohang Wang, JJ (Jingjing) Liu
  
  2020 Empirical Methods in Natural Language Processing | April 2020
  
  DOI PDF Github
- SciAgent: Tool-augmented Language Models for Scientific Reasoning
  
  Yubo Ma, Zhibin Gou, Junheng Hao, Ruochen Xu, Shuohang Wang, Liangming Pan, Yujiu Yang, Yixin Cao, Aixin Sun
  
  2024 Empirical Methods in Natural Language Processing | November 2024
  
  DOI Publication
- PEARL: Prompting Large Language Models to Plan and Execute Actions Over Long Documents
  
  Simeng Sun, Yang Liu, Shuohang Wang, Chenguang Zhu, Mohit Iyyer
  
  May 2023
  
  Publication
- LMGQS: A Large-scale Dataset for Query-focused Summarization
  
  Ruochen Xu, Song Wang, Yang Liu, Shuohang Wang, Yichong Xu, Dan Iter, Chenguang Zhu, Michael Zeng
  
  May 2023
  
  PDF
- Prompting GPT-3 To Be Reliable
  
  Silei Cheng, Zhe Gan, Zhengyuan Yang, Shuohang Wang, Jianfeng Wang, Jordan Boyd-Graber, Lijuan Wang
  
  International Conference on Learning Representations (ICLR 23) | May 2023
  
  PDF
- Generate rather than Retrieve: Large Language Models are Strong Context Generators
  
  Wenhao Yu, Dan Iter, Shuohang Wang, Yichong Xu, Mingxuan Ju, Soumya Sanyal, Chenguang Zhu, Michael Zeng, Meng Jiang
  
  ICLR 2023 | May 2023
  
  Publication
- G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment
  
  Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, Chenguang Zhu
  
  arXiv 2303.16634 | March 2023
  
  Publication
- ParaTag: A Dataset of Paraphrase Tagging for Fine-Grained Labels, NLG Evaluation, and Data Augmentation
  
  Shuohang Wang, Ruochen Xu, Yang Liu, Chenguang Zhu, Michael Zeng
  
  Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing | December 2022
  
  PDF
- Empowering Language Models with Knowledge Graph Reasoning for Open-Domain Question Answering
  
  Ziniu Hu, Yichong Xu, Wenhao Yu, Shuohang Wang, Ziyi Yang, Chenguang Zhu, Kai-Wei Chang, Yizhou Sun
  
  2022 Empirical Methods in Natural Language Processing | December 2022
  
  Best Paper Award at SoCal NLP Symposium 2022
  
  PDF Publication
- Task Compass: Scaling Multi-task Pre-training with Task Prefix
  
  Zhuosheng Zhang, Shuohang Wang, Yichong Xu, Yuwei Fang, Wenhao Yu, Yang Liu, Hai Zhao, Chenguang Zhu, Michael Zeng
  
  EMNLP 2022 | December 2022
  
  Publication
- AdaPrompt: Adaptive Model Training for Prompt-based NLP
  
  Yulong Chen, Yang Liu, Li Dong, Shuohang Wang, Chenguang Zhu, Michael Zeng, Yue Zhang
  
  Findings of Empirical Methods in Natural Language Processing (EMNLP), Abu Dhabi, the United Arab Emirates, 2022. | December 2022
  
  Publication
- Language Models with Image Descriptors are Strong Few-Shot Video-Language Learners
  
  Zhenhailong Wang, Manling Li, Ruochen Xu, Luowei Zhou, Jie Lei, Xudong Lin, Shuohang Wang, Ziyi Yang, Chenguang Zhu, Derek Hoiem, Shih-Fu Chang, Mohit Bansal, Heng Ji
  
  Conference on Neural Information Processing Systems (NeurIPS) 2022 | November 2022
  
  PDF
- Human Parity on CommonsenseQA: Augmenting Self-Attention with External Attention
  
  Yichong Xu, Chenguang Zhu, Shuohang Wang, Siqi Sun, Hao Cheng, Xiaodong Liu, Jianfeng Gao, Pengcheng He, Michael Zeng, Xuedong Huang
  
  IJCAI 2022 | July 2022
  
  human parity result on CommonsenseQA
  
  Publication
- CLIP-Event: Connecting Text and Images with Event Structures
  
  Manling Li, Ruochen Xu, Shuohang Wang, Luowei Zhou, Xudong Lin, Chenguang Zhu, Michael Zeng, Heng Ji, Shih-Fu Chang
  
  Conference on Computer Vision and Pattern Recognition (CVPR), 2022. | June 2022
  
  Publication
- An Empirical Study of Training End-to-End Vision-and-Language Transformers
  
  Zi-Yi Dou, Yichong Xu, Zhe Gan, Jianfeng Wang, Shuohang Wang (shuowa), Lijuan Wang (lijuanw), Chenguang Zhu, Nanyun (Violet) Peng, Zicheng Liu, Michael Zeng
  
  CVPR | June 2022
  
  Publication Publication 项目
- Training Data is More Valuable than You Think: A Simple and Effective Method by Retrieving from Training Data
  
  Shuohang Wang, Yichong Xu, Yuwei Fang, Yang Liu, Siqi Sun, Ruochen Xu, Chenguang Zhu, Michael Zeng
  
  ACL 2022 | May 2022
  
  Publication
- Leveraging Knowledge in Multilingual Commonsense Reasoning
  
  Yuwei Fang, Shuohang Wang, Yichong Xu, Ruochen Xu, Siqi Sun, Chenguang Zhu, Michael Zeng
  
  ACL 2022 | May 2022
  
  Publication
- DICT-BERT: Enhancing Language Model Pre-Training with Dictionary
  
  Wenhao Yu, Chenguang Zhu, Yuwei Fang, Donghan Yu, Shuohang Wang, Yichong Xu, Michael Zeng, Meng Jiang
  
  ACL 2022 | May 2022
  
  Preprint
- KG-FiD: Infusing Knowledge Graph in Fusion-in-Decoder for Open-Domain Question Answering
  
  Donghan Yu, Chenguang Zhu, Yuwei Fang, Wenhao Yu, Shuohang Wang, Yichong Xu, Xiang Ren, Yiming Yang, Michael Zeng
  
  ACL 2022 | May 2022
  
  Publication
- Playing Lottery Tickets with Vision and Language
  
  Zhe Gan, Yen-Chun Chen, Linjie Li, Tianlong Chen, Yu Cheng, Shuohang Wang, Jingjing Liu, Lijuan Wang, Zicheng Liu
  
  AAAI 2022 | February 2022
  
  Publication 项目
- MLP Architectures for Vision-and-Language Modeling: An Empirical Study
  
  Yixin Nie, Linjie Li, Zhe Gan, Shuohang Wang, Chenguang Zhu, Michael Zeng, Zicheng Liu, Mohit Bansal, Lijuan Wang
  
  December 2021
  
  Preprint 项目
- Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Models
  
  Boxin Wang, Chejian Xu, Shuohang Wang, Zhe Gan, Yu Cheng, Jianfeng Gao, Ahmed Awadallah, Bo Li
  
  35th Annual Conference on Neural Information Processing System (NeurIPS 2021) | December 2021
  
  Publication
- NOAHQA: Numerical Reasoning with Interpretable Graph Question Answering Dataset
  
  Qiyuan Zhang, Lei Wang, Sicheng Yu, Shuohang Wang, Yang Wang, Jing Jiang, Ee-Peng Lim
  
  2021 Empirical Methods in Natural Language Processing | September 2021
  
  Publication
- Accelerating Real-Time Question Answering via Question Generation
  
  Yuwei Fang, Shuohang Wang, Zhe Gan, Siqi Sun, Jingjing Liu, Chenguang Zhu
  
  arXiv: Computation and Language | September 2021
- Cluster-Former: Clustering-based Sparse Transformer for Long-Range Dependency Encoding
  
  Shuohang Wang, Luowei Zhou, Zhe Gan, Yen-Chun Chen, Siqi Sun, Yuwei Fang, Yu Cheng, Jingjing Liu
  
  ACL-IJCNLP 2021 | August 2021
- EarlyBERT: Efficient BERT Training via Early-bird Lottery Tickets
  
  Xiaohan Chen, Yu Cheng, Shuohang Wang, Zhe Gan, Atlas Wang, Jingjing Liu
  
  ACL-IJCNLP 2021 | August 2021
  
  项目
- Want To Reduce Labeling Cost? GPT-3 Can Help
  
  Shuohang Wang, Yang Liu, Yichong Xu, Chenguang Zhu, Michael Zeng
  
  Conference on Empirical Methods in Natural Language Processing | August 2021
  
  Publication
- UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-training
  
  Mingyang Zhou, Luowei Zhou, Shuohang Wang, Yu Cheng, Linjie Li, Zhou Yu, Jingjing Liu
  
  IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2021) | June 2021
  
  Publication
- InfoBERT: Improving Robustness of Language Models from An Information Theoretic Perspective
  
  Boxin Wang, Shuohang Wang, Yu Cheng, Zhe Gan, Ruoxi Jia, Bo Li, JJ (Jingjing) Liu
  
  International Conference on Learning Representations (ICLR 2021) | March 2021
  
  Publication
- Language Models with Image Descriptors are Strong Few-Shot Video-Language Learners
  
  Zhenhailong Wang, Manling Li, Ruochen Xu, Luowei Zhou, Jie Lei, Xudong Lin, Shuohang Wang, Ziyi Yang, Chenguang Zhu, Derek Hoiem, Shih-Fu Chang, Mohit Bansal, Heng Ji
  
  Conference on Neural Information Processing Systems (NeurIPS) 2022 | November 2022
  
  PDF
- CLIP-Event: Connecting Text and Images with Event Structures
  
  Manling Li, Ruochen Xu, Shuohang Wang, Luowei Zhou, Xudong Lin, Chenguang Zhu, Michael Zeng, Heng Ji, Shih-Fu Chang
  
  Conference on Computer Vision and Pattern Recognition (CVPR), 2022. | June 2022
  
  Publication
- An Empirical Study of Training End-to-End Vision-and-Language Transformers
  
  Zi-Yi Dou, Yichong Xu, Zhe Gan, Jianfeng Wang, Shuohang Wang (shuowa), Lijuan Wang (lijuanw), Chenguang Zhu, Nanyun (Violet) Peng, Zicheng Liu, Michael Zeng
  
  CVPR | June 2022
  
  Publication Publication 项目
- Playing Lottery Tickets with Vision and Language
  
  Zhe Gan, Yen-Chun Chen, Linjie Li, Tianlong Chen, Yu Cheng, Shuohang Wang, Jingjing Liu, Lijuan Wang, Zicheng Liu
  
  AAAI 2022 | February 2022
  
  Publication 项目
- MLP Architectures for Vision-and-Language Modeling: An Empirical Study
  
  Yixin Nie, Linjie Li, Zhe Gan, Shuohang Wang, Chenguang Zhu, Michael Zeng, Zicheng Liu, Mohit Bansal, Lijuan Wang
  
  December 2021
  
  Preprint 项目
- The Elastic Lottery Ticket Hypothesis
  
  Xiaohan Chen, Yu Cheng, Shuohang Wang, Zhe Gan, Jingjing Liu, Zhangyang Wang
  
  35th Annual Conference on Neural Information Processing System (NeurIPS 2021) | December 2021
  
  Publication 项目
- UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-training
  
  Mingyang Zhou, Luowei Zhou, Shuohang Wang, Yu Cheng, Linjie Li, Zhou Yu, Jingjing Liu
  
  IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2021) | June 2021
  
  Publication

- Decoder-Hybrid-Decoder Architecture for Efficient Reasoning with Long Generation
  
  Liliang Ren, Congcong Chen, Haoran Xu, Young Jin Kim, Adam Atkinson, Zheng Zhan, Jiankai Sun, Baolin Peng, Liyuan Liu, Shuohang Wang, Hao Cheng, Jianfeng Gao, Weizhu Chen, Yelong Shen
  
  NeurIPS 2025 | July 2025
  
  DOI
- Routing Mamba: Scaling State Space Models with Mixture-of-Experts Projection
  
  Zheng Zhan, Liliang Ren, Shuohang Wang, Liyuan Liu, Yang Liu, Yeyun Gong, Yanzhi Wang, Yelong Shen
  
  NeurIPS 2025 | June 2025
  
  DOI Publication Publication
- Reinforcement Learning for Reasoning in Large Language Models with One Training Example
  
  Yiping Wang, Qing Yang, Zhiyuan Zeng, Liliang Ren, Liyuan Liu, Baolin Peng, Hao Cheng, Xuehai He, Kuan Wang, Jianfeng Gao, Weizhu Chen, Shuohang Wang, S. Du, Yelong Shen
  
  NeurIPS 2025 | April 2025
  
  DOI Publication Publication
- SciAgent: Tool-augmented Language Models for Scientific Reasoning
  
  Yubo Ma, Zhibin Gou, Junheng Hao, Ruochen Xu, Shuohang Wang, Liangming Pan, Yujiu Yang, Yixin Cao, Aixin Sun
  
  2024 Empirical Methods in Natural Language Processing | November 2024
  
  DOI Publication
- Sparse Modular Activation for Efficient Sequence Modeling
  
  Liliang Ren, Yang Liu, Shuohang Wang, Yichong Xu, Chenguang Zhu, ChengXiang Zhai
  
  NeurIPS 2023 | July 2023
  
  Publication
- Prompting GPT-3 To Be Reliable
  
  Silei Cheng, Zhe Gan, Zhengyuan Yang, Shuohang Wang, Jianfeng Wang, Jordan Boyd-Graber, Lijuan Wang
  
  International Conference on Learning Representations (ICLR 23) | May 2023
  
  PDF
- Generate rather than Retrieve: Large Language Models are Strong Context Generators
  
  Wenhao Yu, Dan Iter, Shuohang Wang, Yichong Xu, Mingxuan Ju, Soumya Sanyal, Chenguang Zhu, Michael Zeng, Meng Jiang
  
  ICLR 2023 | May 2023
  
  Publication
- ParaTag: A Dataset of Paraphrase Tagging for Fine-Grained Labels, NLG Evaluation, and Data Augmentation
  
  Shuohang Wang, Ruochen Xu, Yang Liu, Chenguang Zhu, Michael Zeng
  
  Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing | December 2022
  
  PDF
- Empowering Language Models with Knowledge Graph Reasoning for Open-Domain Question Answering
  
  Ziniu Hu, Yichong Xu, Wenhao Yu, Shuohang Wang, Ziyi Yang, Chenguang Zhu, Kai-Wei Chang, Yizhou Sun
  
  2022 Empirical Methods in Natural Language Processing | December 2022
  
  Best Paper Award at SoCal NLP Symposium 2022
  
  PDF Publication
- Task Compass: Scaling Multi-task Pre-training with Task Prefix
  
  Zhuosheng Zhang, Shuohang Wang, Yichong Xu, Yuwei Fang, Wenhao Yu, Yang Liu, Hai Zhao, Chenguang Zhu, Michael Zeng
  
  EMNLP 2022 | December 2022
  
  Publication
- AdaPrompt: Adaptive Model Training for Prompt-based NLP
  
  Yulong Chen, Yang Liu, Li Dong, Shuohang Wang, Chenguang Zhu, Michael Zeng, Yue Zhang
  
  Findings of Empirical Methods in Natural Language Processing (EMNLP), Abu Dhabi, the United Arab Emirates, 2022. | December 2022
  
  Publication
- Language Models with Image Descriptors are Strong Few-Shot Video-Language Learners
  
  Zhenhailong Wang, Manling Li, Ruochen Xu, Luowei Zhou, Jie Lei, Xudong Lin, Shuohang Wang, Ziyi Yang, Chenguang Zhu, Derek Hoiem, Shih-Fu Chang, Mohit Bansal, Heng Ji
  
  Conference on Neural Information Processing Systems (NeurIPS) 2022 | November 2022
  
  PDF
- Human Parity on CommonsenseQA: Augmenting Self-Attention with External Attention
  
  Yichong Xu, Chenguang Zhu, Shuohang Wang, Siqi Sun, Hao Cheng, Xiaodong Liu, Jianfeng Gao, Pengcheng He, Michael Zeng, Xuedong Huang
  
  IJCAI 2022 | July 2022
  
  human parity result on CommonsenseQA
  
  Publication
- CLIP-Event: Connecting Text and Images with Event Structures
  
  Manling Li, Ruochen Xu, Shuohang Wang, Luowei Zhou, Xudong Lin, Chenguang Zhu, Michael Zeng, Heng Ji, Shih-Fu Chang
  
  Conference on Computer Vision and Pattern Recognition (CVPR), 2022. | June 2022
  
  Publication
- An Empirical Study of Training End-to-End Vision-and-Language Transformers
  
  Zi-Yi Dou, Yichong Xu, Zhe Gan, Jianfeng Wang, Shuohang Wang (shuowa), Lijuan Wang (lijuanw), Chenguang Zhu, Nanyun (Violet) Peng, Zicheng Liu, Michael Zeng
  
  CVPR | June 2022
  
  Publication Publication 项目
- Training Data is More Valuable than You Think: A Simple and Effective Method by Retrieving from Training Data
  
  Shuohang Wang, Yichong Xu, Yuwei Fang, Yang Liu, Siqi Sun, Ruochen Xu, Chenguang Zhu, Michael Zeng
  
  ACL 2022 | May 2022
  
  Publication
- Leveraging Knowledge in Multilingual Commonsense Reasoning
  
  Yuwei Fang, Shuohang Wang, Yichong Xu, Ruochen Xu, Siqi Sun, Chenguang Zhu, Michael Zeng
  
  ACL 2022 | May 2022
  
  Publication
- DICT-BERT: Enhancing Language Model Pre-Training with Dictionary
  
  Wenhao Yu, Chenguang Zhu, Yuwei Fang, Donghan Yu, Shuohang Wang, Yichong Xu, Michael Zeng, Meng Jiang
  
  ACL 2022 | May 2022
  
  Preprint
- KG-FiD: Infusing Knowledge Graph in Fusion-in-Decoder for Open-Domain Question Answering
  
  Donghan Yu, Chenguang Zhu, Yuwei Fang, Wenhao Yu, Shuohang Wang, Yichong Xu, Xiang Ren, Yiming Yang, Michael Zeng
  
  ACL 2022 | May 2022
  
  Publication
- Playing Lottery Tickets with Vision and Language
  
  Zhe Gan, Yen-Chun Chen, Linjie Li, Tianlong Chen, Yu Cheng, Shuohang Wang, Jingjing Liu, Lijuan Wang, Zicheng Liu
  
  AAAI 2022 | February 2022
  
  Publication 项目
- Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Models
  
  Boxin Wang, Chejian Xu, Shuohang Wang, Zhe Gan, Yu Cheng, Jianfeng Gao, Ahmed Awadallah, Bo Li
  
  35th Annual Conference on Neural Information Processing System (NeurIPS 2021) | December 2021
  
  Publication
- The Elastic Lottery Ticket Hypothesis
  
  Xiaohan Chen, Yu Cheng, Shuohang Wang, Zhe Gan, Jingjing Liu, Zhangyang Wang
  
  35th Annual Conference on Neural Information Processing System (NeurIPS 2021) | December 2021
  
  Publication 项目
- NOAHQA: Numerical Reasoning with Interpretable Graph Question Answering Dataset
  
  Qiyuan Zhang, Lei Wang, Sicheng Yu, Shuohang Wang, Yang Wang, Jing Jiang, Ee-Peng Lim
  
  2021 Empirical Methods in Natural Language Processing | September 2021
  
  Publication
- Cluster-Former: Clustering-based Sparse Transformer for Long-Range Dependency Encoding
  
  Shuohang Wang, Luowei Zhou, Zhe Gan, Yen-Chun Chen, Siqi Sun, Yuwei Fang, Yu Cheng, Jingjing Liu
  
  ACL-IJCNLP 2021 | August 2021
- EarlyBERT: Efficient BERT Training via Early-bird Lottery Tickets
  
  Xiaohan Chen, Yu Cheng, Shuohang Wang, Zhe Gan, Atlas Wang, Jingjing Liu
  
  ACL-IJCNLP 2021 | August 2021
  
  项目
- Want To Reduce Labeling Cost? GPT-3 Can Help
  
  Shuohang Wang, Yang Liu, Yichong Xu, Chenguang Zhu, Michael Zeng
  
  Conference on Empirical Methods in Natural Language Processing | August 2021
  
  Publication
- UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-training
  
  Mingyang Zhou, Luowei Zhou, Shuohang Wang, Yu Cheng, Linjie Li, Zhou Yu, Jingjing Liu
  
  IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2021) | June 2021
  
  Publication
- InfoBERT: Improving Robustness of Language Models from An Information Theoretic Perspective
  
  Boxin Wang, Shuohang Wang, Yu Cheng, Zhe Gan, Ruoxi Jia, Bo Li, JJ (Jingjing) Liu
  
  International Conference on Learning Representations (ICLR 2021) | March 2021
  
  Publication
- Cross-Thought for Sentence Encoder Pre-training
  
  Shuohang Wang, Yuwei Fang, Siqi Sun, Zhe Gan, Yu Cheng, Jing Jiang, JJ (Jingjing) Liu
  
  The 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP 2020) | October 2020
  
  Publication
- Multi-Fact Correction in Abstractive Text Summarization
  
  Yue Dong, Shuohang Wang, Zhe Gan, Yu Cheng, Jackie Chi Kit Cheung, JJ (Jingjing) Liu
  
  The 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP 2020) | October 2020
  
  Publication
- Contrastive Distillation on Intermediate Representations for Language Model Compression
  
  Siqi Sun, Zhe Gan, Yu Cheng, Yuwei Fang, Shuohang Wang, JJ (Jingjing) Liu
  
  The 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP 2020) | October 2020
  
  Publication
- Hierarchical Graph Network for Multi-hop Question Answering
  
  Yuwei Fang, Siqi Sun, Zhe Gan, Rohit Pillai, Shuohang Wang, JJ (Jingjing) Liu
  
  2020 Empirical Methods in Natural Language Processing | April 2020
  
  DOI PDF Github
- In-Context Demonstration Selection with Cross Entropy Difference
  
  Dan Iter, Reid Pryzant, Ruochen Xu, Shuohang Wang, Yang Liu, Chenguang Zhu
  
  May 2023
  
  Arxiv
  
  Publication
- PEARL: Prompting Large Language Models to Plan and Execute Actions Over Long Documents
  
  Simeng Sun, Yang Liu, Shuohang Wang, Chenguang Zhu, Mohit Iyyer
  
  May 2023
  
  Publication
- LMGQS: A Large-scale Dataset for Query-focused Summarization
  
  Ruochen Xu, Song Wang, Yang Liu, Shuohang Wang, Yichong Xu, Dan Iter, Chenguang Zhu, Michael Zeng
  
  May 2023
  
  PDF
- InheritSumm: A General, Versatile and Compact Summarizer by Distilling from GPT
  
  Yichong Xu, Ruochen Xu, Dan Iter, Yang Liu, Shuohang Wang, Chenguang Zhu, Michael Zeng
  
  May 2023
  
  PDF
- MLP Architectures for Vision-and-Language Modeling: An Empirical Study
  
  Yixin Nie, Linjie Li, Zhe Gan, Shuohang Wang, Chenguang Zhu, Michael Zeng, Zicheng Liu, Mohit Bansal, Lijuan Wang
  
  December 2021
  
  Preprint 项目
- G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment
  
  Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, Chenguang Zhu
  
  arXiv 2303.16634 | March 2023
  
  Publication
- Accelerating Real-Time Question Answering via Question Generation
  
  Yuwei Fang, Shuohang Wang, Zhe Gan, Siqi Sun, Jingjing Liu, Chenguang Zhu
  
  arXiv: Computation and Language | September 2021
- LightningDOT: Pre-training Visual-Semantic Embeddings for Real-Time Image-Text Retrieval
  
  Siqi Sun, Yen-Chun Chen, Linjie Li, Shuohang Wang, Yuwei Fang, JJ (Jingjing) Liu
  
  March 2021
  
  Publication

Shuohang Wang

论文与出版物

Decoder-Hybrid-Decoder Architecture for Efficient Reasoning with Long Generation

Routing Mamba: Scaling State Space Models with Mixture-of-Experts Projection

Reinforcement Learning for Reasoning in Large Language Models with One Training Example

SciAgent: Tool-augmented Language Models for Scientific Reasoning

Sparse Modular Activation for Efficient Sequence Modeling

In-Context Demonstration Selection with Cross Entropy Difference

PEARL: Prompting Large Language Models to Plan and Execute Actions Over Long Documents

Generate rather than Retrieve: Large Language Models are Strong Context Generators

InheritSumm: A General, Versatile and Compact Summarizer by Distilling from GPT

LMGQS: A Large-scale Dataset for Query-focused Summarization

Prompting GPT-3 To Be Reliable

G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment

AdaPrompt: Adaptive Model Training for Prompt-based NLP

Empowering Language Models with Knowledge Graph Reasoning for Open-Domain Question Answering

ParaTag: A Dataset of Paraphrase Tagging for Fine-Grained Labels, NLG Evaluation, and Data Augmentation

Task Compass: Scaling Multi-task Pre-training with Task Prefix

Language Models with Image Descriptors are Strong Few-Shot Video-Language Learners

Human Parity on CommonsenseQA: Augmenting Self-Attention with External Attention

An Empirical Study of Training End-to-End Vision-and-Language Transformers

CLIP-Event: Connecting Text and Images with Event Structures

DICT-BERT: Enhancing Language Model Pre-Training with Dictionary

KG-FiD: Infusing Knowledge Graph in Fusion-in-Decoder for Open-Domain Question Answering

Leveraging Knowledge in Multilingual Commonsense Reasoning

Training Data is More Valuable than You Think: A Simple and Effective Method by Retrieving from Training Data

Playing Lottery Tickets with Vision and Language

Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Models

MLP Architectures for Vision-and-Language Modeling: An Empirical Study

The Elastic Lottery Ticket Hypothesis

NOAHQA: Numerical Reasoning with Interpretable Graph Question Answering Dataset

Accelerating Real-Time Question Answering via Question Generation

Cluster-Former: Clustering-based Sparse Transformer for Long-Range Dependency Encoding

EarlyBERT: Efficient BERT Training via Early-bird Lottery Tickets

Want To Reduce Labeling Cost? GPT-3 Can Help

UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-training

LightningDOT: Pre-training Visual-Semantic Embeddings for Real-Time Image-Text Retrieval

InfoBERT: Improving Robustness of Language Models from An Information Theoretic Perspective

Contrastive Distillation on Intermediate Representations for Language Model Compression

Cross-Thought for Sentence Encoder Pre-training

Multi-Fact Correction in Abstractive Text Summarization

Hierarchical Graph Network for Multi-hop Question Answering

Decoder-Hybrid-Decoder Architecture for Efficient Reasoning with Long Generation

Routing Mamba: Scaling State Space Models with Mixture-of-Experts Projection

Reinforcement Learning for Reasoning in Large Language Models with One Training Example

SciAgent: Tool-augmented Language Models for Scientific Reasoning

Sparse Modular Activation for Efficient Sequence Modeling

In-Context Demonstration Selection with Cross Entropy Difference

PEARL: Prompting Large Language Models to Plan and Execute Actions Over Long Documents

LMGQS: A Large-scale Dataset for Query-focused Summarization

InheritSumm: A General, Versatile and Compact Summarizer by Distilling from GPT

Prompting GPT-3 To Be Reliable

ParaTag: A Dataset of Paraphrase Tagging for Fine-Grained Labels, NLG Evaluation, and Data Augmentation

Empowering Language Models with Knowledge Graph Reasoning for Open-Domain Question Answering

Task Compass: Scaling Multi-task Pre-training with Task Prefix

Human Parity on CommonsenseQA: Augmenting Self-Attention with External Attention

CLIP-Event: Connecting Text and Images with Event Structures

Training Data is More Valuable than You Think: A Simple and Effective Method by Retrieving from Training Data

DICT-BERT: Enhancing Language Model Pre-Training with Dictionary

Playing Lottery Tickets with Vision and Language

MLP Architectures for Vision-and-Language Modeling: An Empirical Study

Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Models

The Elastic Lottery Ticket Hypothesis

Cluster-Former: Clustering-based Sparse Transformer for Long-Range Dependency Encoding

EarlyBERT: Efficient BERT Training via Early-bird Lottery Tickets

Want To Reduce Labeling Cost? GPT-3 Can Help

UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-training

LightningDOT: Pre-training Visual-Semantic Embeddings for Real-Time Image-Text Retrieval

InfoBERT: Improving Robustness of Language Models from An Information Theoretic Perspective

Cross-Thought for Sentence Encoder Pre-training

Multi-Fact Correction in Abstractive Text Summarization

Contrastive Distillation on Intermediate Representations for Language Model Compression

Hierarchical Graph Network for Multi-hop Question Answering

SciAgent: Tool-augmented Language Models for Scientific Reasoning

PEARL: Prompting Large Language Models to Plan and Execute Actions Over Long Documents

LMGQS: A Large-scale Dataset for Query-focused Summarization

Prompting GPT-3 To Be Reliable

Generate rather than Retrieve: Large Language Models are Strong Context Generators

G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment

ParaTag: A Dataset of Paraphrase Tagging for Fine-Grained Labels, NLG Evaluation, and Data Augmentation