Pretrainer’s Guide to Training Data: Measuring Effects of Age, Domain Coverage, Quality, & Toxicity
Pretraining is the preliminary and fundamental step in developing capable language models (LM). Despite this, pretraining data design is critically under-documented and often guided by empirically unsupported intuitions. To address this, we pretrain 28 1.5B…
Microsoft Research Forum Episode 4: The future of multimodal models, a new “small” language model, and other AI updates
Explore multimodal & small language models, plus advanced benchmarks for AI evaluation. Microsoft researchers are working on breakthroughs in weather prediction, materials design, even a new kind of computer for AI inference and hard optimization…
AutoVerus
Automatically synthesize proof annotations that help Verus prove the correctness of Rust code.
Research Focus: Week of September 23, 2024
Welcome to Research Focus, a series of blog posts that highlights notable publications, events, code/datasets, new hires and other milestones from across the research community at Microsoft. Time-series forecasting is a technique used to predict…
Research Intern for Visual Computing Group
Number of Openings: 2-4 Location: Beijing/Shanghai General Information: The Visual Computing Group at Microsoft Research Asia is seeking motivated and talented interns with expertise in computer vision and machine learning. This internship offers an exceptional…
Research Intern for Social Computing Group
Number of Openings: 2-4 Location: Beijing Group Introduction: The Social Computing Group at Microsoft Research Asia is dedicated to integrating computer science with social sciences, deeply exploring the profound impact of artificial intelligence on society and human…