DeepSpeed: Advancing MoE inference and training to power next-generation AI scale
In the last three years, the largest trained dense models have increased in size by over 1,000 times, from a few hundred million parameters to over 500 billion parameters in Megatron-Turing NLG 530B (MT-NLG). Improvements…
AI meets PL
In this area of research, we broadly explore combining machine learning and program synthesis in various ways. This is an umbrella project that has spawned several projects exploring applications of such a combination in different…
Reliable Machine Learning
Note: This research project has reached its conclusion. These pages are maintained for reference and archival purposes. High-stakes decision-making in areas like healthcare, finance and governance requires accountability for decisions and for how data is…
Extreme Classification
xtreme classification is a rapidly growing research area in computer vision focusing on multi-class and multi-label problems involving an extremely large number of labels (ranging from thousands to billions).