Efficiently and effectively scaling up language model pretraining for best language representation model on GLUE and SuperGLUE
As part of Microsoft AI at Scale (opens in new tab), the Turing family of NLP models are being used at scale across Microsoft to enable the next generation of AI experiences. Today, we are…
GoodPoints: Distribution Compression in Near-linear Time
A Python package for generating concise, high-quality summaries of a probability distribution. GoodPoints is a collection of tools for compressing a distribution more effectively than independent sampling.
TorchGeo
TorchGeo is a PyTorch (opens in new tab) domain library, similar to torchvision (opens in new tab), that provides datasets, transforms, samplers, and pre-trained models specific to geospatial data. The goal of this library is to make it…
A law of robustness and the importance of overparametrization in deep learning
Microsoft Research Senior Principal Researcher Sebastien Bubeck answers several questions about the NeurIPS 2021 paper, “A Universal Law of Robustness via Isoperimetry.” He discusses what overparametrization is, the law of robustness, and how these two…