Tool
LongRoPE
LongRoPE is a novel method that extends the context window of pre-trained LLMs to an impressive 2048k tokens by non-uniformly rescaling RoPE positional embeddings. LongRoPE has been integrated into Microsoft Phi-3.
Microsoft Research Blog
Microsoft at ICML 2024: Innovations in machine learning
The competitive dynamics of AI agents and a method for learning and applying temporal action abstractions represent just some of Microsoft’s contributions to ICML 2024.
Publication
MGit: A Model Versioning and Management System
Publication
Stealing Part of a Production Language Model
Event
ICML 2024
Microsoft is proud to be a sponsor of The International Conference on Machine Learning (ICML) (opens in new tab), a premier gathering of professionals dedicated to the advancement of the branch of artificial intelligence known…