Artificial Intelligence
Building intelligent machines that complement human reasoning and extend what people can do.
Tool
MInference: Accelerating Pre-filling for Long-context LLMs via Dynamic Sparse Attention
MInference 1.0 leverages the dynamic sparse nature of LLMs’ attention, which exhibits some static patterns, to speed up the pre-filling for long-context LLMs. It first determines offline which sparse pattern each head belongs to, then…
Project
GenAIScript: Scripting for Generative AI
GenAIScript is scripting language where AI and foundation models are first-class entities, enabling a wide range of users to enhance their workflows with AI capabilities.