Artificial Intelligence
Building intelligent machines that complement human reasoning and extend what people can do.
Microsoft Research Blog
Eureka: Evaluating and understanding progress in AI
How can we rigorously evaluate and understand state-of-the-art progress in AI? Eureka is an open-source framework for standardizing evaluations of large foundation models, beyond single-score reporting and rankings. Learn more about the extended findings.
Tool
Eureka ML Insights
This repository contains the code for the Eureka ML Insights, a framework for standardizing evaluations of large foundation models, beyond single-score reporting and rankings. The framework is designed to help researchers and practitioners run reproducible evaluations…
Microsoft Research Blog
Research Focus: Week of September 9, 2024
Investigating vulnerabilities in LLMs; A novel total-duration-aware (TDA) duration model for text-to-speech (TTS); Generative expert metric system through iterative prompt priming; Integrity protection in 5G fronthaul networks:
Microsoft Research Blog
MedFuzz: Exploring the robustness of LLMs on medical challenge problems
Medfuzz tests LLMs by breaking benchmark assumptions, exposing vulnerabilities to bolster real-world accuracy.
Project
Auto-Tuning of Copilot Prompts
AI-agent-based tool for tuning and optimizing system prompts, and particularly those of Microsoft Copilot offerings. It is used to replace manual adaptation of prompts — requiring extensive human labor — in scenarios such as when…