Jean Garcia-Gathright

Principal Applied Scientist

论文与出版物

查看方式：

- What AI Benchmarks Actually Measure: Adapting Convergent and Discriminant Validity to Interrogate Fifty-Six AI Benchmarks
  
  Meera Desai, Sang T. Truong, Hanna Wallach, Alexandra Chouldechova, A. Feder Cooper, Jean Garcia-Gathright, Daniel E. Ho, Abigail Z. Jacobs, Sanmi Koyejo, Nicholas Pangakis, Angelina Wang
  
  September 2026
  
  arXiv | September 2026
  
  Publication
- ASSERT: A Measurement Pipeline for GenAI Audits
  
  Riccardo Fogliato, Abhinav Palia, Xiawei Wang, Emily Sheng, Chad Atalla, Jean Garcia-Gathright, Nicholas Pangakis, Sharman Tan, Dan Vann, Hannah Washington, P. A. Dow, Heba Elfardy, Hanna Wallach, Sandeep Atluri
  
  August 2026
  
  arXiv | August 2026
  
  Publication
- Validating and Refining Measurements for Generative AI Evaluation Via Stakeholder Engagement
  
  Tonya Nguyen, Jean Garcia-Gathright, Hannah Washington, Alexandra Chouldechova, Hanna Wallach, Jennifer Wortman Vaughan
  
  The 2026 ACM Conference on Fairness, Accountability, and Transparency | June 2026
  
  Conference on Fairness, Accountability and Transparency
  
  Publication
- AI-Assisted Systematization for Evaluating GenAI Systems
  
  Dhruv Agarwal, Emily Sheng, Chad Atalla, Jean Garcia-Gathright, Hussein Mozannar, Hannah Washington, Alexandra Chouldechova, Solon Barocas, Hanna Wallach
  
  May 2026
  
  arXiv | May 2026
  
  Publication
- Understanding and Meeting Practitioner Needs When Measuring Representational Harms Caused by LLM-Based Systems
  
  Emma Harvey, Emily Sheng, Su Lin Blodgett, Alex Chouldechova, Jean Garcia-Gathright, Alexandra Olteanu, Hanna Wallach
  
  Findings of the Association for Computational Linguistics: ACL 2025 | July 2025
  
  Publication
- Taxonomizing Representational Harms using Speech Act Theory
  
  Emily Corvi, Hannah Washington, Stefanie Reed, Chad Atalla, Alex Chouldechova, Alex Dow, Jean Garcia-Gathright, Nick Pangakis, Emily Sheng, Dan Vann, Matthew Vogel, Hanna Wallach
  
  March 2025
  
  March 2025
  
  Publication
- Position: Evaluating Generative AI Systems is a Social Science Measurement Challenge
  
  Hanna Wallach, Meera Desai, A. Feder Cooper, Angelina Wang, Chad Atalla, Solon Barocas, Su Lin Blodgett, Alex Chouldechova, Emily Corvi, Alex Dow, Jean Garcia-Gathright, Alexandra Olteanu, Nick Pangakis, Stefanie Reed, Emily Sheng, Dan Vann, Jennifer Wortman Vaughan, Matthew Vogel, Hannah Washington, Abigail Z. Jacobs
  
  ICML 2025 | January 2025
  
  Publication
- A Shared Standard for Valid Measurement of Generative AI Systems’ Capabilities, Risks, and Impacts
  
  Alex Chouldechova, Chad Atalla, Solon Barocas, A. Feder Cooper, Emily Corvi, Alex Dow, Jean Garcia-Gathright, Nick Pangakis, Stefanie Reed, Emily Sheng, Dan Vann, Matthew Vogel, Hannah Washington, Hanna Wallach
  
  December 2024
  
  December 2024
  
  Publication
- Gaps Between Research and Practice When Measuring Representational Harms Caused by LLM-Based Systems
  
  Emma Harvey, Emily Sheng, Su Lin Blodgett, Alex Chouldechova, Jean Garcia-Gathright, Alexandra Olteanu, Hanna Wallach
  
  November 2024
  
  November 2024
  
  Publication
- Evaluating Generative AI Systems is a Social Science Measurement Challenge
  
  Hanna Wallach, Meera Desai, Nick Pangakis, A. Feder Cooper, Angelina Wang, Solon Barocas, Alex Chouldechova, Chad Atalla, Su Lin Blodgett, Emily Corvi, Alex Dow, Jean Garcia-Gathright, Alexandra Olteanu, Stefanie Reed, Emily Sheng, Dan Vann, Jennifer Wortman Vaughan, Matthew Vogel, Hannah Washington, Abigail Z. Jacobs
  
  November 2024
  
  November 2024
  
  Publication

- Validating and Refining Measurements for Generative AI Evaluation Via Stakeholder Engagement
  
  Tonya Nguyen, Jean Garcia-Gathright, Hannah Washington, Alexandra Chouldechova, Hanna Wallach, Jennifer Wortman Vaughan
  
  The 2026 ACM Conference on Fairness, Accountability, and Transparency | June 2026
  
  Conference on Fairness, Accountability and Transparency
  
  Publication
- What AI Benchmarks Actually Measure: Adapting Convergent and Discriminant Validity to Interrogate Fifty-Six AI Benchmarks
  
  Meera Desai, Sang T. Truong, Hanna Wallach, Alexandra Chouldechova, A. Feder Cooper, Jean Garcia-Gathright, Daniel E. Ho, Abigail Z. Jacobs, Sanmi Koyejo, Nicholas Pangakis, Angelina Wang
  
  September 2026
  
  arXiv | September 2026
  
  Publication
- Validating and Refining Measurements for Generative AI Evaluation Via Stakeholder Engagement
  
  Tonya Nguyen, Jean Garcia-Gathright, Hannah Washington, Alexandra Chouldechova, Hanna Wallach, Jennifer Wortman Vaughan
  
  The 2026 ACM Conference on Fairness, Accountability, and Transparency | June 2026
  
  Conference on Fairness, Accountability and Transparency
  
  Publication
- Understanding and Meeting Practitioner Needs When Measuring Representational Harms Caused by LLM-Based Systems
  
  Emma Harvey, Emily Sheng, Su Lin Blodgett, Alex Chouldechova, Jean Garcia-Gathright, Alexandra Olteanu, Hanna Wallach
  
  Findings of the Association for Computational Linguistics: ACL 2025 | July 2025
  
  Publication
- Taxonomizing Representational Harms using Speech Act Theory
  
  Emily Corvi, Hannah Washington, Stefanie Reed, Chad Atalla, Alex Chouldechova, Alex Dow, Jean Garcia-Gathright, Nick Pangakis, Emily Sheng, Dan Vann, Matthew Vogel, Hanna Wallach
  
  March 2025
  
  March 2025
  
  Publication
- Position: Evaluating Generative AI Systems is a Social Science Measurement Challenge
  
  Hanna Wallach, Meera Desai, A. Feder Cooper, Angelina Wang, Chad Atalla, Solon Barocas, Su Lin Blodgett, Alex Chouldechova, Emily Corvi, Alex Dow, Jean Garcia-Gathright, Alexandra Olteanu, Nick Pangakis, Stefanie Reed, Emily Sheng, Dan Vann, Jennifer Wortman Vaughan, Matthew Vogel, Hannah Washington, Abigail Z. Jacobs
  
  ICML 2025 | January 2025
  
  Publication
- A Shared Standard for Valid Measurement of Generative AI Systems’ Capabilities, Risks, and Impacts
  
  Alex Chouldechova, Chad Atalla, Solon Barocas, A. Feder Cooper, Emily Corvi, Alex Dow, Jean Garcia-Gathright, Nick Pangakis, Stefanie Reed, Emily Sheng, Dan Vann, Matthew Vogel, Hannah Washington, Hanna Wallach
  
  December 2024
  
  December 2024
  
  Publication
- Gaps Between Research and Practice When Measuring Representational Harms Caused by LLM-Based Systems
  
  Emma Harvey, Emily Sheng, Su Lin Blodgett, Alex Chouldechova, Jean Garcia-Gathright, Alexandra Olteanu, Hanna Wallach
  
  November 2024
  
  November 2024
  
  Publication
- Evaluating Generative AI Systems is a Social Science Measurement Challenge
  
  Hanna Wallach, Meera Desai, Nick Pangakis, A. Feder Cooper, Angelina Wang, Solon Barocas, Alex Chouldechova, Chad Atalla, Su Lin Blodgett, Emily Corvi, Alex Dow, Jean Garcia-Gathright, Alexandra Olteanu, Stefanie Reed, Emily Sheng, Dan Vann, Jennifer Wortman Vaughan, Matthew Vogel, Hannah Washington, Abigail Z. Jacobs
  
  November 2024
  
  November 2024
  
  Publication
- What AI Benchmarks Actually Measure: Adapting Convergent and Discriminant Validity to Interrogate Fifty-Six AI Benchmarks
  
  Meera Desai, Sang T. Truong, Hanna Wallach, Alexandra Chouldechova, A. Feder Cooper, Jean Garcia-Gathright, Daniel E. Ho, Abigail Z. Jacobs, Sanmi Koyejo, Nicholas Pangakis, Angelina Wang
  
  September 2026
  
  arXiv | September 2026
  
  Publication
- ASSERT: A Measurement Pipeline for GenAI Audits
  
  Riccardo Fogliato, Abhinav Palia, Xiawei Wang, Emily Sheng, Chad Atalla, Jean Garcia-Gathright, Nicholas Pangakis, Sharman Tan, Dan Vann, Hannah Washington, P. A. Dow, Heba Elfardy, Hanna Wallach, Sandeep Atluri
  
  August 2026
  
  arXiv | August 2026
  
  Publication
- AI-Assisted Systematization for Evaluating GenAI Systems
  
  Dhruv Agarwal, Emily Sheng, Chad Atalla, Jean Garcia-Gathright, Hussein Mozannar, Hannah Washington, Alexandra Chouldechova, Solon Barocas, Hanna Wallach
  
  May 2026
  
  arXiv | May 2026
  
  Publication
- Taxonomizing Representational Harms using Speech Act Theory
  
  Emily Corvi, Hannah Washington, Stefanie Reed, Chad Atalla, Alex Chouldechova, Alex Dow, Jean Garcia-Gathright, Nick Pangakis, Emily Sheng, Dan Vann, Matthew Vogel, Hanna Wallach
  
  March 2025
  
  March 2025
  
  Publication
- Position: Evaluating Generative AI Systems is a Social Science Measurement Challenge
  
  Hanna Wallach, Meera Desai, A. Feder Cooper, Angelina Wang, Chad Atalla, Solon Barocas, Su Lin Blodgett, Alex Chouldechova, Emily Corvi, Alex Dow, Jean Garcia-Gathright, Alexandra Olteanu, Nick Pangakis, Stefanie Reed, Emily Sheng, Dan Vann, Jennifer Wortman Vaughan, Matthew Vogel, Hannah Washington, Abigail Z. Jacobs
  
  ICML 2025 | January 2025
  
  Publication
- A Shared Standard for Valid Measurement of Generative AI Systems’ Capabilities, Risks, and Impacts
  
  Alex Chouldechova, Chad Atalla, Solon Barocas, A. Feder Cooper, Emily Corvi, Alex Dow, Jean Garcia-Gathright, Nick Pangakis, Stefanie Reed, Emily Sheng, Dan Vann, Matthew Vogel, Hannah Washington, Hanna Wallach
  
  December 2024
  
  December 2024
  
  Publication
- Gaps Between Research and Practice When Measuring Representational Harms Caused by LLM-Based Systems
  
  Emma Harvey, Emily Sheng, Su Lin Blodgett, Alex Chouldechova, Jean Garcia-Gathright, Alexandra Olteanu, Hanna Wallach
  
  November 2024
  
  November 2024
  
  Publication
- Evaluating Generative AI Systems is a Social Science Measurement Challenge
  
  Hanna Wallach, Meera Desai, Nick Pangakis, A. Feder Cooper, Angelina Wang, Solon Barocas, Alex Chouldechova, Chad Atalla, Su Lin Blodgett, Emily Corvi, Alex Dow, Jean Garcia-Gathright, Alexandra Olteanu, Stefanie Reed, Emily Sheng, Dan Vann, Jennifer Wortman Vaughan, Matthew Vogel, Hannah Washington, Abigail Z. Jacobs
  
  November 2024
  
  November 2024
  
  Publication
- AI-Assisted Systematization for Evaluating GenAI Systems
  
  Dhruv Agarwal, Emily Sheng, Chad Atalla, Jean Garcia-Gathright, Hussein Mozannar, Hannah Washington, Alexandra Chouldechova, Solon Barocas, Hanna Wallach
  
  May 2026
  
  arXiv | May 2026
  
  Publication
- Understanding and Meeting Practitioner Needs When Measuring Representational Harms Caused by LLM-Based Systems
  
  Emma Harvey, Emily Sheng, Su Lin Blodgett, Alex Chouldechova, Jean Garcia-Gathright, Alexandra Olteanu, Hanna Wallach
  
  Findings of the Association for Computational Linguistics: ACL 2025 | July 2025
  
  Publication

- Validating and Refining Measurements for Generative AI Evaluation Via Stakeholder Engagement
  
  Tonya Nguyen, Jean Garcia-Gathright, Hannah Washington, Alexandra Chouldechova, Hanna Wallach, Jennifer Wortman Vaughan
  
  The 2026 ACM Conference on Fairness, Accountability, and Transparency | June 2026
  
  Conference on Fairness, Accountability and Transparency
  
  Publication
- Validating and Refining Measurements for Generative AI Evaluation Via Stakeholder Engagement
  
  Tonya Nguyen, Jean Garcia-Gathright, Hannah Washington, Alexandra Chouldechova, Hanna Wallach, Jennifer Wortman Vaughan
  
  The 2026 ACM Conference on Fairness, Accountability, and Transparency | June 2026
  
  Conference on Fairness, Accountability and Transparency
  
  Publication
- What AI Benchmarks Actually Measure: Adapting Convergent and Discriminant Validity to Interrogate Fifty-Six AI Benchmarks
  
  Meera Desai, Sang T. Truong, Hanna Wallach, Alexandra Chouldechova, A. Feder Cooper, Jean Garcia-Gathright, Daniel E. Ho, Abigail Z. Jacobs, Sanmi Koyejo, Nicholas Pangakis, Angelina Wang
  
  September 2026
  
  arXiv | September 2026
  
  Publication
- ASSERT: A Measurement Pipeline for GenAI Audits
  
  Riccardo Fogliato, Abhinav Palia, Xiawei Wang, Emily Sheng, Chad Atalla, Jean Garcia-Gathright, Nicholas Pangakis, Sharman Tan, Dan Vann, Hannah Washington, P. A. Dow, Heba Elfardy, Hanna Wallach, Sandeep Atluri
  
  August 2026
  
  arXiv | August 2026
  
  Publication
- AI-Assisted Systematization for Evaluating GenAI Systems
  
  Dhruv Agarwal, Emily Sheng, Chad Atalla, Jean Garcia-Gathright, Hussein Mozannar, Hannah Washington, Alexandra Chouldechova, Solon Barocas, Hanna Wallach
  
  May 2026
  
  arXiv | May 2026
  
  Publication
- Taxonomizing Representational Harms using Speech Act Theory
  
  Emily Corvi, Hannah Washington, Stefanie Reed, Chad Atalla, Alex Chouldechova, Alex Dow, Jean Garcia-Gathright, Nick Pangakis, Emily Sheng, Dan Vann, Matthew Vogel, Hanna Wallach
  
  March 2025
  
  March 2025
  
  Publication
- A Shared Standard for Valid Measurement of Generative AI Systems’ Capabilities, Risks, and Impacts
  
  Alex Chouldechova, Chad Atalla, Solon Barocas, A. Feder Cooper, Emily Corvi, Alex Dow, Jean Garcia-Gathright, Nick Pangakis, Stefanie Reed, Emily Sheng, Dan Vann, Matthew Vogel, Hannah Washington, Hanna Wallach
  
  December 2024
  
  December 2024
  
  Publication
- Gaps Between Research and Practice When Measuring Representational Harms Caused by LLM-Based Systems
  
  Emma Harvey, Emily Sheng, Su Lin Blodgett, Alex Chouldechova, Jean Garcia-Gathright, Alexandra Olteanu, Hanna Wallach
  
  November 2024
  
  November 2024
  
  Publication
- Evaluating Generative AI Systems is a Social Science Measurement Challenge
  
  Hanna Wallach, Meera Desai, Nick Pangakis, A. Feder Cooper, Angelina Wang, Solon Barocas, Alex Chouldechova, Chad Atalla, Su Lin Blodgett, Emily Corvi, Alex Dow, Jean Garcia-Gathright, Alexandra Olteanu, Stefanie Reed, Emily Sheng, Dan Vann, Jennifer Wortman Vaughan, Matthew Vogel, Hannah Washington, Abigail Z. Jacobs
  
  November 2024
  
  November 2024
  
  Publication
- Understanding and Meeting Practitioner Needs When Measuring Representational Harms Caused by LLM-Based Systems
  
  Emma Harvey, Emily Sheng, Su Lin Blodgett, Alex Chouldechova, Jean Garcia-Gathright, Alexandra Olteanu, Hanna Wallach
  
  Findings of the Association for Computational Linguistics: ACL 2025 | July 2025
  
  Publication
- Position: Evaluating Generative AI Systems is a Social Science Measurement Challenge
  
  Hanna Wallach, Meera Desai, A. Feder Cooper, Angelina Wang, Chad Atalla, Solon Barocas, Su Lin Blodgett, Alex Chouldechova, Emily Corvi, Alex Dow, Jean Garcia-Gathright, Alexandra Olteanu, Nick Pangakis, Stefanie Reed, Emily Sheng, Dan Vann, Jennifer Wortman Vaughan, Matthew Vogel, Hannah Washington, Abigail Z. Jacobs
  
  ICML 2025 | January 2025
  
  Publication

Jean Garcia-Gathright

论文与出版物

What AI Benchmarks Actually Measure: Adapting Convergent and Discriminant Validity to Interrogate Fifty-Six AI Benchmarks

ASSERT: A Measurement Pipeline for GenAI Audits

Validating and Refining Measurements for Generative AI Evaluation Via Stakeholder Engagement

AI-Assisted Systematization for Evaluating GenAI Systems

Understanding and Meeting Practitioner Needs When Measuring Representational Harms Caused by LLM-Based Systems

Taxonomizing Representational Harms using Speech Act Theory

Position: Evaluating Generative AI Systems is a Social Science Measurement Challenge

A Shared Standard for Valid Measurement of Generative AI Systems’ Capabilities, Risks, and Impacts

Gaps Between Research and Practice When Measuring Representational Harms Caused by LLM-Based Systems

Evaluating Generative AI Systems is a Social Science Measurement Challenge

Validating and Refining Measurements for Generative AI Evaluation Via Stakeholder Engagement

What AI Benchmarks Actually Measure: Adapting Convergent and Discriminant Validity to Interrogate Fifty-Six AI Benchmarks

Validating and Refining Measurements for Generative AI Evaluation Via Stakeholder Engagement

Understanding and Meeting Practitioner Needs When Measuring Representational Harms Caused by LLM-Based Systems

Taxonomizing Representational Harms using Speech Act Theory

Position: Evaluating Generative AI Systems is a Social Science Measurement Challenge

A Shared Standard for Valid Measurement of Generative AI Systems’ Capabilities, Risks, and Impacts

Gaps Between Research and Practice When Measuring Representational Harms Caused by LLM-Based Systems

Evaluating Generative AI Systems is a Social Science Measurement Challenge

What AI Benchmarks Actually Measure: Adapting Convergent and Discriminant Validity to Interrogate Fifty-Six AI Benchmarks

ASSERT: A Measurement Pipeline for GenAI Audits

AI-Assisted Systematization for Evaluating GenAI Systems

Taxonomizing Representational Harms using Speech Act Theory

Position: Evaluating Generative AI Systems is a Social Science Measurement Challenge

A Shared Standard for Valid Measurement of Generative AI Systems’ Capabilities, Risks, and Impacts

Gaps Between Research and Practice When Measuring Representational Harms Caused by LLM-Based Systems

Evaluating Generative AI Systems is a Social Science Measurement Challenge

AI-Assisted Systematization for Evaluating GenAI Systems

Understanding and Meeting Practitioner Needs When Measuring Representational Harms Caused by LLM-Based Systems

Validating and Refining Measurements for Generative AI Evaluation Via Stakeholder Engagement

Validating and Refining Measurements for Generative AI Evaluation Via Stakeholder Engagement

What AI Benchmarks Actually Measure: Adapting Convergent and Discriminant Validity to Interrogate Fifty-Six AI Benchmarks

ASSERT: A Measurement Pipeline for GenAI Audits

AI-Assisted Systematization for Evaluating GenAI Systems

Taxonomizing Representational Harms using Speech Act Theory

A Shared Standard for Valid Measurement of Generative AI Systems’ Capabilities, Risks, and Impacts

Gaps Between Research and Practice When Measuring Representational Harms Caused by LLM-Based Systems

Evaluating Generative AI Systems is a Social Science Measurement Challenge

Understanding and Meeting Practitioner Needs When Measuring Representational Harms Caused by LLM-Based Systems

Position: Evaluating Generative AI Systems is a Social Science Measurement Challenge