PRISM2: Unlocking Multi-Modal General Pathology AI with Clinical Dialogue
- George Shaikovski ,
- E. Vorontsov ,
- Adam Casson ,
- Julian Viret ,
- Eric Zimmermann ,
- Neil Tenenholtz ,
- Yun Wang ,
- Jan H Bernhard ,
- R. Godrich ,
- J. Retamero ,
- Razik Yousfi ,
- Nicolo Fusi ,
- Thomas J Fuchs ,
- Kristen Severson ,
- Siqi Liu
arXiv
Recent rapid progress in the field of computational pathology has been enabled by foundation models. These models are beginning to move beyond encoding image patches towards whole-slide understanding but their clinical utility remains limited. In this work, we present PRISM2, a multimodal slide-level foundation model trained on data from 700,000 diagnostic specimen-report pairs, the largest vision (2.3 million whole slide images) and language (14M question-answer pairs) histopathology dataset to date. By learning through clinical-dialogue supervision, PRISM2 aligns histomorphologic features with the language of diagnostic reasoning, producing slide-level representations that support both direct diagnostic question-answering and transferable embeddings for downstream tasks. Without additional training, PRISM2 matches or exceeds the cancer-detection performance of clinical-grade products. This is observed without loss of generality on other tasks, where PRISM2 achieves top performance. Finally, using survival prediction as the example, we show that task-specific finetuning with a large dataset can outperform task-specific models, further improving performance. These results demonstrate how language-supervised pretraining provides a scalable, clinically grounded signal for learning generalizable pathology representations, bridging human diagnostic reasoning and foundation-model performance.