Advances in digital pathology and artificial intelligence have presented the potential to build models for objective diagnosis, prognosis and therapeutic-response and resistance prediction. In this talk we will discuss our work on: (1) Data-efficient methods for weakly-supervised whole slide classification with examples in cancer diagnosis and subtyping (Nature BME, 2021), identifying origins for cancers of unknown primary (Nature, 2021) and allograft rejection (Nature Medicine, 2022) (2) Discovering integrative histology-genomic prognostic markers via interpretable multimodal deep learning (Cancer Cell, 2022; IEEE TMI, 2020; ICCV, 2021; CVPR, 2024; ICML, 2024). (3) Building unimodal and multimodal foundation models for pathology, contrasting with language and genomics (Nature Medicine, 2024a, Nature Medicine 2024b, CVPR 2024). (4) Developing a universal multimodal generative co-pilot and chatbot for pathology (Nature, 2024). (5) 3D Computational Pathology (Cell, 2024) (6) Bias and fairness in computational pathology algorithms (Nature Medicine, 2024; Nature BME 2023).

Biomedical image analysis is a core part of clinical decision-making and scientific discovery. However, most clinicians and clinical researchers lack the computational resources or machine learning expertise required to train models for their specific imaging tasks. Existing solutions are often narrowly tailored to predefined tasks, and even foundation model-based approaches typically require fine-tuning, limiting accessibility and generalizability.

I will present frameworks for a new paradigm for biomedical image analysis -- where a single multimodal model is designed to solve a broad spectrum of imaging tasks without the need for task-specific training, fine-tuning, or ML expertise. We design new adaptable systems that tackle diverse and unseen tasks out-of-the-box. By removing the barriers of computational cost and ML specialization, these approaches promise to empower clinicians and researchers to tackle workflows that were not previously possible.