| Title: | Raptor: Scalable Train-Free Embeddings for 3D Medical Volumes Leveraging Pretrained 2D Foundation Models. |
| Journal: | Proceedings of machine learning research |
| Published: | 1 Jul 2025 |
| Pubmed: | https://pubmed.ncbi.nlm.nih.gov/41684746/ |
Publication 17463
WARNING: the interactive features of this website use CSS3, which your browser does not support. To use the full features of this website, please update your browser.
Abstract
Current challenges in developing foundational models for volumetric imaging data, such as magnetic resonance imaging (MRI), stem from the computational complexity of training state-of-the-art architectures in high dimensions and curating sufficiently large datasets of volumes. To address these challenges, we introduce Raptor (Random Planar Tensor Reduction), a train-free method for generating semantically rich embeddings for volumetric data. Raptor leverages a frozen 2D foundation model, pretrained on natural images, to extract visual tokens from individual cross-sections of medical volumes. These tokens are then spatially compressed using random projections, significantly reducing computational complexity while retaining semantic information. Extensive experiments on ten diverse medical volume tasks verify the superior performance of Raptor over state-of-the-art methods, including those pretrained exclusively on medical volumes (+3% SuPreM, +6% MISFM, +10% Merlin, +13% VoCo, and +14% SLIViT), while entirely bypassing the need for costly training. Our results highlight the effectiveness and versatility of Raptor as a foundation for advancing deep learning-based methods for medical volumes (code: github.com/sriramlab/raptor).</p>
6 Authors
- Ulzee An
- Moonseong Jeong
- Simon A Lee
- Aditya Gorla
- Yuzhe Yang
- Sriram Sankararaman
1 Application
| Application ID | Title |
|---|---|
| 33127 | Methods for large-scale medical and population genetic data |
Enabling scientific discoveries that improve human health