| Title: | Generalizability of polygenic prediction models: how is the R2 defined on test data? |
| Journal: | BMC Medical Genomics |
| Published: | 16 May 2024 |
| Pubmed: | https://pubmed.ncbi.nlm.nih.gov/38755654/ |
| DOI: | https://doi.org/10.1186/s12920-024-01905-8 |
| URL: | https://bmcmedgenomics.biomedcentral.com/counter/pdf/10.1186/s12920-024-01905-8 |
Publication 12070
WARNING: the interactive features of this website use CSS3, which your browser does not support. To use the full features of this website, please update your browser.
Abstract
BackgroundPolygenic risk scores (PRS) quantify an individual's genetic predisposition for different traits and are expected to play an increasingly important role in personalized medicine. A crucial challenge in clinical practice is the generalizability and transferability of PRS models to populations with different ancestries. When assessing the generalizability of PRS models for continuous traits, the R2$$R^2$$ is a commonly used measure to evaluate prediction accuracy. While the R2$$R^2$$ is a well-defined goodness-of-fit measure for statistical linear models, there exist different definitions for its application on test data, which complicates interpretation and comparison of results.MethodsBased on large-scale genotype data from the UK Biobank, we compare three definitions of the R2$$R^2$$ on test data for evaluating the generalizability of PRS models to different populations. Polygenic models for several phenotypes, including height, BMI and lipoprotein A, are derived based on training data with European ancestry using state-of-the-art regression methods and are evaluated on various test populations with different ancestries.ResultsOur analysis shows that the choice of the R2$$R^2$$ definition can lead to considerably different results on test data, making the comparison of R2$$R^2$$ values from the literature problematic. While the definition as the squared correlation between predicted and observed phenotypes solely addresses the discriminative performance and always yields values between 0 and 1, definitions of the R2$$R^2$$ based on the mean squared prediction error (MSPE) with reference to intercept-only models assess both discrimination and calibration. These MSPE-based definitions can yield negative values indicating miscalibrated predictions for out-of-target populations. We argue that the choice of the most appropriate definition depends on the aim of PRS analysis - whether it primarily serves for risk stratification or also for individual phenotype prediction. Moreover, both correlation-based and MSPE-based definitions of R2$$R^2$$ can provide valuable complementary information.ConclusionsAwareness of the different definitions of the R2$$R^2$$ on test data is necessary to facilitate the reporting and interpretation of results on PRS generalizability. It is recommended to explicitly state which definition was used when reporting R2$$R^2$$ values on test data. Further research is warranted to develop and evaluate well-calibrated polygenic models for diverse populations.</p>
5 Authors
- Christian Staerk
- Hannah Klinkhammer
- Tobias Wistuba
- Carlo Maj
- Andreas Mayr
1 Application
| Application ID | Title |
|---|---|
| 81202 | Association and prediction models for complex phenotypes characterized by different genetic architecture. |
Enabling scientific discoveries that improve human health