| Title: | Pre-training genomic language model with variants for better modeling functional genomics |
| Journal: | npj Artificial Intelligence |
| Published: | 20 Apr 2026 |
| Pubmed: | https://pubmed.ncbi.nlm.nih.gov/42022284/ |
| DOI: | https://doi.org/10.1038/s44387-026-00103-4 |
| URL: | https://www.nature.com/articles/s44387-026-00103-4.pdf |
Publication 17919
WARNING: the interactive features of this website use CSS3, which your browser does not support. To use the full features of this website, please update your browser.
Abstract
Sequence-to-function models can predict gene expression from sequence data and be used to link genetic information with transcriptomics data to understand regulatory processes and their effects on complex phenotypes. The genomic language models are pre-trained with large-scale DNA sequences. These models can generate robust representations of DNA sequences by learning the genomic context. However, few studies can estimate the predictability of gene expression levels and bridge these two classes of models together to explore individualized gene expression prediction. In this manuscript, we propose UKBioBERT as a DNA language model pre-trained with genetic variants from UK BioBank. We demonstrate that UKBioBERT generates informative embeddings capable of identifying gene functions and improving gene expression prediction in cell lines. Therefore, UKBioBERT can enhance our understanding of gene expression predictability. Building upon these embeddings, we combine UKBioBERT with state-of-the-art sequence-to-function architectures, Enformer and Borzoi, to create UKBioFormer and UKBioZoi. These models exhibit better performance in predicting highly predictable gene expression levels and can be generalized across different cohorts. Furthermore, UKBioFormer effectively captures the relationship between genetic variants and expression variations, enabling in-silico mutation analyses and eQTL identification. Collectively, our findings underscore the value of integrating genomic language models and sequence-to-function approaches for advancing functional genomics research.</p>
6 Authors
- Tianyu Liu
- Xiangyu Zhang
- Jiecong Lin
- Luca Pinello
- Rex Ying
- Hongyu Zhao
1 Application
| Application ID | Title |
|---|---|
| 29900 | Defining physical activity phenotypes based on objective measures and dissecting the genetic and phenotypic relationships of physical activity with diseases and health outcomes |
Enabling scientific discoveries that improve human health