Domain Adaptation Strategies for Transformer-Based Disease Prediction Using Electronic Health Records

Abstract

Abstract Electronic Health Records (EHRs) offer rich data for machine learning, but model generalizability across institutions is hindered by statistical and coding biases. This study investigates domain adaptation (DA) techniques to improve model transfer, focusing on the Ex-Med-BERT transformer architecture for structured EHR data. We compare supervised and un-supervised DA methods in transferring predictive capabilities from the large-scale IBM Explorys database (U.S.) to the UK Biobank. Results across six clinical endpoints show that DA methods outperform fine-tuning, especially with limited target data. These findings emphasize selecting DA strategies based on target data availability and the benefit of incorporating source domain data for robust adaptation.