Risk calculators are used to judge disease risk for tens of millions of patients, making their accuracy crucial. But when national models are adapted for local populations, they often deteriorate, losing accuracy and interpretability. Investigators from Brigham and Women’s Hospital, a founding member of the Mass General Brigham healthcare system, used advanced machine learning to extend the accuracy of a national cardiovascular risk calculator while preserving its interpretability and original risk associations. Their results showed higher accuracy overall in an electronic health records cohort from Mass General Brigham and reclassified roughly one in ten patients into a special risk category to facilitate more precise treatment decisions. The outcomes are published in JAMA Cardiology.
“Risk calculators are incredibly vital as they’re an integral a part of the conversation between providers and patients on risk prevention,” said first writer Aniket Zinzuwadia, MD, a resident physician in Internal Medicine at Brigham and Women’s Hospital. “But sometimes, when applying these global calculators to local populations, there’s variability inherent to the character of an area — whether that’s different demographic characteristics, different physician practice patterns, or different risk aspects — so we wanted to seek out a method to tailor the foundational heart problems risk model to local populations in a protected way that builds upon what’s already being done.”
The American Heart Association released the Predicting Risk of Cardiovascular Disease Events (PREVENT) calculator in 2023 for adults ages 30-79. This recent and improved tool helps predict the likelihood of an individual developing a heart attack, stroke, or heart failure in 10 years and in 30 years. While the PREVENT equations have done well at assessing risk at a national level, the researchers desired to test if their technique could higher calibrate the chance assessment for more local populations.
Within the study, researchers used electronic health record data from 95,326 Mass General Brigham patients who were 55 or older in 2007 and who had at the least one lipid or blood pressure measurement between 1997-2006 and at the least one encounter with the hospital system between 2007-2016. The team used XGBoost, an open-source machine learning library, to recalibrate PREVENT’s equations while still preserving the associations of known risk aspects with the outcomes observed in the unique model. The outcomes demonstrated greater accuracy and the reclassification of 1 out of ten patients on this population.
“This might theoretically represent a bunch of patients that may not have been prescribed statin therapies in the unique application of the model, for instance, but who might need benefited from them,” said Zinzuwadia.
While more steps are needed before this method might be applied to patient care, the team would love to see the way it performs within the local populations of other healthcare systems and, eventually, for clinicians and researchers to make use of the tool to tailor global risk models.
“A significant challenge of applying AI to medical research is ensuring that machine learning models usually are not just flexible, but in addition transparent, reliable, and grounded in domain knowledge,” said co-senior writer Olga Demler, PhD, an associate biostatistician at Brigham and Women’s Hospital’s Division of Preventive Medicine. “Our approach shows that it is feasible to avoid the ‘black box’ nature of AI applications and will offer a path forward where sophisticated algorithms can retain their flexibility while producing guarantees of their performance.”
Authorship: Additional authors include Olga Mineeva, Chunying Li, Zareen Farukhi, Franco Giulianini, Brian Cade, Lin Chen, Elizabeth Karlson, Nina Paynter, and Samia Mora.
Disclosures: Samia Mora has served as a consultant to Pfizer for work outside the present study. Olga Demler and Nina Paynter have received funding from Kowa Research Institute for work unrelated to the present study. Aniket Zinzuwadia has served as an worker of Heartbeat Health for work outside the present study.
Funding: Researchers were supported by the National Heart, Lung, and Blood Institute (K24 HL136852, R21 HL156174, R21HL167173, K01HL135342, and R21125962), the American Heart Association (17IGMV33860009), the Swiss Federal Institute of Technology (ETH, Zurich, Switzerland), Dataspectrum4CVD from the Swiss Data Science Center/Personalized Health & Related Technologies, Zurich, Switzerland, and the National Human Genome Research Institute (U01HG008685).
