Comparison of machine learning techniques to predict all-cause mortality using fitness data: The Henry Ford exercIse testing (FIT) project

Sherif Sakr, Radwa Elshawi, Amjad M. Ahmed, Waqas T. Qureshi, Clinton A. Brawner, Steven J. Keteyian, Michael J. Blaha, Mouaz H. Al-Mallah

Research output: Contribution to journalArticle

7 Citations (Scopus)

Abstract

Background: Prior studies have demonstrated that cardiorespiratory fitness (CRF) is a strong marker of cardiovascular health. Machine learning (ML) can enhance the prediction of outcomes through classification techniques that classify the data into predetermined categories. The aim of this study is to present an evaluation and comparison of how machine learning techniques can be applied on medical records of cardiorespiratory fitness and how the various techniques differ in terms of capabilities of predicting medical outcomes (e.g. mortality). Methods: We use data of 34,212 patients free of known coronary artery disease or heart failure who underwent clinician-referred exercise treadmill stress testing at Henry Ford Health Systems Between 1991 and 2009 and had a complete 10-year follow-up. Seven machine learning classification techniques were evaluated: Decision Tree (DT), Support Vector Machine (SVM), Artificial Neural Networks (ANN), Naïve Bayesian Classifier (BC), Bayesian Network (BN), K-Nearest Neighbor (KNN) and Random Forest (RF). In order to handle the imbalanced dataset used, the Synthetic Minority Over-Sampling Technique (SMOTE) is used. Results: Two set of experiments have been conducted with and without the SMOTE sampling technique. On average over different evaluation metrics, SVM Classifier has shown the lowest performance while other models like BN, BC and DT performed better. The RF classifier has shown the best performance (AUC = 0.97) among all models trained using the SMOTE sampling. Conclusions: The results show that various ML techniques can significantly vary in terms of its performance for the different evaluation metrics. It is also not necessarily that the more complex the ML model, the more prediction accuracy can be achieved. The prediction performance of all models trained with SMOTE is much better than the performance of models trained without SMOTE. The study shows the potential of machine learning methods for predicting all-cause mortality using cardiorespiratory fitness data.

Original languageEnglish
Article number174
JournalBMC Medical Informatics and Decision Making
Volume17
Issue number1
DOIs
Publication statusPublished - 19 Dec 2017

Fingerprint

Exercise
Mortality
Decision Trees
Health
Area Under Curve
Medical Records
Machine Learning
Coronary Artery Disease
Heart Failure
Cardiorespiratory Fitness
Support Vector Machine

Keywords

  • All-cause mortality
  • FIT (Henry Ford ExercIse testing) project
  • Machine learning

Cite this

Sakr, Sherif ; Elshawi, Radwa ; Ahmed, Amjad M. ; Qureshi, Waqas T. ; Brawner, Clinton A. ; Keteyian, Steven J. ; Blaha, Michael J. ; Al-Mallah, Mouaz H. / Comparison of machine learning techniques to predict all-cause mortality using fitness data : The Henry Ford exercIse testing (FIT) project. In: BMC Medical Informatics and Decision Making. 2017 ; Vol. 17, No. 1.
@article{0471dcd52c0e4984b2e4f6302d7fe00a,
title = "Comparison of machine learning techniques to predict all-cause mortality using fitness data: The Henry Ford exercIse testing (FIT) project",
abstract = "Background: Prior studies have demonstrated that cardiorespiratory fitness (CRF) is a strong marker of cardiovascular health. Machine learning (ML) can enhance the prediction of outcomes through classification techniques that classify the data into predetermined categories. The aim of this study is to present an evaluation and comparison of how machine learning techniques can be applied on medical records of cardiorespiratory fitness and how the various techniques differ in terms of capabilities of predicting medical outcomes (e.g. mortality). Methods: We use data of 34,212 patients free of known coronary artery disease or heart failure who underwent clinician-referred exercise treadmill stress testing at Henry Ford Health Systems Between 1991 and 2009 and had a complete 10-year follow-up. Seven machine learning classification techniques were evaluated: Decision Tree (DT), Support Vector Machine (SVM), Artificial Neural Networks (ANN), Na{\"i}ve Bayesian Classifier (BC), Bayesian Network (BN), K-Nearest Neighbor (KNN) and Random Forest (RF). In order to handle the imbalanced dataset used, the Synthetic Minority Over-Sampling Technique (SMOTE) is used. Results: Two set of experiments have been conducted with and without the SMOTE sampling technique. On average over different evaluation metrics, SVM Classifier has shown the lowest performance while other models like BN, BC and DT performed better. The RF classifier has shown the best performance (AUC = 0.97) among all models trained using the SMOTE sampling. Conclusions: The results show that various ML techniques can significantly vary in terms of its performance for the different evaluation metrics. It is also not necessarily that the more complex the ML model, the more prediction accuracy can be achieved. The prediction performance of all models trained with SMOTE is much better than the performance of models trained without SMOTE. The study shows the potential of machine learning methods for predicting all-cause mortality using cardiorespiratory fitness data.",
keywords = "All-cause mortality, FIT (Henry Ford ExercIse testing) project, Machine learning",
author = "Sherif Sakr and Radwa Elshawi and Ahmed, {Amjad M.} and Qureshi, {Waqas T.} and Brawner, {Clinton A.} and Keteyian, {Steven J.} and Blaha, {Michael J.} and Al-Mallah, {Mouaz H.}",
year = "2017",
month = "12",
day = "19",
doi = "10.1186/s12911-017-0566-6",
language = "English",
volume = "17",
journal = "BMC Medical Informatics and Decision Making",
issn = "1472-6947",
publisher = "BioMed Central Ltd.",
number = "1",

}

Comparison of machine learning techniques to predict all-cause mortality using fitness data : The Henry Ford exercIse testing (FIT) project. / Sakr, Sherif; Elshawi, Radwa; Ahmed, Amjad M.; Qureshi, Waqas T.; Brawner, Clinton A.; Keteyian, Steven J.; Blaha, Michael J.; Al-Mallah, Mouaz H.

In: BMC Medical Informatics and Decision Making, Vol. 17, No. 1, 174, 19.12.2017.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Comparison of machine learning techniques to predict all-cause mortality using fitness data

T2 - The Henry Ford exercIse testing (FIT) project

AU - Sakr, Sherif

AU - Elshawi, Radwa

AU - Ahmed, Amjad M.

AU - Qureshi, Waqas T.

AU - Brawner, Clinton A.

AU - Keteyian, Steven J.

AU - Blaha, Michael J.

AU - Al-Mallah, Mouaz H.

PY - 2017/12/19

Y1 - 2017/12/19

N2 - Background: Prior studies have demonstrated that cardiorespiratory fitness (CRF) is a strong marker of cardiovascular health. Machine learning (ML) can enhance the prediction of outcomes through classification techniques that classify the data into predetermined categories. The aim of this study is to present an evaluation and comparison of how machine learning techniques can be applied on medical records of cardiorespiratory fitness and how the various techniques differ in terms of capabilities of predicting medical outcomes (e.g. mortality). Methods: We use data of 34,212 patients free of known coronary artery disease or heart failure who underwent clinician-referred exercise treadmill stress testing at Henry Ford Health Systems Between 1991 and 2009 and had a complete 10-year follow-up. Seven machine learning classification techniques were evaluated: Decision Tree (DT), Support Vector Machine (SVM), Artificial Neural Networks (ANN), Naïve Bayesian Classifier (BC), Bayesian Network (BN), K-Nearest Neighbor (KNN) and Random Forest (RF). In order to handle the imbalanced dataset used, the Synthetic Minority Over-Sampling Technique (SMOTE) is used. Results: Two set of experiments have been conducted with and without the SMOTE sampling technique. On average over different evaluation metrics, SVM Classifier has shown the lowest performance while other models like BN, BC and DT performed better. The RF classifier has shown the best performance (AUC = 0.97) among all models trained using the SMOTE sampling. Conclusions: The results show that various ML techniques can significantly vary in terms of its performance for the different evaluation metrics. It is also not necessarily that the more complex the ML model, the more prediction accuracy can be achieved. The prediction performance of all models trained with SMOTE is much better than the performance of models trained without SMOTE. The study shows the potential of machine learning methods for predicting all-cause mortality using cardiorespiratory fitness data.

AB - Background: Prior studies have demonstrated that cardiorespiratory fitness (CRF) is a strong marker of cardiovascular health. Machine learning (ML) can enhance the prediction of outcomes through classification techniques that classify the data into predetermined categories. The aim of this study is to present an evaluation and comparison of how machine learning techniques can be applied on medical records of cardiorespiratory fitness and how the various techniques differ in terms of capabilities of predicting medical outcomes (e.g. mortality). Methods: We use data of 34,212 patients free of known coronary artery disease or heart failure who underwent clinician-referred exercise treadmill stress testing at Henry Ford Health Systems Between 1991 and 2009 and had a complete 10-year follow-up. Seven machine learning classification techniques were evaluated: Decision Tree (DT), Support Vector Machine (SVM), Artificial Neural Networks (ANN), Naïve Bayesian Classifier (BC), Bayesian Network (BN), K-Nearest Neighbor (KNN) and Random Forest (RF). In order to handle the imbalanced dataset used, the Synthetic Minority Over-Sampling Technique (SMOTE) is used. Results: Two set of experiments have been conducted with and without the SMOTE sampling technique. On average over different evaluation metrics, SVM Classifier has shown the lowest performance while other models like BN, BC and DT performed better. The RF classifier has shown the best performance (AUC = 0.97) among all models trained using the SMOTE sampling. Conclusions: The results show that various ML techniques can significantly vary in terms of its performance for the different evaluation metrics. It is also not necessarily that the more complex the ML model, the more prediction accuracy can be achieved. The prediction performance of all models trained with SMOTE is much better than the performance of models trained without SMOTE. The study shows the potential of machine learning methods for predicting all-cause mortality using cardiorespiratory fitness data.

KW - All-cause mortality

KW - FIT (Henry Ford ExercIse testing) project

KW - Machine learning

UR - http://www.scopus.com/inward/record.url?scp=85038420210&partnerID=8YFLogxK

U2 - 10.1186/s12911-017-0566-6

DO - 10.1186/s12911-017-0566-6

M3 - Article

AN - SCOPUS:85038420210

VL - 17

JO - BMC Medical Informatics and Decision Making

JF - BMC Medical Informatics and Decision Making

SN - 1472-6947

IS - 1

M1 - 174

ER -