Objective:

To develop a highly interpretable machine learning model to predict 30 day perioperative cerebrovascular accidents (CVA).

Background:

Traditional logistic regression (LR) based calculators to predict perioperative cerebrovascular accidents (CVA) use one weight or beta coefficient per variable irrespective of patient characteristics which can limit accuracy. While machine learning (ML) models do not have this limitation, they are often considered “black-box” models which leads to poor adoption into clinical use. Explainable Boosting Machines (EBMs) are an interpretable model based on generalized additive models but use ML techniques such as bagging and boosting to enhance accuracy. Unlike traditional logistic regression, the weights in an EBM can change depending on specific patient characteristics providing much higher granularity.

Design/Methods:

We trained an EBM to predict 30-day perioperative CVA using the 2019 American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) database. Perioperative CVA was defined as hemorrhagic or ischemic stroke from any cause within 30 days of the surgery. We included 1,076,441 cases with 79 preoperative variables as features, splitting the data into training (60%), validation (20%), and testing (20%).

Results:

Top features to predict 30-day mortality by global feature importance in descending order of importance are inpatient/outpatient status, American Society of Anesthesiologists (ASA) classification, and age. The EBM model had a Brier score of 0.00722, AUROC of 0.949, and AUC-PR of 0.282. Above the age of 56 the risk of perioperative CVA increases.

Conclusions:

EBMs offer comparable or improved discriminatory characteristics but with significantly increased interpretability over current risk calculators. The granularity of EBMs allows us to show that patients over 56 years old start to have an increased risk of perioperative CVA. When integrating risk calculators into electronic medical records, explainable boosting machines should be strongly considered.