Objective:

This scoping review aims to map current literature assessing inter-rater reliability, intra-rater reliability, or accuracy of modalities used to measure carotid stenosis in standard clinical practice.

Background:

High-quality measurement of carotid stenosis is an essential part of assessing candidacy for carotid endarterectomy or stenting. It is unclear how the common modalities (Doppler ultrasound (DUS), CT angiography (CTA), MR angiography (MRA), and conventional angiography) compare in terms of reliability or accuracy.

Design/Methods:

A search strategy was applied to four databases between 2010 and 2022. Included studies described inter-rater reliability, intra-rater reliability, and/or accuracy of DUS, CTA, MRA, and conventional angiography in the measurement of extracranial internal carotid stenosis. Studies were excluded if they only examined novel techniques, assessed patients following carotid revascularization, were published before 2010 or in a non-English language.

Results:

We screened 963 manuscripts and included 24 for analysis. Although conventional angiography was the most common gold standard, all four modalities were used as the comparator in different studies. Studies assessed 13 different metrics, with highly variable results: sensitivity (27.8%-100%); specificity (20%-100%); positive predictive value (17.5%-100%); negative predictive value (10%-100%); agreement (42.9%-90.3%), kappa (0.25-0.97), weighted kappa (0.45-0.94); Spearman correlation coefficient (0.49-1), accuracy (55.2%-99%), Pearson’s correlation coefficient (0.49-1), volume under receiver operating characteristics surface (0.735-0.761), interclass correlation coefficient (0.83-0.98), and intraclass correlation coefficient (0.83-0.98). Inter-rater reliability ranged from 0.41-0.97 (11 studies; six metrics), intra-rater reliability from 0.59-1 (six studies; five metrics), and accuracy from 0.1-1 (22 studies; 11 metrics).

Conclusions:

Few studies have investigated the measurement of carotid stenosis since 2010. These studies demonstrate significant heterogeneity in modality assessed, method of assessment, and results. No clear conclusion can be drawn about the superiority of one modality over another. As imaging bears major implications in carotid stenosis management, these findings serve as a call for future research to better guide decision-making.