A 3D convolutional neural network for diagnosing prostate cancer using volumetric T2-weighted MRI.
Pritesh Mehta1, Michela Antonelli1, Shonit Punwani2, and Sebastien Ourselin3

1Biomedical Engineering and Medical Physics, University College London, London, United Kingdom, 2UCL Centre for Medical Imaging, University College London, London, United Kingdom, 3School of Biomedical Engineering & Imaging Sciences, King's College London, London, United Kingdom


In this work, we designed and evaluated a convolutional neural network for prostate cancer diagnosis using volumetric T2-weighted MRI. Our key contribution is a 3D implementation of a residual network (ResNet), optimised to perform a classification between patients with prostate cancer and patients with benign conditions. On this task, cross-validation on a dataset consisting of 240 patients, produced a mean area under the receiver operating characteristic curve of 0.78, which was on par with an experienced radiologist.


Prostate cancer (PCa) is the most common cancer amongst men in the UK, with incidence rates projected to rise by 12% in the period 2014-20351.

Studies such as PICTURE2 and PROMIS3 have shown that MRI of the prostate can radically improve PCa diagnosis, non-invasively. A computer-aided diagnosis system that utilises volumetric MRI to produce a diagnosis would favour applications in patient triage and first/second reader alongside a clinician. With capacity in the NHS under constant pressure and PCa incidence rates increasing, these are important applications.

The aim of this work was to design and evaluate a 3D convolutional neural network (CNN) for PCa diagnosis, where the input modality considered is volumetric T2-weighted (T2w) MRI. Our key contribution is a 3D implementation of a residual convolutional neural network4 (ResNet), designed to classify between patients with PCa and patients with benign conditions, where PCa is defined as Gleason score 3+3 disease and above.


The patient data used in this work is a subset of the data collected during the PICTURE study. The PICTURE study was a “paired-cohort confirmatory study” designed to assess the diagnostic accuracy of radiologists reading mpMRI. A 3 Tesla MRI scanner with a pelvic-phased array coil was used to collect mpMRI sequences. Men underwent a transperineal template prostate-mapping (TTPM) biopsy as a reference test.

For our classification task, we considered a total of 240 patients for training and inference, consisting of 30 patients with benign conditions, 66 patients with Gleason 3+3 PCa, 110 patients with Gleason 3+4 PCa, 29 patients with Gleason 4+3 PCa and 5 patients with Gleason >=4+4 PCa.

A pre-processing pipeline was used to prepare the T2-weighted images of each patient, featuring 3D groupwise registration, prostate area segmentation and cropping and intensity standardisation and normalisation.

A 3D ResNet architecture was developed to classify patients using volumetric T2w imaging. ResNets are a specific type of CNN architecture that feature “residual blocks” with identity “skip connections”. By design, they feature implicit regularisation and an ease of optimisation. Therefore, a 3D ResNet is a good choice for our task, which features a relatively small patient cohort. The network architecture, as determined by a hyperparameter search process, is shown in figure 1.

We evaluated our classification task using 5-fold cross validation, through inspection of the mean Receiver Operating Characteristic (ROC) curve and measurement of the mean Area Under the ROC curve (AUROC) and mean sensitivity at 50% specificity.

Results and Discussion

On the task of classifying between patients with PCa and patients with benign conditions, we obtained a mean AUROC of 0.78 and a mean sensitivity of 83% at 50% specificity. Figure 2 shows the mean ROC curve, calculated over 5 folds.

The result is on par with a radiologist with 10 years of experience in reading prostate MRI. For the same task, the radiologist achieved a calculated AUROC of 0.78, utilising the full breadth of multiparametric MRI (mpMRI) sequences and clinical data available to them, blinded to the results of the TTPM biopsy.


The result achieved shows that a 3D CNN which utilises only volumetric T2-weighted imaging, is on par with an experienced radiologist who had access to mpMRI and other clinical data.

As a next step, we will consider modifying our network architecture to accept other imaging modalities in addition to T2w imaging e.g. Apparent Diffusion Coefficient (ADC) maps and Dynamic Contrast Enhanced (DCE) imaging. We would also like to explore the addition of clinical features such as age, ethnicity and Prostate-Specific Antigen density.

A further area of research will be performing a sub-classification of patients with PCa into Gleason score categories, indicating cancer aggressiveness. This would help guide the decision of immediate treatment or entry onto an active surveillance programme.


This work was supported by the EPSRC-funded UCL Centre for Doctoral Training in Medical Imaging (EP/L016478/1) and the Department of Health’s NIHR-funded Biomedical Research Centre (BRC) at University College London Hospitals.


1. Cancer Research UK. Prostate cancer statistics. Retrieved July 13, 2018, from https://www.cancerresearchuk.org/health-professional/cancer-statistics/statistics-by-cancer-type/prostate-cancer.

2. Simmons L, et al. The PICTURE study: Diagnostic accuracy of multiparametric MRI in men requiring a repeat prostate biopsy. 2017. British Journal of Cancer, 116(9), 1159–1165.

3. Ahmed H. Multi-parametric MRI scans prior to biopsy for improving diagnosis of prostate cancer. 2017. MRC Clinical Trials Unit, UCL.

4. He K, et al. Deep Residual Learning for Image Recognition. 2015. CoRR, 1512.03385.


Figure 1: 3D residual network. BN: Batch Normalisation. ReLU: Rectified Linear Unit.

Figure 2: Mean ROC curve for CNN classification of patients with PCa and patients with benign conditions.

Proc. Intl. Soc. Mag. Reson. Med. 27 (2019)