Yohan Jun^{1}, Taejoon Eo^{1}, Hyungseob Shin^{1}, Taeseong Kim^{1}, Hojoon Lee^{2,3}, and Dosik Hwang^{1}

A deep parallel imaging network (“DPI-net”) was developed to reconstruct 3D multi-channel MRA from undersampled data. It comprises two deep-learning networks: a network of multi-stream

Our purpose was to restore the undersampled multi-channel 3D MR images to a fully sampled 3D MR image using deep-learning networks. Thus, the objective function can be formulated as the following minimization equation:

$$\arg\min_{{\bf{\theta}}}\begin{Vmatrix}{{\bf{y}}-D_{H}({\bf{x}};{\bf{\theta}})}\end{Vmatrix}_2^2=\arg\min_{\theta}{\begin{Vmatrix}{{\sqrt{\sum_{c=1}^{N_{c}}{\mid{{{\bf I}_f^c}}\mid}^2}}-D_{H}(\begin{bmatrix}{\mid{{\bf{I}}_u^1}\mid},{\mid{{\bf I}_u^2}\mid},...,{\mid{{\bf{I}}_u^{N_{c}}}\mid}\end{bmatrix};{\theta})}\end{Vmatrix}}_2^2$$

where $$${\bf I}_f^c\in{C}^{{N_{k_{x}}{\times}N_{k_{y}}{\times}N_{k_{z}}}}$$$ is the fully sampled image of the $$$c$$$-th coil ($$$c=1,...,N_{c}$$$), $$${\bf I}_u^c\in{C}^{{N_{k_{x}}{\times}N_{k_{y}}{\times}N_{k_{z}}}}$$$ is the undersampled image of the $$$c$$$-th coil, $$${\bf{y}}={\sqrt{\sum_{c=1}^{N_{c}}{\mid{{{\bf I}_f^c}}\mid}^2}}\in{R}^{{N_{k_{x}}{\times}N_{k_{y}}{\times}N_{k_{z}}}}$$$ is the desired output (the square root of sum-of-squares (SSOS) of the fully sampled multi-channel 3D MR images), $$${\bf{x}}=\begin{bmatrix}{\mid{{\bf{I}}_u^1}\mid},{\mid{{\bf I}_u^2}\mid},...,{\mid{{\bf{I}}_u^{N_{c}}}\mid}\end{bmatrix}\in{R}^{{N_{k_{x}}{\times}N_{k_{y}}{\times}N_{k_{z}}}}$$$ is the magnitude of the undersampled multi-channel 3D MR images, and $$$D_{H}$$$ is the hypothesis function of the deep-learning network with parameters $$${\theta}$$$. The objective was to find $$${\theta}$$$ that minimized the $$${l_{2}}$$$ difference between $$${\bf{y}}$$$ and $$$D_{H}({\bf{x}};{\bf{\theta}})$$$ for the given training data set. DPI-net’s deep-learning architecture is based on fully CNNs and consists of two main networks: MS-net, comprising multi-stream CNNs for extracting feature maps of multi-channel images, and RC-net, comprising deep reconstruction CNNs for reconstructing the images. In this study, multi-stream CNNs architecture was introduced to address MR images acquired from multiple channels. Each channel’s undersampled MR image was fed in parallel as input into the multi-stream CNNs and output feature maps were obtained. Then, reconstruction CNNs processed the feature maps and the final MR image was reconstructed from the output of the reconstruction CNNs. The overall architecture is presented in Fig. 1.

MRI was performed using a 3.0T scanner with a 32-channel sensitivity-encoding head coil. Data for seven subjects were acquired using 3D TOF sequences, with three subjects’ data sets used for the training set, three for the test set, and one for the validation set. The parameters of 3D TOF sequence were as follows: TR, 20 ms; TE, 3 ms; flip angle, 18 degrees; matrix size, 432 $$${\times}$$$ 432; pixel resolution, 0.49 $$${\times}$$$ 0.49 mm^{2}; slice thickness, 0.5 mm; and acquisition time, 11 min 51 s. Three slabs were acquired using 3D TOF sequences, each with 56 slices. A total of 120 slices were reconstructed from the three slabs, with the matrix size of the 3D volume image being 432 $$${\times}$$$ 432 $$${\times}$$$ 120. We retrospectively undersampled 3D TOF MRA k-space data for each slab with a variable-density Poisson-disk sampling pattern on $$$(k_{y},k_{z})$$$ domain such that the k-space data was fully sampled in the read-out direction $$$(k_{x})$$$. The reduction factors was *R* = 5.7.

[1] White PM, Teasdale EM, Wardlaw JM, Easton V. Intracranial aneurysms: CT angiography and MR angiography for detection—prospective blinded comparison in a large patient cohort. Radiology. 2001;219(3):739–749.

[2] Gibbs GF, Huston J, Bernstein MA, Riederer SJ, Brown RD. Improved image quality of intracranial aneurysms: 3.0-T versus 1.5-T time-of-flight MR angiography. Am J Neuroradiol. 2004;25(1):84–87.

[3] Weber J, Veith P, Jung B, et al. MR angiography at 3 Tesla to assess proximal internal carotid artery stenoses: contrast-enhanced or 3D time-of-flight MR angiography? Clin Neuroradiol. 2015;25(1):41–48.

[4] Wilson GJ, Hoogeveen RM, Willinek WA, Muthupillai R, Maki JH. Parallel imaging in MR angiography. Top Magn Reson Imaging. 2004;15(3):169–185.

[5] Yang G, Yu S, Dong H, et al. Dagan: Deep de-aliasing generative adversarial networks for fast compressed sensing MRI reconstruction. IEEE Trans Med Imaging. 2018;37(6):1310–1321.

[6] Kwon K, Kim D, Park H. A parallel MR imaging method using multilayer perceptron. Med Phys. 2017;44(12):6209–6224.

[7] Hammernik K, Klatzer T, Kobler E, et al. Learning a variational network for reconstruction of accelerated MRI data. Magn Reson Med. 2018;79(6):3055–3071.

[8] Eo T, Jun Y, Kim T, Jang J, Lee HJ, Hwang D. KIKI‐net: cross‐domain convolutional neural networks for reconstructing undersampled magnetic resonance images. Magn Reson Med. 2018. doi: 10.1002/mrm.27201.

[9] Hyun CM, Kim HP, Lee SM, Lee S, Seo JK. Deep learning for undersampled MRI reconstruction. Phys Med Biol. 2018;63:135007.

[10] Shin PJ, Larson PE, Ohliger MA, et al. Calibrationless parallel imaging reconstruction based on structured low‐rank matrix completion. Magn Reson Med. 2014;72(4):959–970.

[11] Uecker M, Lai P, Murphy MJ, et al. ESPIRiT—an eigenvalue approach to autocalibrating parallel MRI: where SENSE meets GRAPPA. Magn Reson Med. 2014;71(3):990–1001.

[12] Ronneberger O, Fischer P, Brox T. U-net: Convolutional networks for biomedical image segmentation. In Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany, 2015. p. 234–241.

Figure 1. DPI-net’s deep-learning architecture. This comprises two networks: MS-net, a network of multi-stream convolutional neural networks (CNNs); and RC-net, a network of reconstruction CNNs. MS-net extracts the feature maps of multi-channel images and RC-net reconstructs images from the MS-net output feature maps.

Figure 2. Reconstruction results for 3D time-of-flight images reconstructed by fully sampled, zero-filled, SAKE, ESPIRiT, U-net, and DPI-net, with the reduction factor *R* = 5.7. (a) Axial slices. (b) Magnified images. (c) Difference images. The black and grey arrows indicate vessel signals. As indicated by the arrows, DPI-net shows better performance than the conventional methods in recovering the vessel signals.

Figure 3. Reconstruction results for 3D time-of-flight images reconstructed by fully sampled, zero-filled, SAKE, ESPIRiT, U-net, and DPI-net, with the reduction factor *R* = 5.7. (a) Sagittal maximum intensity projection images. (b) Magnified images. The black and grey arrows indicate blood vessels. As indicated by the arrows, DPI-net shows better performance than the conventional methods in recovering the vessel signals.

Figure 4. Reconstruction results for 3D time-of-flight images reconstructed by fully sampled, zero-filled, SAKE, ESPIRiT, U-net, and DPI-net, with the reduction factor *R* = 5.7. (a) Coronal maximum intensity projection images. (b) Magnified images. The black and grey arrows indicated blood vessels. The arrows show that SAKE, ESPIRiT, and U-net hardly restore blood vessels, whereas DPI-net recovers them.

Figure 5. Vessel sharpness comparison on
four different vessels (right MCA, left MCA, right PCA, and left PCA) with
different reconstruction methods including zero-filled, SAKE, ESPIRiT, U-net,
and DPI-net with the reduction factor *R*
= 5.7. Paired t-tests were used to calculate *P*-values. **P* < 0.05, ***P* <
0.01, ****P* < 0.001.