Jing Cheng^{1}, Haifeng Wang^{1}, Leslie Ying^{2}, and Dong Liang^{1,3}

We introduce a novel deep learning network which combines elements of model and data driven approaches for fast MR imaging, termed modified Learned PD. The network is inspired by the first-order primal dual algorithm, where the convolutional neural network blocks are used to learn the proximal operators. Learned PD network works directly from undersampled k-space data and reconstructs MR images by updating in k-space and image domain alternatively. This approach has been evaluated by in vivo MR datasets and achieves accurate MR reconstructions, outperforming other comparing methods across various quantitative metrics.

**Introduction**

**Methods**

The reconstruction of CS-MRI^{8} can be
formulated as follows:$$min_p{\left\{F(Ap)+G(p)\right\}} (1)$$ where $$$ F(Ap)=||Ap-y||_2^2$$$, $$$p$$$ is the image to be reconstructed, $$$A$$$ denotes the undersampled Fourier transform, $$$y$$$ is the acquired data, $$$G(p)$$$ is a penalty function
that incorporates the prior knowledge. The reconstruction problem
can be solved by a first-order primal dual algorithm (also known as CP
algorithm^{7}) as following iteration:
$$\begin{cases}d_{n+1}=prox_\sigma\left[F^*\right](d_n+\sigma A\overline{p}_n) \\p_{n+1}=prox_\tau\left[G\right](p_n-\tau A^*d_{n+1}) \\ \overline{p}_{n+1}=p_{n+1}+\theta(p_{n+1}-p_n) \end{cases} (2)$$ $$$\sigma$$$, $$$\tau$$$ and $$$\theta$$$ are algorithm parameters and $$$prox$$$ denotes the proximal mapping which can work directly with
non-smooth objective functions.
In this work, we use a Convolutional
Neural Network (CNN) block to replace the proximal mapping to learn the
parameterized proximal operators. The new algorithm, called Learned PD, can be
formulated as:
$$\begin{cases}d_{n+1}=\Gamma(d_n,Ap_n,y) \\p_{n+1}=\Lambda(p_n, A^*d_{n+1}) \end{cases} (3)$$

The architecture of the Learned PD is described in Figure 1. Here we further break the fixed structure between the primal and dual variables and let the network freely learn the relations. Learned PD consists of dual update and primal update. The dual and primal updates have the same structure which has 3 convolutional layers in each iteration. To train the network more easily, we made it a residual network. The convolutions are all 3x3 pixel size, and the number of channels is 4-32-32-2 for each primal update, and 6-32-32-2 for dual update where the number of outputs 2 denotes the real and imaginary parts of the data. The nonlinearities were chosen to be Rectified Linear Unites (ReLU) functions and we let the number of iterations be 10. Since each iteration involves 2 3-layer networks, the total depth of the Learned PD is 60 convolutional layers.

We validated the method using in-vivo MR datasets. Overall 200 fully sampled multi-contrast data from 2 subjects with a 3T scanner (MAGNETOM Trio, SIEMENS AG, Erlgen, Germany) were collected and informed consent was obtained from the imaging object in compliance with the IRB policy. After normalization and image augmentation through rotation and flipping, we got 1600 k-space data, where 1400 for training and 200 for validation. The model was trained and evaluated on an Ubuntu 16.04 LTS (64-bit) operating system equipped with a Tesla TITAN Xp Graphics Processing Unit (GPU, 12GB memory) in the open framework Tensorflow with CUDA and CUDNN support. We also have tested Learned PD on the data acquired from other two 3T scanners of GE (GE Healthcare, Waukesha, WI) and UIH (United Imaging Healthcare, Shanghai, China).

**Results **

We compared the Learned PD with other
CS-MRI reconstruction methods: 1) rec_PF^{9}, traditional CSMR
reconstruction method, 2) ADMM-net^{3}, state-of-the-art model-driven
CSMR reconstruction method. Several similarity metrics, including MSE, SSIM and
PSNR, were used to compare the reconstruction results of different methods.

From quantitative metrics shown in Table 1, with different acceleration factors, the Learned PD all shows significant improvement over other methods. Figure 2 illustrates the reconstructions of the different methods and the corresponding error maps with acceleration factor of 4. Figure 3 and Figure 4 show the visual comparisons of different methods and the zoom-in views demonstrate the ability of the Learned PD that preserves more fine details while removing the undersampling artifacts.

Figure 5 illustrates the reconstructions of data from UIH scanner. Detailed accuracy metrics are also shown in figure.

1. Wang S, Su Z, Ying L, et al. Accelerating magnetic resonance imaging via deep learning. ISBI 514-517 (2016)

2. Schlemper J, Caballero J, Hajnal J V, et al. A Deep Cascade of Convolutional Neural Networks for Dynamic MR Image Reconstruction. IEEE TMI 2018; 37(2): 491-503.

3. Yang Y, Sun J, Li H, Xu Z. Deep ADMM-Net for Compressive Sensing MRI. NIPS, Barcelona, Spain, 2016.

4. Hammernik K, Klatzer T, Kobler E, et al. Learning a Variational Network for Reconstruction of Accelerated MRI Data. Magn Reson Med 2018, 79(6): 3055-3071.

5. Zhu B, Liu J Z, Cauley S F, et al. Image reconstruction by domain-transform manifold learning. Nature 2018; 555(7697): 487-492.

6. Adler J, Oktem O. Learned Primal-Dual Reconstruction. IEEE TMI 2018; 37(6): 1322-1332.

7. Chambolle A, Pock T. A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging. J. Math. Imag. Vis. 2011; 40:120-145.

8. Lustig M, Donoho DL, Pauly JM. Sparse MRI: The application of compressed sensing for rapid MR imaging. Magn Reson Med 2007; 58:1182–1195.

9. Yang J, Zhang Y, Yin W, et al. A Fast Alternating Direction Method for TVL1-L2 Signal Reconstruction From Partial Fourier Data. IEEE JSTSP 2010, 4(2): 288-297.

Figure
1. Architecture
of the Learned PD model for MR reconstruction.

Table 1.
The detailed comparison on performance metrics of different reconstruction
methods with different acceleration factor on the SIEMENS axial brain data.

Figure 2.
Comparison of reconstruction methods with R=4 on GE axial data. The images in
second row are the sampling mask and the corresponding error maps between the
reconstructions and the reference. The Learned PD achieves the highest accuracy
reconstruction.

Figure 3.
Comparison of different reconstruction methods for R=6 on SIEMENS axial data.
The enclosed part is enlarged for a close-up comparison. The zoom-in
visualization shows the Learned PD preserves more details.

Figure 4.
Visual comparison for R=6 on GE sagittal data. The enclosed part is enlarged
for a close-up comparison. The zoom-in visualization and the corresponding
error maps show the improvement of the Learned PD in detail preservation.

Figure 5.
Comparison of reconstruction methods with R=5 on UIH data. Sampling mask and
the corresponding error maps are on the second row. The Learned PD still achieves
the highest accuracy.