Yihui Huang^{1}, Jinkui Zhao^{1}, Zi Wang^{1}, Di Guo^{2}, and Xiaobo Qu^{1}

^{1}Department of Electronic Science, National Institute for Data Science in Health and Medicine, Xiamen University, Xiamen, China, ^{2}School of Computer and Information Engineering, Xiamen University of Technology, Xiamen, China

Nuclear Magnetic Resonance (NMR) spectroscopy is regarded as an important tool in bio-engineering while often suffers from its time-consuming acquisition. Non-Uniformly Sampling (NUS) method can speed up the acquisition, but the missing FID signals need to be reconstructed with proper method.. In this work, we proposed a deep learning reconstruction method based on unrolling the iterative process of a state-of-the-art model-based low rank Hankel matrix method. Experimental results show that the proposed method provides a better approximation of low rank and preserves the low-intensity signals much better.

Inspired by LRHMF5, 6 to obtain reconstructed FID $$$\mathbf{\hat{x}}$$$, we unfold its iterative process into three updating modules $$$\mathbf{P},\mathbf{Q},\mathbf{D}$$$ and one data consistency module (Fig.1(a)), corresponding to the updating of four intermediate variables in the iterative process. The initial variables that input the neural network are calculated by factorizing $$$\mathsf{\mathcal{R}}\mathbf{y}\text{=}\mathbf{PQ}_{{}}^{H}$$$, where $$$\mathsf{\mathcal{R}}$$$ is the Hankel operator which turns a vector to a Hankel matrix, while $$$\mathbf{D}$$$ is initialized by zero matrix.

The updating modules of $$$\mathbf{P},\mathbf{Q}$$$ in the k-th (k=1, 2…, K) iteration block can be modified as:

$$\begin{align} & {{\mathbf{P}}^{k+1}}={{\mathsf{\mathcal{P}}}^{k}}((\mathsf{\mathcal{R}}{{{\mathbf{\hat{x}}}}^{k}}+{{\mathbf{D}}^{k}}){{\mathbf{Q}}^{k}},{{\mathbf{Q}}^{k}},{{\mathbf{P}}^{k}}) \\ & {{\mathbf{Q}}^{k+1}}={{\mathsf{\mathcal{Q}}}^{k}}({{(\mathsf{\mathcal{R}}{{{\mathbf{\hat{x}}}}^{k}}+{{\mathbf{D}}^{k}})}^{H}}{{\mathbf{P}}^{k+1}},{{\mathbf{P}}^{k+1}},{{\mathbf{Q}}^{k}}) \\ \end{align},\ (1) $$

where the variables $$$ (\mathsf{\mathcal{R}}{{\mathbf{\hat{x}}}^{k}}+{{\mathbf{D}}^{k}}){{\mathbf{Q}}^{k}}$$$, $$${{\mathbf{Q}}^{k}}$$$, $$${{\mathbf{P}}^{k}}$$$ are concatenated (Eq. (1)) to be the input of updating module $$$\mathbf{P}$$$ (Fig.1(b)), which is an 8-layers densely connected convolutional neural network

The updating module $$$\mathbf{D}$$$ is then calculated by:

$${{\mathbf{D}}^{k+1}}={{\mathbf{D}}^{k}}+\tau (\mathsf{\mathcal{R}}{{\mathbf{\hat{x}}}^{k}}-{{\mathbf{P}}^{k+1}}{{({{\mathbf{Q}}^{k+1}})}^{H}}),\ (2) $$

where $$$\tau $$$ is set as a constant.

The data consistency module is designed to ensure that reconstructed time-domain signal is aligned to the sampled FID $$$\mathbf{y}$$$ . Given the updated variables $$${{\mathbf{P}}^{k+1}}$$$, $$${{\mathbf{Q}}^{k+1}}$$$ and $$${{\mathbf{D}}^{k+1}}$$$, the reconstructed spectrum is modified as:

$${{\mathbf{\hat{x}}}^{k+1}}=\mathsf{\mathcal{S}}(\mathbf{y},{{\mathsf{\mathcal{R}}}^{*}}({{\mathbf{P}}^{k+1}}{{({{\mathbf{Q}}^{k+1}})}^{H}}-{{\mathbf{D}}^{k+1}})),\ (3)$$

where $$$\mathsf{\mathcal{S}}$$$ denotes the data consistency operator, indicating that the signal at the location of sampled FID should maintain a trade-off between the sampled and reconstructed FID.

The overall loss function in our implementation contains two parts, which are the mean square error between reconstructed $$${{\mathbf{\hat{x}}}^{k+1}}$$$ and fully sampled FID $$$\mathbf{x}$$$, matrix $$${{\mathbf{P}}^{k+1}}{{({{\mathbf{Q}}^{k+1}})}^{H}}$$$ and $$$\mathsf{\mathcal{R}}\mathbf{x}$$$ in all K blocks.

The analysis of intermediate reconstructed results of synthetic FID (Fig. 2) indicate that, in each block (Figs. 2(h)-(l)), the DHMF provides a much better approximation of singular values than DLNMR. At the last block (Fig. 2(l)), DHMF provides very close singular values, although they are not exactly the same, to that of the fully sampled FID. These observations imply that the proposed method provides a better approximation of low rank and better interpretation of the reconstruction in the network. Synthetic FID (Fig. 3) consists of five peaks with at most 20 times spectral intensity. Peak intensity correlation (Fig. 3(g)) demonstrates that DHMF provides the most consistent spectral peak shape and intensity to the fully sampled peak. DLNMR hardly retrieves the weakest peaks while both LRHM and LRHMF introduce pseudo peak around the ground-truth weak peak.

For realistic NMR spectra, one 2D

This work was supported in part by the National Natural Science Foundation of China (61971361, 61871341, 61811530021 and U1632274), the National Key R&D Program of China (2017YFC0108703), the Natural Science Foundation of Fujian Province of China (2018J06018), the Fundamental Research Funds for the Central Universities (20720180056 and 20720200065), and Xiamen University Nanqiang Outstanding Talents Program.

The correspondence should be sent to Dr. Xiaobo Qu (Email: quxiaobo@xmu.edu.cn).

[1] Qu X, Mayzel M, Cai J-F, et al. Accelerated NMR spectroscopy with low-rank reconstruction. Angew. Chem.-Int. Edit. 2015;54(3):852-854.

[2] Nguyen H M, Peng X, Do M N, et al. Denoising MR spectroscopic imaging data with low-rank approximations. IEEE Trans. Biomed. Eng. 2013;60(1):78-89.

[3] Ying J, Lu H, Wei Q, et al. Hankel matrix nuclear norm regularized tensor completion for N-dimensional exponential signals. IEEE Trans. Signal Process. 2017;65(14):3702-3717.

[4] Ying J, Cai J-F, Guo D, et al. Vandermonde factorization of Hankel matrix for complex exponential signal recovery—application in fast NMR spectroscopy. IEEE Trans. Signal Process. 2018;66(21):5520-5533.

[5] Guo D, Lu H and Qu X. A fast low rank Hankel matrix factorization reconstruction method for non-uniformly sampled magnetic resonance spectroscopy. IEEE Access 2017;516033-16039.

[6] Lu H, Zhang X, Qiu T, et al. Low rank enhanced matrix recovery of hybrid time and frequency data in fast magnetic resonance spectroscopy. IEEE Trans. Biomed. Eng. 2018;65(4):809-820.

[7] Qu X, Huang Y, Lu H, et al. Accelerated nuclear magnetic resonance spectroscopy with deep learning. Angew. Chem.-Int. Edit. 2020;59(26):10297-10300.

[8] Chen D, Wang Z, Guo D, et al. Review and prospect: Deep learning in nuclear magnetic resonance spectroscopy. Chem.-Eur. J. 2020;2610391-10401.

[9] Huang G, Liu Z, Van Der Maaten L, et al. Densely connected convolutional networks, in Proc. IEEE Comput. Soc. Conf. Comput. Vision Pattern Recognit., 2017;4700-4708

Figure 1. The
architecture of DHMF. (a) the general process of the k-th block, (b) P
and Q modules with time domain convolution in the basic DHMF, (c) P and Q
modules with frequency domain convolution in the enhanced DHMF, (d) dense
convolutional neural network.

Figure 2. The reconstructed
spectra and singular values at each block. (a) fully sampled spectrum, (b-f)
the reconstructed spectrum by the 1st to 5th blocks, (g) the nuclear norm of
Hankel matrix of time-domain signal, and (h-l) denote corresponding singular
values of the output of each block. Note: To show small singular value clearly,
there exists a break from 0.084 to 0.085 in Y axis of (h-l).

Figure 3.
Reconstructed synthetic signals with low-intensity peaks. (a) is the fully
sampled noise-free signal, (b) is the noisy data with the additive Gaussian
noise under standard deviation of 0.05, (c-f) are reconstructions obtained by
LRHM, LRHMF, DLNMR, and DHMF under 25% sampling ratio, respectively, (h-m) are
the zoomed in weakest peaks marked by the arrow. (g) is the Pearsons linear
correlation coefficient of each peak.

Figure 4. The
reconstruction of 2D 1H-15N TROSY spectrum of ubiquitin
under 25% sampling ratio. (a) is the fully sampled NMR spectrum, (b-e) are the reconstructions
by LRHM, LRHMF, DLNMR, and DHMF, respectively. Gaussian noise with standard
deviation 2x10^{-2} is added to the fully sampled spectrum. (f) and (g)
are zoomed out 1D 15N traces, (h-k) are peak intensity correlations,
and the green, cyan, purple, blue, and red lines represent the spectra obtained
with full sampling, LRHM, LRHMF, DLNMR and DHMF, respectively.