Artificial Observer and Cost Function for Image Registration, MARLINA: Mean Absolute Regional LINear correlation Algorithm
Roman Fleysher1, Lazar Fleysher2, Asif Suri1, Molly Zimmerman3, Mark Jenkinson4, Craig A Branch1, and Michael L Lipton1

1Department of Radiology, Gruss Magnetic Resonance Research Center, Albert Einstein College of Medicine and Montefiore Medical Center, Bronx, NY, United States, 2Department of Radiology, Mount Sinai School of Medicine, New York, NY, United States, 3Department of Psychology, Fordham University, Bronx, NY, United States, 4Nuffield Department of Clinical Neurosciences, Wellcome Centre for Integrative Neuroimaging, FMRIB Centre, University of Oxford, Oxford, United Kingdom


Upon visual inspection of intra-subject rigid body registrations in large studies, we have observed higher than desired rate of unsatisfactory alignments. To address misregistartions, we designed a battery of 13 candidate transformations, one of which was selected as best during visual inspection. Tediousness of the inspections stimulated development of artificial observer to aid and subsequently to replace the human inspector. Here, we describe artificial observer MARLINA, characterize its ability to identify the best rigid body transformation as compared to human inspectors and propose it as a future cost function.


Intra-subject rigid body image registration is crucial step in image analyses. Success of the search for the best transformation relies largely on two components: the optimization algorithm and the cost function. Identification and matching of homologous anatomical features is complicated by spatial resolution and contrast, therefore cost functions are sometimes assisted by tissue masks1,2. Despite existence of great many good cost functions3,4, upon visual inspection of intra-subject rigid body registrations in large studies, we have observed higher than desired rate of unsatisfactory alignments. This observation stimulated development of artificial observer to aid the visual inspection process and subsequently to replace the human inspector. We named the artificial observer MARLINA: Mean Absolute Regional LINear correlation Algorithm.


This study was approved by institutional review board of Albert Einstein College of Medicine. Initial development and testing was performed using data from the Einstein Aging Study and the Einstein Soccer Study. For rigorous testing and performance assessment, we used 75 datasets obtained as part of ongoing Einstein Lifespan Study (ELS), age 18-75 years, 49.3% female. All images were reviewed by an experienced neuroradiologist and determined to be free of visible structural abnormalities. Imaging was performed using a 3.0T Philips Achieva TX scanner (Philips Medical Systems, Best, The Netherlands) utilizing its 32-channel head coil consisting of 3 acquisitions: 1mm isotropic T1W, 2mm isotropic DTI and 4mm isotropic field map as previously reported5. DTI data were eddy corrected; all brain extractions were visually inspected and corrected as necessary.

To avoid misinterpretation of image distortions as misregistration, input and reference images must have the same distortions. Therefore, field map was first distorted the same way DTI and T1W using FUGUE of FSL3. These differently distorted field maps served as sources for B0map to DTI and B0map to T1W registrations. The best of these registrations were used to correct EPI distortions in DTI and small distortions in T1W. These corrected images were used for DTI to T1W registration. All registrations were limited to 6 degrees of freedom because they are intra-subject.

Each pair of images was registered using 13 different algorithms (Figure 1). Quality of each registration was evaluated by MARLINA, constructed as follows. To deal with varying contrasts of inter-modality images and with inhomogeneous coil profiles, linear correlation coefficient between non-zero image intensities was computed in patches of 14mm cubes (Figure 2). Patches having too low spread of non-zero intensities were discarded as having no internal features to match. The average of absolute magnitudes of correlations from all patches was deducted from 1 to obtain final value, ranging from 0 (best match) to 1. This construction mimics behavior of a human observer evaluating match of local features, one locality at a time. The remaining 12 transformations were ranked according to distance to the MARLINA best (Figure 3).

For visual inspection, the 13 candidates were randomly shuffled and presented in fslview together with the target image. The inspector was forced to choose the best registration and to assess if it is acceptable. The standard inspection protocol began with rejection of grossly bad registrations, followed by inspection of smaller structures down to a few very similar candidates where decision was based on a displacement of half a thin sulcus in the part of the brain with the most prominent motion between the candidates.


According to MARLINA and visual inspection, all 13 registration methods produce good registrations in varying proportions and none of them is universally good (Figure 1) confirming our observation which motivated the study. Good agreement between MARLINA and inspectors is shown in Figure 4. If 12 out of 13 candidates were very bad, then MARLINA and inspectors would surely agree on the best. Inspection process and distance measurements between MARLINA best and the other 12 candidates reveal very tough competition for the title of the best transformation (Figure 4). In particular, last few registrations of B0map to DTI at the end of inspection process were extremely similar often requiring inspectors to choose at random. Once random, seldom agreement is expected with larger distances to MARLINA (Figure 5).


MARLINA demonstrated good agreement with human observers in identifying the best transformation. Because none of the 13 registration methods is universally good, to increase robustness of registrations all algorithms can be used concurrently to produce 13 candidates to be forwarded to MARLINA for inspection and final selection. We expect that converting MARLINA from an artificial observer to the actual cost function will produce as good or more ideal transformations without assistance from tissue segmentation.


Support for this research was provided in part by the National Institute on Aging, grant 5P01AG003949-34 and by National Institute of Neurological Disorders and Stroke, grant 5R01NS082432-06.


1. Greve D, Fischl B. Accurate and robust brain image alignment using boundary-based registration. NeuroImage 2009; 48: 63–72.

2. Fleysher R, Lipton M, Noskin O, et al. White matter structural integrity and transcranial Doppler blood flow pulsatility in normal aging. Magn Reson Imaging. 2018; 47: 97-102.

3. Jenkinson M, Bannister P, Brady M, Smith S. Improved Optimization for the Robust and Accurate Linear Registration and Motion Correction of Brain Images. NeuroImage 2002; 17: 825–841.

4. Saad Z, Glen D, Chen G, et al. A new method for improving functional-to-structural MRI alignment using local Pearson correlation. NeuroImage 2009; 44: 839–848.

5. Fleysher R, Gil N, Lipton M, Branch C. Registration Quality Filtering Improves Robustness of Voxel-Wise Analyses to the Choice of Brain Template. These Proceedings.


Figure 1. List of methods used to generate 13 candidate transformations using FLIRT of FSL. Inputs and references are either brains or tissues segmented by FAST of FSL. The transformations were applied to the input brain to obtain brain registered to the target. These 13 transformations are meant to emulate convergence during registration process as it may happen inside FLIRT. Last column is how often the method was MARLINA’s best.

Figure 2. Schematic scatter plot of intensities of well-registered images. Tissue intensities do not necessarily follow the same order due to different MRI contrast of the images. Red dotted line represents correlation over the full brain. Its maximization would force incorrect image rotation/translation to match either low-to-low intensity (tissue 1 of image 1 to tissue 3 of image 2) and high-to-high (tissue 3 of image 1 to tissue 2 of image 2) or low-to-high and high-to-low. Green lines represent correlations computed locally where only two tissues dominate. Patch size of 14mm is based on preliminary testing.

Figure 3. Let Mm be the transformation chosen by MARLINA and Mc one of the other 12 transforms. We computed difference transform Md=Mc-1 Mm and converted it to absolute displacement in each voxel. This is the transformation that must be applied after Mc to turn it into MARLINA best. The 99th percentile of these displacements over the intersection of the brains is used as a measure of the distance between the candidate transformation and MARLINA. That is, 99% of voxels are shifted by less than this number of millimeters compared to the MARLINA best.

Figure 4. MARLINA cost function was used to determine the best transformation; the remaining 12 transformations were assigned a distance score (blue crosses) according to the method shown in Figure 3. This distance score was used to compare the choice made by MARLINA (cross at y=0) with those made by the human observers (green circles), in a way that was not dependent on the cost function (which would be circular) and had a practical, intuitive interpretation in units of millimeters. A total of 53 sets of 13 registrations were inspected

Figure 5. Example disposition of 13 registration candidates with several similarly good. White is the ideal registration (not one of the 13), blue is MARLINA best, green are the other 12. Human randomly selects the blue or one of the other 4 within the inner red circle because they are all similarly good. Human and MARLINA may appear to disagree and distance between human and MARLINA best may appear high. In fact, MARLINA mimics human behavior well. Only when the next best candidate is sufficiently far can human and MARLINA be absolutely unanimous.

Proc. Intl. Soc. Mag. Reson. Med. 27 (2019)