publications

2025

1. Super-Resolution in Clinically Available Spinal Cord MRIs Enables Automated Atrophy Analysis

   Blake E. Dewey, Samuel W. Remedios, Muraleetharan Sanjayan, Nicole Bou Rjeily, Alexandra Zambriczki Lee, Chelsea Wyche, Safiya Duncan, Jerry L. Prince, Peter A. Calabresi, Kathryn C. Fitzgerald, and Ellen M. Mowry

   American Journal of Neuroradiology, 2025

   Abs DOI

   

   BACKGROUND AND PURPOSE: Measurement of the mean upper cervical cord area (MUCCA) is an important biomarker in the study of neurodegeneration. However, dedicated high-resolution (HR) scans of the cervical spinal cord are rare in standard-of-care imaging due to timing and clinical usability. Most clinical cervical spinal cord imaging is sagittally acquired in 2D with thick slices and anisotropic voxels. As a solution, previous work describes HR T1-weighted brain imaging for measuring the upper cord area, but this is still not common in clinical care.MATERIALS AND METHODS: We propose using a zero-shot super-resolution technique, synthetic multi-orientation resolution enhancement (SMORE), already validated in the brain, to enhance the resolution of 2D-acquired scans for upper cord area calculations. To incorporate super-resolution in spinal cord analysis, we validate SMORE against HR research imaging and in a real-world longitudinal data analysis.RESULTS: Super-resolved (SR) images reconstructed by using SMORE showed significantly greater similarity to the ground truth than low-resolution (LR) images across all tested resolutions (P < .001 for all resolutions in peak signal-to-noise ratio [PSNR] and mean structural similarity [MSSIM]). MUCCA results from SR scans demonstrate excellent correlation with HR scans (r > 0.973 for all resolutions) compared with LR scans. Additionally, SR scans are consistent between resolutions (r > 0.969), an essential factor in longitudinal analysis. Compared with clinical outcomes such as walking speed or disease severity, MUCCA values from LR scans have significantly lower correlations than those from HR scans. SR results have no significant difference. In a longitudinal real-world data set, we show that these SR volumes can be used in conjunction with T1-weighted brain scans to show a significant rate of atrophy (-0.790, P = .020 versus -0.438, P = .301 with LR).CONCLUSIONS: Super-resolution is a valuable tool for enabling large-scale studies of cord atrophy, as LR images acquired in clinical practice are common and available.9HPT9-hole peg testCSCcervical spinal cordEDSSExpanded Disability Status ScaleHRhigh-resolutionLRlow-resolutionMSFCMS functional compositeMSSIMmean structural similarityMUCCAmean upper cervical cord areaPMJpontomedullary junctionPSNRpeak signal-to-noise ratioSMOREsynthetic multi-orientation resolution enhancementSRsuper-resolvedT25FWtimed 25-foot walk

2. ECLARE: Efficient cross-planar learning for anisotropic resolution enhancement

   Samuel W. Remedios, Shuwen Wei, Shuo Han, Jinwei Zhang, Aaron Carass, Kurt G. Schilling, Dzung L. Pham, Jerry L. Prince, and Blake E. Dewey

   2025

   Abs DOI arXiv

   

   In clinical imaging, magnetic resonance (MR) image volumes are often acquired as stacks of 2D slices with decreased scan times, improved signal-to-noise ratio, and image contrasts unique to 2D MR pulse sequences. While this is sufficient for clinical evaluation, automated algorithms designed for 3D analysis perform poorly on multi-slice 2D MR volumes, especially those with thick slices and gaps between slices. Super-resolution (SR) methods aim to address this problem, but previous methods do not address all of the following: slice profile shape estimation, slice gap, domain shift, and non-integer or arbitrary upsampling factors. In this paper, we propose ECLARE (Efficient Cross-planar Learning for Anisotropic Resolution Enhancement), a self-SR method that addresses each of these factors. ECLARE uses a slice profile estimated from the multi-slice 2D MR volume, trains a network to learn the mapping from low-resolution to high-resolution in-plane patches from the same volume, and performs SR with anti-aliasing. We compared ECLARE to cubic B-spline interpolation, SMORE, and other contemporary SR methods. We used realistic and representative simulations so that quantitative performance against ground truth can be computed, and ECLARE outperformed all other methods in both signal recovery and downstream tasks. Importantly, as ECLARE does not use external training data it cannot suffer from domain shift between training and testing.

2024

1. Development of Medical Imaging Data Standardization for Imaging-Based Observational Research: OMOP Common Data Model Extension

   Woo Yeon Park, Kyulee Jeon, Teri Sippel Schmidt, Haridimos Kondylakis, Tarik Alkasab, Blake E. Dewey, Seng Chan You, and Paul Nagy

   Journal of Imaging Informatics in Medicine, Apr 2024

   Abs DOI

   

   The rapid growth of artificial intelligence (AI) and deep learning techniques require access to large inter-institutional cohorts of data to enable the development of robust models, e.g., targeting the identification of disease biomarkers and quantifying disease progression and treatment efficacy. The Observational Medical Outcomes Partnership Common Data Model (OMOP CDM) has been designed to accommodate a harmonized representation of observational healthcare data. This study proposes the Medical Imaging CDM (MI-CDM) extension, adding two new tables and two vocabularies to the OMOP CDM to address the structural and semantic requirements to support imaging research. The tables provide the capabilities of linking DICOM data sources as well as tracking the provenance of imaging features derived from those images. The implementation of the extension enables phenotype definitions using imaging features and expanding standardized computable imaging biomarkers. This proposal offers a comprehensive and unified approach for conducting imaging research and outcome studies utilizing imaging features.

2023

1. Self-Supervised Super-Resolution for Anisotropic MR Images with and Without Slice Gap

   Samuel W. Remedios, Shuo Han, Lianrui Zuo, Aaron Carass, Dzung L. Pham, Jerry L. Prince, and Blake E. Dewey

   In Simulation and Synthesis in Medical Imaging, Apr 2023

   Abs DOI

   

   Magnetic resonance (MR) images are often acquired as multi-slice volumes to reduce scan time and motion artifacts while improving signal-to-noise ratio. These slices often are thicker than their in-plane resolution and sometimes are acquired with gaps between slices. Such thick-slice image volumes (possibly with gaps) can impact the accuracy of volumetric analysis and 3D methods. While many super-resolution (SR) methods have been proposed to address thick slices, few have directly addressed the slice gap scenario. Furthermore, data-driven methods are sensitive to domain shift due to the variability of resolution, contrast in acquisition, pathology, and differences in anatomy. In this work, we propose a self-supervised SR technique to address anisotropic MR images with and without slice gap. We compare against competing methods and validate in both signal recovery and downstream task performance on two open-source datasets and show improvements in all respects. Our code publicly available at https://gitlab.com/iacl/smore.

2. HACA3: A unified approach for multi-site MR image harmonization

   Lianrui Zuo, Yihao Liu, Yuan Xue, Blake E. Dewey, Samuel W. Remedios, Savannah P. Hays, Murat Bilgel, Ellen M. Mowry, Scott D. Newsome, Peter A. Calabresi, Susan M. Resnick, Jerry L. Prince, and 1 more author

   Computerized Medical Imaging and Graphics, Apr 2023

   Abs DOI

   

   The lack of standardization and consistency of acquisition is a prominent issue in magnetic resonance (MR) imaging. This often causes undesired contrast variations in the acquired images due to differences in hardware and acquisition parameters. In recent years, image synthesis-based MR harmonization with disentanglement has been proposed to compensate for the undesired contrast variations. The general idea is to disentangle anatomy and contrast information from MR images to achieve cross-site harmonization. Despite the success of existing methods, we argue that major improvements can be made from three aspects. First, most existing methods are built upon the assumption that multi-contrast MR images of the same subject share the same anatomy. This assumption is questionable, since different MR contrasts are specialized to highlight different anatomical features. Second, these methods often require a fixed set of MR contrasts for training (e.g., both T1-weighted and T2-weighted images), limiting their applicability. Lastly, existing methods are generally sensitive to imaging artifacts. In this paper, we present Harmonization with Attention-based Contrast, Anatomy, and Artifact Awareness (HACA3), a novel approach to address these three issues. HACA3 incorporates an anatomy fusion module that accounts for the inherent anatomical differences between MR contrasts. Furthermore, HACA3 can be trained and applied to any combination of MR contrasts and is robust to imaging artifacts. HACA3 is developed and evaluated on diverse MR datasets acquired from 21 sites with varying field strengths, scanner platforms, and acquisition protocols. Experiments show that HACA3 achieves state-of-the-art harmonization performance under multiple image quality metrics. We also demonstrate the versatility and potential clinical impact of HACA3 on downstream tasks including white matter lesion segmentation for people with multiple sclerosis and longitudinal volumetric analyses for normal aging subjects. Code is available at https://github.com/lianruizuo/haca3.

2022

1. Chapter 11 - Medical image harmonization through synthesis

   Blake E. Dewey, Yufan He, Yihao Liu, Lianrui Zuo, and Jerry L. Prince

   In Biomedical Image Synthesis and Simulation, Apr 2022

   Abs DOI

   

   Differences in acquisition hardware and acquisition protocols often cause medical image analysis methods to be unreliable or inaccurate. This is especially problematic in modern deep learning-based methods, which are typically highly tuned to a particular type of input data during training. Image synthesis can be used to overcome acquisition variability by creating harmonized versions of images, which mimic a standard acquisition and produce robust results. This chapter considers how modern image synthesis can be used to create realistic and reliable harmonized images in magnetic resonance imaging and optical coherence tomography.

2021

1. SMORE: A Self-Supervised Anti-Aliasing and Super-Resolution Algorithm for MRI Using Deep Learning

   Can Zhao, Blake E. Dewey, Dzung L. Pham, Peter A. Calabresi, Daniel S. Reich, and Jerry L. Prince

   IEEE Transactions on Medical Imaging, Apr 2021

   Abs DOI

   

   High resolution magnetic resonance (MR) images are desired in many clinical and research applications. Acquiring such images with high signal-to-noise (SNR), however, can require a long scan duration, which is difficult for patient comfort, is more costly, and makes the images susceptible to motion artifacts. A very common practical compromise for both 2D and 3D MR imaging protocols is to acquire volumetric MR images with high in-plane resolution, but lower through-plane resolution. In addition to having poor resolution in one orientation, 2D MRI acquisitions will also have aliasing artifacts, which further degrade the appearance of these images. This paper presents an approach SMORE based on convolutional neural networks (CNNs) that restores image quality by improving resolution and reducing aliasing in MR images. This approach is self-supervised, which requires no external training data because the high-resolution and low-resolution data that are present in the image itself are used for training. For 3D MRI, the method consists of only one self-supervised super-resolution (SSR) deep CNN that is trained from the volumetric image data. For 2D MRI, there is a self-supervised anti-aliasing (SAA) deep CNN that precedes the SSR CNN, also trained from the volumetric image data. Both methods were evaluated on a broad collection of MR data, including filtered and downsampled images so that quantitative metrics could be computed and compared, and actual acquired low resolution images for which visual and sharpness measures could be computed and compared. The super-resolution method is shown to be visually and quantitatively superior to previously reported methods

2. Unsupervised MR harmonization by learning disentangled representations using information bottleneck theory

   Lianrui Zuo, Blake E. Dewey, Yihao Liu, Yufan He, Scott D. Newsome, Ellen M. Mowry, Susan M. Resnick, Jerry L. Prince, and Aaron Carass

   NeuroImage, Apr 2021

   Abs DOI

   

   In magnetic resonance (MR) imaging, a lack of standardization in acquisition often causes pulse sequence-based contrast variations in MR images from site to site, which impedes consistent measurements in automatic analyses. In this paper, we propose an unsupervised MR image harmonization approach, CALAMITI (Contrast Anatomy Learning and Analysis for MR Intensity Translation and Integration), which aims to alleviate contrast variations in multi-site MR imaging. Designed using information bottleneck theory, CALAMITI learns a globally disentangled latent space containing both anatomical and contrast information, which permits harmonization. In contrast to supervised harmonization methods, our approach does not need a sample population to be imaged across sites. Unlike traditional unsupervised harmonization approaches which often suffer from geometry shifts, CALAMITI better preserves anatomy by design. The proposed method is also able to adapt to a new testing site with a straightforward fine-tuning process. Experiments on MR images acquired from ten sites show that CALAMITI achieves superior performance compared with other harmonization approaches.

3. Information-Based Disentangled Representation Learning for Unsupervised MR Harmonization

   Lianrui Zuo, Blake E. Dewey, Aaron Carass, Yihao Liu, Yufan He, Peter A. Calabresi, and Jerry L. Prince

   In Information Processing in Medical Imaging, Apr 2021

   Abs DOI

   

   Accuracy and consistency are two key factors in computer-assisted magnetic resonance (MR) image analysis. However, contrast variation from site to site caused by lack of standardization in MR acquisition impedes consistent measurements. In recent years, image harmonization approaches have been proposed to compensate for contrast variation in MR images. Current harmonization approaches either require cross-site traveling subjects for supervised training or heavily rely on site-specific harmonization models to encourage harmonization accuracy. These requirements potentially limit the application of current harmonization methods in large-scale multi-site studies. In this work, we propose an unsupervised MR harmonization framework, CALAMITI (Contrast Anatomy Learning and Analysis for MR Intensity Translation and Integration), based on information bottleneck theory. CALAMITI learns a disentangled latent space using a unified structure for multi-site harmonization without the need for traveling subjects. Our model is also able to adapt itself to harmonize MR images from a new site with fine tuning solely on images from the new site. Both qualitative and quantitative results show that the proposed method achieves superior performance compared with other unsupervised harmonization approaches.

4. MTT and Blood-Brain Barrier Disruption within Asymptomatic Vascular WM Lesions

   B.E. Dewey, X. Xu, L. Knutsson, A. Jog, J.L. Prince, P.B. Barker, P.C.M. Zijl, R. Leigh, and P. Nyquist

   American Journal of Neuroradiology, Apr 2021

   Abs DOI

   

   BACKGROUND AND PURPOSE: White matter lesions of presumed ischemic origin are associated with progressive cognitive impairment and impaired BBB function. Studying the longitudinal effects of white matter lesion biomarkers that measure changes in perfusion and BBB patency within white matter lesions is required for long-term studies of lesion progression. We studied perfusion and BBB disruption within white matter lesions in asymptomatic subjects.MATERIALS AND METHODS: Anatomic imaging was followed by consecutive dynamic contrast-enhanced and DSC imaging. White matter lesions in 21 asymptomatic individuals were determined using a Subject-Specific Sparse Dictionary Learning algorithm with manual correction. Perfusion-related parameters including CBF, MTT, the BBB leakage parameter, and volume transfer constant were determined.RESULTS: MTT was significantly prolonged (7.88 [SD, 1.03] seconds) within white matter lesions compared with normal-appearing white (7.29 [SD, 1.14] seconds) and gray matter (6.67 [SD, 1.35] seconds). The volume transfer constant, measured by dynamic contrast-enhanced imaging, was significantly elevated (0.013 [SD, 0.017] minutes-1) in white matter lesions compared with normal-appearing white matter (0.007 [SD, 0.011] minutes-1). BBB disruption within white matter lesions was detected relative to normal white and gray matter using the DSC-BBB leakage parameter method so that increasing BBB disruption correlated with increasing white matter lesion volume (Spearman correlation coefficient = 0.44; P < .046).CONCLUSIONS: A dual-contrast-injection MR imaging protocol combined with a 3D automated segmentation analysis pipeline was used to assess BBB disruption in white matter lesions on the basis of quantitative perfusion measures including the volume transfer constant (dynamic contrast-enhanced imaging), the BBB leakage parameter (DSC), and MTT (DSC). This protocol was able to detect early pathologic changes in otherwise healthy individuals.cSVDcerebrovascular small-vessel diseaseDCEdynamic contrast-enhancedGdgadoliniumK2BBB leakage parameterKtransvolume transfer constantWMLwhite matter lesion

2020

1. A Disentangled Latent Space for Cross-Site MRI Harmonization

   Blake E. Dewey, Lianrui Zuo, Aaron Carass, Yufan He, Yihao Liu, Ellen M. Mowry, Scott Newsome, Jiwon Oh, Peter A. Calabresi, and Jerry L. Prince

   In Medical Image Computing and Computer Assisted Intervention – MICCAI 2020, Apr 2020

   Abs DOI

   

   Accurate interpretation and quantification of magnetic resonance imaging (MRI) is vital to medical research and clinical practice. However, lack of MRI standardization and differences in acquisition protocols often lead to measurement inconsistencies across sites. Image harmonization techniques have been shown to improve qualitative and quantitative consistency between differently acquired scans. Unfortunately, these methods typically require paired training data from traveling subjects (for supervised methods) or assumptions about anatomical similarities between the populations (for unsupervised methods). We propose a deep learning-based harmonization technique with limited supervision for use in standardization across scanners and sites. By leveraging a disentangled latent space represented by a high-resolution anatomical information component (}}\backslashbeta }}β) and a low-dimensional contrast component (}}\backslashtheta }}θ), the proposed method trains a cross-site harmonization model using databases of multi-modal image pairs acquired separately from each of the scanners to be harmonized. In this manuscript, we show that by using T}}_1}}1-weighted and T}}_2}}2-weighted images acquired from different subjects at three different sites, we can achieve a stable extraction of }}\backslashbeta }}\betawith a continuous representation of }}\backslashtheta }}θ. We also demonstrate that this allows cross-site harmonization without the need for paired data between sites.

2019

1. DeepHarmony: A deep learning approach to contrast harmonization across scanner changes

   Blake E. Dewey, Can Zhao, Jacob C. Reinhold, Aaron Carass, Kathryn C. Fitzgerald, Elias S. Sotirchos, Shiv Saidha, Jiwon Oh, Dzung L. Pham, Peter A. Calabresi, Peter C.M. van Zijl, and Jerry L. Prince

   Magnetic Resonance Imaging, Apr 2019

   Abs DOI

   

   Magnetic resonance imaging (MRI) is a flexible medical imaging modality that often lacks reproducibility between protocols and scanners. It has been shown that even when care is taken to standardize acquisitions, any changes in hardware, software, or protocol design can lead to differences in quantitative results. This greatly impacts the quantitative utility of MRI in multi-site or long-term studies, where consistency is often valued over image quality. We propose a method of contrast harmonization, called DeepHarmony, which uses a U-Net-based deep learning architecture to produce images with consistent contrast. To provide training data, a small overlap cohort (n = 8) was scanned using two different protocols. Images harmonized with DeepHarmony showed significant improvement in consistency of volume quantification between scanning protocols. A longitudinal MRI dataset of patients with multiple sclerosis was also used to evaluate the effect of a protocol change on atrophy calculations in a clinical research setting. The results show that atrophy calculations were substantially and significantly affected by protocol change, whereas such changes have a less significant effect and substantially reduced overall difference when using DeepHarmony. This establishes that DeepHarmony can be used with an overlap cohort to reduce inconsistencies in segmentation caused by changes in scanner protocol, allowing for modernization of hardware and protocol design in long-term studies without invalidating previously acquired data.

2. Evaluating the impact of intensity normalization on MR image synthesis

   Jacob C. Reinhold, Blake E. Dewey, Aaron Carass, and Jerry L. Prince

   In Medical Imaging 2019: Image Processing, Apr 2019

   Abs DOI

   Image synthesis learns a transformation from the intensity features of an input image to yield a different tissue contrast of the output image. This process has been shown to have application in many medical image analysis tasks including imputation, registration, and segmentation. To carry out synthesis, the intensities of the input images are typically scaled—i.e., normalized—both in training to learn the transformation and in testing when applying the transformation, but it is not presently known what type of input scaling is optimal. In this paper, we consider seven different intensity normalization algorithms and three different synthesis methods to evaluate the impact of normalization. Our experiments demonstrate that intensity normalization as a preprocessing step improves the synthesis results across all investigated synthesis algorithms. Furthermore, we show evidence that suggests intensity normalization is vital for successful deep learning-based MR image synthesis.