arXiv subjects

Valentina Salvatelli

Publications and source records attributed to Valentina Salvatelli.

At least 19 recordsLinked to original sources

arXiv2025

Closing the Performance Gap Between AI and Radiologists in Chest X-Ray Reporting

AI-assisted report generation offers the opportunity to reduce radiologists' workload stemming from expanded screening guidelines, complex cases and workforce shortages, while maintaining diagnostic accuracy. In addition to describing pathological findings in chest X-ray reports, interpreting lines and tubes (L&T) is demanding and repetitive for radiologists, especially with high patient volumes. We introduce MAIRA-X, a clinically evaluated multimodal AI model for longitudinal chest X-ray (CXR) report generation, that encompasses both clinical findings and L&T reporting. Developed using a large-scale, multi-site, longitudinal dataset of 3.1 million studies (comprising 6 million images from 806k patients) from Mayo Clinic, MAIRA-X was evaluated on three holdout datasets and the public MIMIC-CXR dataset, where it significantly improved AI-generated reports over the state of the art on lexical quality, clinical correctness, and L&T-related elements. A novel L&T-specific metrics framework was developed to assess accuracy in reporting attributes such as type, longitudinal change and placement. A first-of-its-kind retrospective user evaluation study was conducted with nine radiologists of varying experience, who blindly reviewed 600 studies from distinct subjects. The user study found comparable rates of critical errors (3.0% for original vs. 4.6% for AI-generated reports) and a similar rate of acceptable sentences (97.8% for original vs. 97.4% for AI-generated reports), marking a significant improvement over prior user studies with larger gaps and higher error rates. Our results suggest that MAIRA-X can effectively assist radiologists, particularly in high-volume clinical settings.

cs.CL↗

arXiv2025

Comprehensive language-image pre-training for 3D medical image understanding

In the 3D medical image domain, vision-language pre-training is used to create vision-language encoders (VLEs) that can support radiologists by retrieving patients with similar abnormalities, predicting likelihoods of abnormality, or, with downstream adaptation, generating radiological reports. While the methodology holds promise, three challenges limit the capabilities of current 3D VLEs: data scarcity due to privacy concerns, high computational costs resulting from the volumetric nature of the images, and a domain shift between the long reports used for training and the short prompts used during inference for, e.g., zero-shot classification. As a consequence, natural-image VLE recipes do not directly transfer to 3D medical imaging. In this paper, we overcome these challenges by injecting additional supervision via a report generation objective and combining vision-language with vision-only pre-training, allowing us to leverage both image-only and paired image-text 3D datasets. Further, we propose a novel loss that addresses the domain shift between long reports and short textual prompts. Through these additional objectives, paired with best practices of the 3D medical imaging domain, we develop the Comprehensive Language-Image Pre-training (COLIPRI) encoder family. Our COLIPRI encoders achieve state-of-the-art performance in report generation, semantic segmentation, classification probing, and zero-shot classification. The model weights and inference code are freely available at https://huggingface.co/microsoft/colipri.

cs.CV↗

arXiv2025

Data Scaling Laws for Radiology Foundation Models

Foundation vision encoders such as CLIP and DINOv2, trained on web-scale data, exhibit strong transfer performance across tasks and datasets. However, medical imaging foundation models remain constrained by smaller datasets, limiting our understanding of how data scale and pretraining paradigms affect performance in this setting. In this work, we systematically study continual pretraining of two vision encoders, MedImageInsight (MI2) and RAD-DINO representing the two major encoder paradigms CLIP and DINOv2, on up to 3.5M chest x-rays from a single institution, holding compute and evaluation protocols constant. We evaluate on classification (radiology findings, lines and tubes), segmentation (lines and tubes), and radiology report generation. While prior work has primarily focused on tasks related to radiology findings, we include lines and tubes tasks to counterbalance this bias and evaluate a model's ability to extract features that preserve continuity along elongated structures. Our experiments show that MI2 scales more effectively for finding-related tasks, while RAD-DINO is stronger on tube-related tasks. Surprisingly, continually pretraining MI2 with both reports and structured labels using UniCL improves performance, underscoring the value of structured supervision at scale. We further show that for some tasks, as few as 30k in-domain samples are sufficient to surpass open-weights foundation models. These results highlight the utility of center-specific continual pretraining, enabling medical institutions to derive significant performance gains by utilizing in-domain data.

cs.CV↗

arXiv2024

MAIRA-Seg: Enhancing Radiology Report Generation with Segmentation-Aware Multimodal Large Language Models

There is growing interest in applying AI to radiology report generation, particularly for chest X-rays (CXRs). This paper investigates whether incorporating pixel-level information through segmentation masks can improve fine-grained image interpretation of multimodal large language models (MLLMs) for radiology report generation. We introduce MAIRA-Seg, a segmentation-aware MLLM framework designed to utilize semantic segmentation masks alongside CXRs for generating radiology reports. We train expert segmentation models to obtain mask pseudolabels for radiology-specific structures in CXRs. Subsequently, building on the architectures of MAIRA, a CXR-specialised model for report generation, we integrate a trainable segmentation tokens extractor that leverages these mask pseudolabels, and employ mask-aware prompting to generate draft radiology reports. Our experiments on the publicly available MIMIC-CXR dataset show that MAIRA-Seg outperforms non-segmentation baselines. We also investigate set-of-marks prompting with MAIRA and find that MAIRA-Seg consistently demonstrates comparable or superior performance. The results confirm that using segmentation masks enhances the nuanced reasoning of MLLMs, potentially contributing to better clinical outcomes.

cs.CV↗

arXiv2024

MAIRA-2: Grounded Radiology Report Generation

Radiology reporting is a complex task requiring detailed medical image understanding and precise language generation, for which generative multimodal models offer a promising solution. However, to impact clinical practice, models must achieve a high level of both verifiable performance and utility. We augment the utility of automated report generation by incorporating localisation of individual findings on the image - a task we call grounded report generation - and enhance performance by incorporating realistic reporting context as inputs. We design a novel evaluation framework (RadFact) leveraging the logical inference capabilities of large language models (LLMs) to quantify report correctness and completeness at the level of individual sentences, while supporting the new task of grounded reporting. We develop MAIRA-2, a large radiology-specific multimodal model designed to generate chest X-ray reports with and without grounding. MAIRA-2 achieves state of the art on existing report generation benchmarks and establishes the novel task of grounded report generation.

cs.CL↗

arXiv2024

Exploring scalable medical image encoders beyond text supervision

Language-supervised pre-training has proven to be a valuable method for extracting semantically meaningful features from images, serving as a foundational element in multimodal systems within the computer vision and medical imaging domains. However, the computed features are limited by the information contained in the text, which is particularly problematic in medical imaging, where the findings described by radiologists focus on specific observations. This challenge is compounded by the scarcity of paired imaging-text data due to concerns over leakage of personal health information. In this work, we fundamentally challenge the prevailing reliance on language supervision for learning general-purpose biomedical imaging encoders. We introduce RAD-DINO, a biomedical image encoder pre-trained solely on unimodal biomedical imaging data that obtains similar or greater performance than state-of-the-art biomedical language-supervised models on a diverse range of benchmarks. Specifically, the quality of learned representations is evaluated on standard imaging tasks (classification and semantic segmentation), and a vision-language alignment task (text report generation from images). To further demonstrate the drawback of language supervision, we show that features from RAD-DINO correlate with other medical records (e.g., sex or age) better than language-supervised models, which are generally not mentioned in radiology reports. Finally, we conduct a series of ablations determining the factors in RAD-DINO's performance; notably, we observe that RAD-DINO's downstream performance scales well with the quantity and diversity of training data, demonstrating that image-only supervision is a scalable approach for training a foundational biomedical image encoder. Model weights of RAD-DINO trained on publicly available datasets are available at https://huggingface.co/microsoft/rad-dino.

cs.CV↗

arXiv2023

RadEdit: stress-testing biomedical vision models via diffusion image editing

Biomedical imaging datasets are often small and biased, meaning that real-world performance of predictive models can be substantially lower than expected from internal testing. This work proposes using generative image editing to simulate dataset shifts and diagnose failure modes of biomedical vision models; this can be used in advance of deployment to assess readiness, potentially reducing cost and patient harm. Existing editing methods can produce undesirable changes, with spurious correlations learned due to the co-occurrence of disease and treatment interventions, limiting practical applicability. To address this, we train a text-to-image diffusion model on multiple chest X-ray datasets and introduce a new editing method RadEdit that uses multiple masks, if present, to constrain changes and ensure consistency in the edited images. We consider three types of dataset shifts: acquisition shift, manifestation shift, and population shift, and demonstrate that our approach can diagnose failures and quantify model robustness without additional data collection, complementing more qualitative tools for explainable AI.

cs.CV↗

arXiv2023

MAIRA-1: A specialised large multimodal model for radiology report generation

We present a radiology-specific multimodal model for the task for generating radiological reports from chest X-rays (CXRs). Our work builds on the idea that large language model(s) can be equipped with multimodal capabilities through alignment with pre-trained vision encoders. On natural images, this has been shown to allow multimodal models to gain image understanding and description capabilities. Our proposed model (MAIRA-1) leverages a CXR-specific image encoder in conjunction with a fine-tuned large language model based on Vicuna-7B, and text-based data augmentation, to produce reports with state-of-the-art quality. In particular, MAIRA-1 significantly improves on the radiologist-aligned RadCliQ metric and across all lexical metrics considered. Manual review of model outputs demonstrates promising fluency and accuracy of generated reports while uncovering failure modes not captured by existing evaluation practices. More information and resources can be found on the project website: https://aka.ms/maira.

cs.CL↗

arXiv2022

Exploring the Limits of Synthetic Creation of Solar EUV Images via Image-to-Image Translation

The Solar Dynamics Observatory (SDO), a NASA multi-spectral decade-long mission that has been daily producing terabytes of observational data from the Sun, has been recently used as a use-case to demonstrate the potential of machine learning methodologies and to pave the way for future deep-space mission planning. In particular, the idea of using image-to-image translation to virtually produce extreme ultra-violet channels has been proposed in several recent studies, as a way to both enhance missions with less available channels and to alleviate the challenges due to the low downlink rate in deep space. This paper investigates the potential and the limitations of such a deep learning approach by focusing on the permutation of four channels and an encoder--decoder based architecture, with particular attention to how morphological traits and brightness of the solar surface affect the neural network predictions. In this work we want to answer the question: can synthetic images of the solar corona produced via image-to-image translation be used for scientific studies of the Sun? The analysis highlights that the neural network produces high-quality images over three orders of magnitude in count rate (pixel intensity) and can generally reproduce the covariance across channels within a 1% error. However the model performance drastically diminishes in correspondence of extremely high energetic events like flares, and we argue that the reason is related to the rareness of such events posing a challenge to model training.

astro-ph.SR↗

arXiv2021

Unsupervised Change Detection of Extreme Events Using ML On-Board

In this paper, we introduce RaVAEn, a lightweight, unsupervised approach for change detection in satellite data based on Variational Auto-Encoders (VAEs) with the specific purpose of on-board deployment. Applications such as disaster management enormously benefit from the rapid availability of satellite observations. Traditionally, data analysis is performed on the ground after all data is transferred - downlinked - to a ground station. Constraint on the downlink capabilities therefore affects any downstream application. In contrast, RaVAEn pre-processes the sampled data directly on the satellite and flags changed areas to prioritise for downlink, shortening the response time. We verified the efficacy of our system on a dataset composed of time series of catastrophic events - which we plan to release alongside this publication - demonstrating that RaVAEn outperforms pixel-wise baselines. Finally we tested our approach on resource-limited hardware for assessing computational and memory limitations.

cs.LG↗

arXiv2020

Multi-Channel Auto-Calibration for the Atmospheric Imaging Assembly using Machine Learning

Solar activity plays a quintessential role in influencing the interplanetary medium and space-weather around the Earth. Remote sensing instruments onboard heliophysics space missions provide a pool of information about the Sun's activity via the measurement of its magnetic field and the emission of light from the multi-layered, multi-thermal, and dynamic solar atmosphere. Extreme UV (EUV) wavelength observations from space help in understanding the subtleties of the outer layers of the Sun, namely the chromosphere and the corona. Unfortunately, such instruments, like the Atmospheric Imaging Assembly (AIA) onboard NASA's Solar Dynamics Observatory (SDO), suffer from time-dependent degradation, reducing their sensitivity. Current state-of-the-art calibration techniques rely on periodic sounding rockets, which can be infrequent and rather unfeasible for deep-space missions. We present an alternative calibration approach based on convolutional neural networks (CNNs). We use SDO-AIA data for our analysis. Our results show that CNN-based models could comprehensively reproduce the sounding rocket experiments' outcomes within a reasonable degree of accuracy, indicating that it performs equally well compared with the current techniques. Furthermore, a comparison with a standard "astronomer's technique" baseline model reveals that the CNN approach significantly outperforms this baseline. Our approach establishes the framework for a novel technique to calibrate EUV instruments and advance our understanding of the cross-channel relation between different EUV channels.

astro-ph.SR↗

arXiv2019

Using U-Nets to Create High-Fidelity Virtual Observations of the Solar Corona

Understanding and monitoring the complex and dynamic processes of the Sun is important for a number of human activities on Earth and in space. For this reason, NASA's Solar Dynamics Observatory (SDO) has been continuously monitoring the multi-layered Sun's atmosphere in high-resolution since its launch in 2010, generating terabytes of observational data every day. The synergy between machine learning and this enormous amount of data has the potential, still largely unexploited, to advance our understanding of the Sun and extend the capabilities of heliophysics missions. In the present work, we show that deep learning applied to SDO data can be successfully used to create a high-fidelity virtual telescope that generates synthetic observations of the solar corona by image translation. Towards this end we developed a deep neural network, structured as an encoder-decoder with skip connections (U-Net), that reconstructs the Sun's image of one instrument channel given temporally aligned images in three other channels. The approach we present has the potential to reduce the telemetry needs of SDO, enhance the capabilities of missions that have less observing channels, and transform the concept development of future missions.

astro-ph.SR↗

arXiv2019

Auto-Calibration of Remote Sensing Solar Telescopes with Deep Learning

As a part of NASA's Heliophysics System Observatory (HSO) fleet of satellites,the Solar Dynamics Observatory (SDO) has continuously monitored the Sun since2010. Ultraviolet (UV) and Extreme UV (EUV) instruments in orbit, such asSDO's Atmospheric Imaging Assembly (AIA) instrument, suffer time-dependent degradation which reduces instrument sensitivity. Accurate calibration for (E)UV instruments currently depends on periodic sounding rockets, which are infrequent and not practical for heliophysics missions in deep space. In the present work, we develop a Convolutional Neural Network (CNN) that auto-calibrates SDO/AIA channels and corrects sensitivity degradation by exploiting spatial patterns in multi-wavelength observations to arrive at a self-calibration of (E)UV imaging instruments. Our results remove a major impediment to developing future HSOmissions of the same scientific caliber as SDO but in deep space, able to observe the Sun from more vantage points than just SDO's current geosynchronous orbit.This approach can be adopted to perform autocalibration of other imaging systems exhibiting similar forms of degradation

astro-ph.SR↗

arXiv2016

Power law cosmology model comparison with CMB scale information

Despite the ability of the cosmological concordance model ($\Lambda$CDM) to describe the cosmological observations exceedingly well, power law expansion of the Universe scale radius, $R(t)\propto t^n$, has been proposed as an alternative framework. We examine here these models, analyzing their ability to fit cosmological data using robust model comparison criteria. Type Ia supernovae (SNIa), baryonic acoustic oscillations (BAO) and acoustic scale information from the cosmic microwave background (CMB) have been used. We find that SNIa data either alone or combined with BAO can be well reproduced by both $\Lambda$CDM and power law expansion models with $n\sim 1.5$, while the constant expansion rate model $(n=1)$ is clearly disfavored. Allowing for some redshift evolution in the SNIa luminosity essentially removes any clear preference for a specific model. The CMB data are well known to provide the most stringent constraints on standard cosmological models, in particular, through the position of the first peak of the temperature angular power spectrum, corresponding to the sound horizon at recombination, a scale physically related to the BAO scale. Models with $n\geq 1$ lead to a divergence of the sound horizon and do not naturally provide the relevant scales for the BAO and the CMB. We retain an empirical footing to overcome this issue: we let the data choose the preferred values for these scales, while we recompute the ionization history in power law models, to obtain the distance to the CMB. In doing so, we find that the scale coming from the BAO data is not consistent with the observed position of the first peak of the CMB temperature angular power spectrum for any power law cosmology. Therefore, we conclude that when the three standard probes are combined, the $\Lambda$CDM model is very strongly favored over any of these alternative models, which are then essentially ruled out.

astro-ph.CO↗

arXiv2016

Constraints on modified gravity from Planck 2015: when the health of your theory makes the difference

We use the effective field theory of dark energy (EFT of DE) formalism to constrain dark energy models belonging to the Horndeski class with the recent Planck 2015 CMB data. The space of theories is spanned by a certain number of parameters determining the linear cosmological perturbations, while the expansion history is set to that of a standard $\Lambda$CDM model. We always demand that the theories be free of fatal instabilities. Additionally, we consider two optional conditions, namely that scalar and tensor perturbations propagate with subliminal speed. Such criteria severely restrict the allowed parameter space and are thus very effective in shaping the posteriors. As a result, we confirm that no theory performs better than $\Lambda$CDM when CMB data alone are analysed. Indeed, the healthy dark energy models considered here are not able to reproduce those phenomenological behaviours of the effective Newton constant and gravitational slip parameters that, according to previous studies, best fit the data.

astro-ph.CO↗

arXiv2014

Indications of a late-time interaction in the dark sector

We show that a general late-time interaction between cold dark matter and vacuum energy is favoured by current cosmological datasets. We characterize the strength of the coupling by a dimensionless parameter $q_V$ that is free to take different values in four redshift bins from the primordial epoch up to today. This interacting scenario is in agreement with measurements of cosmic microwave background temperature anisotropies from the Planck satellite, supernovae Ia from Union 2.1 and redshift space distortions from a number of surveys, as well as with combinations of these different datasets. We show that a non-zero interaction is very likely at late times. We then focus on the case $q_V\not=0$ in a single low-redshift bin, obtaining a nested one parameter extension of the standard $\Lambda$CDM model. We study the Bayesian evidence, with respect to $\Lambda$CDM, of this late-time interaction model, finding moderate evidence for an interaction starting at $z=0.9$, dependent upon the prior range chosen for the interaction strength parameter $q_V$. For this case the null interaction ($q_V=0$, i.e.$\Lambda$CDM) is excluded at 99% c.l..

astro-ph.CO↗

arXiv2013

Far-infrared Fine-Structure Line Diagnostics of Ultraluminous Infrared Galaxies

We present Herschel observations of six fine-structure lines in 25 Ultraluminous Infrared Galaxies at z<0.27. The lines, [O III]52, [N III]57, [O I]63, [N II]122, [O I]145, and [C II]158, are mostly single gaussians with widths <600 km s-1 and luminosities of 10^7 - 10^9 Solar. There are deficits in the [O I]63/L_IR, [N II]/L_IR, [O I]145/L_IR, and [C II]/L_IR ratios compared to lower luminosity systems. The majority of the line deficits are consistent with dustier H II regions, but part of the [C II] deficit may arise from an additional mechanism, plausibly charged dust grains. This is consistent with some of the [C II] originating from PDRs or the ISM. We derive relations between far-IR line luminosities and both IR luminosity and star formation rate. We find that [N II] and both [O I] lines are good tracers of IR luminosity and star formation rate. In contrast, [C II] is a poor tracer of IR luminosity and star formation rate, and does not improve as a tracer of either quantity if the [C II] deficit is accounted for. The continuum luminosity densities also correlate with IR luminosity and star formation rate. We derive ranges for the gas density and ultraviolet radiation intensity of 10^1 < n < 10^2.5 and 10^2.2 < G_0 < 10^3.6, respectively. These ranges depend on optical type, the importance of star formation, and merger stage. We do not find relationships between far-IR line properties and several other parameters; AGN activity, merger stage, mid-IR excitation, and SMBH mass. We conclude that these far-IR lines arise from gas heated by starlight, and that they are not strongly influenced by AGN activity.

astro-ph.CO↗

arXiv2013

Updated constraints from the PLANCK experiment on modified gravity

A modification of the action of the general relativity produces a different pattern for the growth of the cosmic structures below a certain length-scale leaving an imprint on the cosmic microwave background (CMB) anisotropies. We re-examine the upper limits on the length-scale parameter B0 of f (R) models using the recent data from the Planck satellite experiment. We also investigate the combined constraints obtained when including the Hubble Space Telescope H0 measurement and the baryon acoustic oscillations measurements from the SDSS, WiggleZ and BOSS surveys.

astro-ph.CO↗