arXiv Science⌕ Search

arXiv subjects

Sebastien Ourselin

Publications and source records attributed to Sebastien Ourselin.

At least 19 recordsLinked to original sources

Retrieval-Augmented Correction for Generalist Medical Image Segmentation

Automated segmentation is routinely used as the first step in quantitative image-based clinical workflows, where predicted masks are subsequently reviewed and corrected by human operators. Automated correction of the residual structural errors is currently addressed one anatomy at a time, so the number of trained correctors scales with the number of segmentation targets. We present GRACE, a retrieval-augmented refiner that edits an input mask under context drawn from a shared knowledge base of verified prior cases. The refiner is trained once on a multi-organ corpus spanning compact organs, cavities, branching vessels, and tumour lesions. In the held-out dataset-transfer evaluation, three held-out datasets query a fixed knowledge base built from training data only. A separate leave-one-anatomy-out experiment tests parameter-free index extension: verified cases of an unseen solid organ enter the knowledge base, and the refiner stays frozen. A light shape-aware projection head on a frozen multimodal encoder maps embedding similarity to the shape agreement of the masks, so the knowledge base returns cases that the refiner can use as a shape prior. We evaluate GRACE on 18 public datasets spanning CT, MRI and ultrasound, using nine refinement configurations and seven upstream segmentors that span weak, mid, and strong operating regimes. We report each change with the DSC of its input mask. On the weak tier, the input DSC is 0.386 and GRACE adds up to +0.35 DSC. On a strong segmentor, the input DSC is 0.932 and the mean change is small. Transfer to an imaging modality that is absent from training and from the index is partial and depends on the anatomy, and remains an open research direction. Our results suggest retrieval-conditioned refinement is a practical alternative to anatomy-specific correction models. Code: https://github.com/robustml-eurecom/grace

cs.CV↗

Scan-specific k-space interpolation using deep Koopman autoencoders

Purpose: To develop a scan-specific, interpretable k-space interpolation technique for the reconstruction of accelerated MRI data. Methods: A deep Koopman autoencoder for nonlinear k-space interpolation was developed consisting of an encoder, a linear layer for interpolation and a decoder. The resulting network is implemented using mean square error loss function on k-space neighbourhoods derived from the autocalibration signals (ACS), combining a prediction loss and an autoencoding loss. The approach was compared to GRAPPA and RAKI in 281 retrospectively undersampled brain datasets, using quantitative reconstruction metrics. In a subset of 52 randomly selected images, a clinical reader study compared the proposed approach and RAKI in a forced choice scenario and using a five-point ordinal scoring, assessing sharpness, SNR, aliasing and overall image quality. Additionally, the effect of varying parameters such as the number of autocalibration signals, the contribution of the autoencoding loss, the kernel size, and the latent space dimension for the proposed technique was studied. Results: The proposed approach showed improved quantitative metrics compared to GRAPPA and RAKI. Visually, the method showed higher resilience to noise amplification and improved preservation of sharp details. In the clinical reader study, the proposed approach was preferred overall to RAKI and yielded higher scores than GRAPPA and RAKI. The proposed method showed robustness to parameter variations. Conclusion: Deep Learning based k-space interpolation can be achieved using a Koopman autoencoder for k-space neighbourhoods. Keywords: Parallel imaging, Koopman autoencoder, nonlinear mapping, k-space interpolation, deep learning reconstruction

physics.med-ph↗

High-Precision RCM Motion for Ophthalmic Robotics via Hierarchical QP Control and Authority-Guided Recovery Planning

General-purpose manipulators can position ophthalmic instruments under a software Remote Centre of Motion (RCM). However, at some reachable configurations that are nonsingular for the tip task alone, the RCM and tip-tracking objectives conflict and the servo oscillates. We formalise this failure as a constraint-induced task singularity: the RCM-projected tip Jacobian loses rank while the unprojected tip Jacobian stays well conditioned. Thus, the effective tip authority collapses although the target remains reachable. We present a high-precision RCM servo architecture with built-in singularity recovery: for each commanded target, an authority-guided planner selects a well-conditioned arm configuration that reaches it and plans an RCM-consistent route, executed by a fixed hierarchical-QP controller. On a 200-scenario kinematic benchmark, the complete nominal-and-recovery protocol reaches 128/128 near-singular targets and 71/72 near-joint-limit targets (100% and 99%, respectively), with the software-RCM residual below 0.142 mm. On a physical 6-DOF arm it executes across four RCM entry points, with a worst-case kinematically reconstructed RCM residual of 178 μm and a median tip-regulation error of 32μm, within the 200μm control requirement adopted for the ophthalmic task.

cs.RO↗

MID: Mask-Image Distributional Divergence for Evaluating Medical Image Segmentation

Current evaluation protocols for medical image segmentation remain largely anchored in sample-wise comparisons, e.g., Dice and Hausdorff distance, which require each prediction to be paired with a ground-truth mask. Yet two limitations arise in practice: they are inapplicable when predictions are made on unlabeled data; and even when annotations are available, they cannot assess whether a collection of predictions matches the distributional structure of the ground-truth population. We introduce MID (Mask-Image Distributional Divergence), a distribution-level evaluation metric for medical image segmentation. MID builds upon the observation that segmentation quality is a property of the joint image-mask distribution: a mask may be anatomically plausible in isolation yet incorrect for the image it accompanies. MID compares the joint distribution of images and masks between a reference set and a set of predictions, without requiring sample-wise correspondence. This captures both mask realism and image-mask consistency. We evaluate MID across 13 publicly available datasets spanning CT, MRI, and MRA. Under anatomically realistic corruptions across 56 organ groups, MID tracks mean Dice and Hausdorff distance without per-sample ground-truth pairing. MID also detects distributional shifts across modalities and anatomies, identifies image-mask misalignment through controlled shuffling experiments, and produces reliable model rankings across real segmentation models. Official implementation available at github.com/erc-caravel/pytorch-mid

cs.CV↗

Photoacoustic-guided reflection-mode wavefront shaping through dynamic biological tissue

Strong optical scattering limits the depth at which light can be focused and used for imaging in biological tissue. Photoacoustic (PA)-guided wavefront shaping enables non-invasive light focusing in scattering media, but most existing implementations require optical and acoustic access from opposite sides of the sample and a rigidly aligned free-space optical path, which have confined these demonstrations to thin or static samples. Here, we combine a real-valued intensity transmission matrix (RVITM) algorithm with a photoacoustic guidestar (PA-RVITM) in a reflection-mode configuration. Modulated light is delivered through a flexible multimode fiber passing through the central aperture of a focused ultrasound transducer, enabling optical illumination and acoustic detection from the same side of the sample without a free-space combiner. Fiber integration preserved the wavefront shaping performance, and the reflection-mode PA-RVITM system achieved sub-second wavefront optimization with a ~14-fold PA signal enhancement through a diffuser. In ex vivo chicken breast tissue, the enhancement decreased with tissue thickness, reaching ~3.5 through a 2-mm chicken breast layer, with a speckle decorrelation time of ~450 ms. By performing an independent RVITM optimization at each scan position, our system enabled photoacoustic microscopy of a black nylon wire through the same layer, reducing the imaged line width from ~141 μm to ~102 μm, close to the nominal diameter of ~100 μm. This work advances PA-guided wavefront shaping toward a compact, same-side implementation for future biomedical therapeutic and imaging applications.

physics.optics↗

Making Cross-Continental Federated Learning Repeatable with FLIP: a Multi-Application Study

Federated learning (FL) in healthcare remains challenging, as the overhead of rebuilding governance guarantees for every collaboration stops most projects at the proof-of-concept stage. Here we present FLIP (Federated Learning Interoperability Platform), an open-source, multi-application platform that makes FL training and evaluation repeatable. FLIP implements common FL workflows as a set of composable services: cohort queries against per-site structured databases, on-demand DICOM retrieval from institutional PACS, per-site project approval, and reusable FL job types. To demonstrate FLIP, we ran two distinct use cases, federated fine-tuning and federated evaluation, on synthetic chest X-ray cohorts across two client nodes based in the United Kingdom (UK) and Thailand. In FLIP, each institution independently approves its participation in each project and operates its own node under local IT security processes. This study makes an operational rather than an algorithmic claim. It does not compare federated with centralised training; for that question, we refer the reader to existing systematic reviews and meta-analyses. The central result is evidence that such platforms enable international FL collaboration and improve repeatability, auditability, and site-specific governance. We also present a comprehensive comparison of existing platforms to help researchers and operators choose the right platform for their use case.

cs.LG↗

Metadata Supervised Imaging Representations for Modelling and Controlling Acquisition Variability

Biomedical imaging data exhibit substantial acquisition variability, where identical biological structures can appear markedly different due to differences in imaging devices, acquisition protocols, sites, and reconstruction settings. Consequently, learned representations often entangle underlying biological information with acquisition-dependent appearance, limiting interpretability, generalisation, and clinical deployment. We show that these sources of variation can be disentangled by jointly modelling medical images and acquisition metadata. Using large-scale clinical brain MRI data as a case study, we learn representations that disentangle anatomical structure from contrast-dependent appearance. The resulting framework enables the organisation of heterogeneous imaging protocols, sequence understanding, the detection of image-metadata inconsistencies and imaging artifacts, while preserving biologically relevant anatomical features across diverse acquisitions. Building on these disentangled representations, it further supports generative and translational capabilities, performing both metadata-conditioned synthesis of realistic 3D brain MRIs and anatomy-preserving harmonisation for cross-modality and cross-site adaptation. Our findings demonstrate that acquisition variability is a structured component of the imaging process that can be modeled, audited, synthesised, and controlled, establishing a foundation for acquisition-aware representation learning in large-scale biomedical imaging.

cs.CV↗

WeakMCN: Multi-task Collaborative Network for Weakly Supervised Referring Expression Comprehension and Segmentation

Weakly supervised referring expression comprehension(WREC) and segmentation(WRES) aim to learn object grounding based on a given expression using weak supervision signals like image-text pairs. While these tasks have traditionally been modeled separately, we argue that they can benefit from joint learning in a multi-task framework. To this end, we propose WeakMCN, a novel multi-task collaborative network that effectively combines WREC and WRES with a dual-branch architecture. Specifically, the WREC branch is formulated as anchor-based contrastive learning, which also acts as a teacher to supervise the WRES branch. In WeakMCN, we propose two innovative designs to facilitate multi-task collaboration, namely Dynamic Visual Feature Enhancement(DVFE) and Collaborative Consistency Module(CCM). DVFE dynamically combines various pre-trained visual knowledge to meet different task requirements, while CCM promotes cross-task consistency from the perspective of optimization. Extensive experimental results on three popular REC and RES benchmarks, i.e., RefCOCO, RefCOCO+, and RefCOCOg, consistently demonstrate performance gains of WeakMCN over state-of-the-art single-task alternatives, e.g., up to 3.91% and 13.11% on RefCOCO for WREC and WRES tasks, respectively. Furthermore, experiments also validate the strong generalization ability of WeakMCN in both semi-supervised REC and RES settings against existing methods, e.g., +8.94% for semi-REC and +7.71% for semi-RES on 1% RefCOCO. The code is publicly available at https://github.com/MRUIL/WeakMCN.

cs.CV↗

A cautionary tale on the cost-effectiveness of collaborative AI in real-world medical applications

Federated learning (FL) has gained wide popularity as a collaborative learning paradigm allowing collaborative Artificial Intelligence (AI) in sensitive healthcare applications. Nevertheless, the practical implementation of FL presents technical and organizational challenges, as it generally requires complex communication infrastructures. In this context, consensus-based learning (CBL) may represent a promising alternative for collaborative learning, allowing the combination of local knowledge into a federated decision system, while potentially reducing deployment overhead. Nevertheless, a comprehensive assessment of the viability of consensus-based learning as a cost-effective and sustainable alternative to federated learning has not yet been conducted. In this work we propose an extensive benchmark of the accuracy, cost-effectiveness, and sustainability of a panel of FL and CBL methods in a wide range of collaborative medical data analysis scenarios. The benchmark includes 7 different medical datasets, encompassing 3 machine learning tasks, 8 different data modalities, and multi-centric settings involving 3 to 23 clients. Our results reveal that CBL is a cost-effective and more sustainable alternative to FL. When compared across the panel of medical datasets in the considered benchmark, CBL methods provide equivalent accuracy to the one achieved by FL. Nonetheless, CBL significantly reduces training time and communication cost (resp. 15 fold and 131 fold decrease) (p-value < 0.05). This study opens a novel perspective on the deployment of collaborative AI in real-world applications, whereas the adoption of cost-effective methods is instrumental to achieve sustainability and democratization of AI by alleviating the need for extensive computational resources.

cs.LG↗

MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal Prostate MRI Segmentation

Active Surveillance (AS) is a treatment option for managing low and intermediate-risk prostate cancer (PCa), aiming to avoid overtreatment while monitoring disease progression through serial MRI and clinical follow-up. Accurate prostate segmentation is an important preliminary step for automating this process, enabling automated detection and diagnosis of PCa. However, existing deep-learning segmentation models are often trained on single-time-point, expertly annotated datasets, making them unsuitable for longitudinal AS analysis, where multiple time points and a scarcity of expert labels hinder effective fine-tuning. To address these challenges, we propose MambaX-Net, a novel semi-supervised, dual-scan 3D segmentation architecture that computes the segmentation for time point t by leveraging the MRI and the corresponding segmentation mask from the previous time point. We introduce two new components: (i) a Mamba-enhanced Cross-Attention Module, which integrates the Mamba block into cross-attention to efficiently capture temporal evolution and long-range spatial dependencies, and (ii) a Shape Extractor Module that encodes the previous segmentation mask into a latent anatomical representation for refined zone delineation. Moreover, we use a semi-supervised self-training strategy that leverages pseudo-labels generated from a pre-trained nnU-Net, enabling effective learning without expert annotations. MambaX-Net was evaluated on a longitudinal AS dataset, and results showed that it significantly outperforms state-of-the-art U-Net and Transformer-based models, achieving superior prostate zone segmentation even when trained on limited and noisy data.

cs.CV↗

Progressive Experience Fusion for Multi-Task World Model Control in Endovascular Navigation

Autonomous endovascular navigation could support the delivery of mechanical thrombectomy to underserved areas, but controllers must navigate long, multi-stage paths across varying vascular anatomies. This study investigates Progressive Experience Fusion (PEF) to train a multi-task TD-MPC2 controller. We additionally evaluate a heuristic that changes the Model Predictive Path Integral planning horizon using residual action-sequence dispersion, and fine-tuning in a patient-specific simulation. Across five subtasks in ten known training anatomies with held-out targets, PEF achieved a mean success rate of 74%, compared with 37% for Soft Actor-Critic (p < 0.001) and 65% for base TD-MPC2 (p = 0.053). A PEF controller with adaptive-horizon planning trained on 30 vasculatures achieved a mean success rate of 90% in ten held-out vasculatures. The PEF agent successfully transferred to an unseen in vitro stroke patient vasculature under fluoroscopy, achieving a mean path ratio improvement from 63% to 80% with fine-tuning (p < 0.001), following 40x103 fine-tuning steps (corresponding to approximately 107 min of clinical inter-hospital transfer time). This work represents a proof of concept for multi-vasculature training and patient-specific adaptation, while further validation is required before clinical deployment.

cs.RO↗

Eversion-based robots can enable safe access,steering and endoscopic imaging within the spinal subarachnoid space

Safe navigation within the spinal subarachnoid space is constrained by its narrow, compliant, and delicate anatomy. Conventional catheters and continuum robots rely on proximal pushing, generating friction and shear along the tissue device interface that limit distal controllability and increase the risk of neural injury. Here, we present a 2 mm diameter eversion-growing robotic platform that enables friction minimised extension and steering within the human spinal subarachnoid space, validated through computational modelling, phantom experiments, and intact human cadaver studies. The robot integrates a miniature endoscope for real time intrathecal visualisation and advances by pressure driven tip eversion, localising motion to the distal tip while minimising translational sliding of the deployed body. Phantom experiments demonstrated reductions of 65.2% in mean interaction force and 48.0% in peak interaction force compared with matched push-based insertion. Physics based modelling showed that eversion based growth redistributed tissue loading, reducing local stress concentrations and interfacial shear relative to conventional insertion. In an intact human cadaver, the system achieved 150 mm of controlled intrathecal extension with concurrent fluoroscopic and endoscopic visualisation, providing access across multiple vertebral levels from a standard lumbar entry point. Postprocedural laminectomy and durotomy revealed no observable macroscopic disruption of the dura mater or surrounding neural structures. These results provide the first mechanically characterised and multimodally validated demonstration of eversion-based robotic navigation in intact human spinal anatomy, establishing a quantitative and procedural foundation for future intrathecal interventions. Further validation in larger anatomical cohorts and under physiological conditions will be required before clinical translation.

cs.RO↗

Stabilizing Temporal Inference Dynamics for Online Surgical Phase Recognition

Online Surgical Phase Recognition (SPR) models can reach high frame-wise accuracy, yet their predictions often lack temporal stability, fragmenting workflow understanding and reducing the reliability of downstream assistance. We show that this instability is not random noise but arises from two mechanisms: early misclassifications corrupt temporal feature states and propagate forward to form error cascades, and phase transitions follow evidence-accumulation dynamics whereas most online SPR systems rely on memoryless frame-wise decisions, making them sensitive to transient confidence fluctuations. We propose a unified Train-Inference-Evaluation framework that explicitly stabilizes temporal inference dynamics using model-agnostic, plug-and-play components. For training, the Temporal Error-Cascade (TEC) loss suppresses error onset and mitigates forward error propagation by stabilizing temporal feature evolution. For inference, the Evidence-Gated Transition Predictor (EGTP) enforces evidence-driven state transitions, allowing phase changes only when accumulated evidence exceeds a confidence boundary. For evaluation, we introduce the Temporal Fragmentation Index (TFI), a reliability-aware metric that quantifies instability-induced temporal disagreement beyond conventional frame-wise and token-based measures. Experiments on Cholec80 and AutoLaparo across three representative backbones show that the proposed framework substantially improves temporal stability and reduces prediction fragmentation, while maintaining or modestly improving frame-wise performance.

cs.CV↗

Real-time 3D Ultrasonic Needle Tracking with a Photoacoustic Beacon

Many minimally invasive procedures, such as core needle biopsy of focal liver lesions, nerve blocks, and fetal and vascular interventions, are typically performed under ultrasound guidance, which provides real-time, high-resolution visualisation of tissue anatomy. Accurate and efficient localisation of the needle tip relative to patient anatomy is essential for guiding the needle towards the procedure target, avoiding adverse events and reducing the need for repeat procedures. However, the 3D nature of the procedure and poor image contrast of the needle in heterogeneous tissue or at steep insertion angles often lead to confusion over the true location of the tip within the 2D guidance images, and existing methods to enhance needle visibility largely remain limited to 2D. Here, we present a novel interventional ultrasound system capable of 2D B-mode imaging and 3D needle tracking. The tip location is determined from the time-of-flight of ultrasound generated by a photoacoustic beacon embedded in the needle bevel and received by a sparse receiver array distributed around the imaging system's curvilinear ultrasound probe. The measured tracking accuracy was better than 2 mm for depths up to 140 mm in water, and approximately 2 mm on average in an ex vivo tissue phantom, with referenced positions derived from X-ray computed tomography. In a usability study involving 12 clinicians performing biopsy procedures in a ex vivo tissue phantom, the failure rate was reduced by 35%, from 15.8% to 10.3% after only a few minutes of training. These results demonstrate that the proposed system has strong potential to support a wide range of minimally invasive procedures by enabling clinicians to accurately target small anatomical structures, improving the efficiency and effectiveness of diagnostic sampling and therapeutic delivery or ablation, and reducing the risk of adverse events.

physics.med-ph↗

Three-dimensional intraoperative light field fluorescence imaging system for PpIX-guided tumour resection

Conventional 2D fluorescence imaging in glioma surgery cannot separate intrinsic fluorophore strength from attenuation with depth, creating depth-intensity ambiguity that can compromise assessment of residual tumour and fluorescence based grading. This study develops and validates a dual mode light field imaging system that could capture 3D structure and depth corrected fluorescence in a single snapshot by adapting a commercial Lytro Illum camera. A custom 3D printed depth standard was used to optimise main lens focal length and to derive a grayscale - distance linearity from Lytro Desktop depth maps. CdSe/ZnS quantum dot targets and fluorescent brain phantoms were imaged to establish fluorescence intensity distance attenuation models and to recover intrinsic fluorescence. In system optimisation, the increasing FU strengthened grayscale depth linearity and achieved millimetre scale vertical resolution ($R^{2}$ > 0.95) for FU $\ge$ 60 mm. Higher concentration quantum dot wells of the fluorescent target showed consistent attenuation. In fluorescence mode, the deviations of distance estimations across six regions of a fluorescent brain phantom were 0.14 to 2.45% with intensity prediction errors from -11.73% to 6.08% based on the fluorescence intensity-distance model, enabling recovery of intrinsic quantum dot concentrations which are mimicking PpIX characteristics in glioma. This research supports light field imaging as a practical approach for depth resolved quantitative fluorescence and improved intraoperative tumour characterisation.

physics.med-ph↗

CLoPA: Continual Low Parameter Adaptation of Interactive Segmentation for Medical Image Annotation

Interactive segmentation enables clinicians to guide annotation, but existing zero-shot models like nnInteractive fail to consistently reach expert-level performance across diverse medical imaging tasks. Because annotation campaigns produce a growing stream of task-specific labelled data, online adaptation of the segmentation model is a natural complement to zero-shot inference. We propose CLoPA, a continual adaptation strategy that tunes a small fraction of nnInteractive's parameters on the annotation cache, triggered by lightweight episode scheduling. CLoPA requires no new parameters or changes to the inference pipeline, and operates entirely within the existing annotation workflow. Across eight Medical Segmentation Decathlon tasks spanning diverse anatomical targets and imaging characteristics, CLoPA rapidly elevates performance to expert-level, even for tasks where nnInteractive previously failed, with the majority of gains realised after a single training episode. We show that the benefits of tuning different parameter groups depends on task characteristics and data regimes. Also, that for targets with complex geometries (e.g., hepatic vessels), instance normalisation and low-level feature tuning saturates, suggesting a need for deeper feature-representation alignment in the most challenging scenarios.

cs.CV↗

Deep generative computed perfusion-deficit mapping of ischaemic stroke

Focal deficits in ischaemic stroke result from impaired perfusion downstream of a critical vascular occlusion. While parenchymal lesions are traditionally used to predict clinical deficits, the underlying pattern of disrupted perfusion provides information upstream of the lesion, potentially yielding earlier predictive and localizing signals. Such perfusion maps can be derived from routine CT angiography (CTA) widely deployed in clinical practice. Analysing computed perfusion maps from 1,393 CTA-imaged-patients with acute ischaemic stroke, we use deep generative inference to localise neural substrates of NIHSS sub-scores. We show that our approach replicates known lesion-deficit relations without knowledge of the lesion itself and reveals novel neural dependents. The high achieved anatomical fidelity suggests acute CTA-derived computed perfusion maps may be of substantial clinical-and-scientific value in rich phenotyping of acute stroke. Using only hyperacute imaging, deep generative inference could power highly expressive models of functional anatomical relations in ischaemic stroke within the pre-interventional window.

q-bio.QM↗

From Pre- to Intra-operative MRI: Predicting Brain Shift in Temporal Lobe Resection for Epilepsy Surgery

Introduction: In neurosurgery, image-guided Neurosurgery Systems (IGNS) highly rely on preoperative brain magnetic resonance images (MRI) to assist surgeons in locating surgical targets and determining surgical paths. However, brain shift invalidates the preoperative MRI after dural opening. Updated intraoperative brain MRI with brain shift compensation is crucial for enhancing the precision of neuronavigation systems and ensuring the optimal outcome of surgical interventions. Methodology: We propose NeuralShift, a U-Net-based model that predicts brain shift entirely from pre-operative MRI for patients undergoing temporal lobe resection. We evaluated our results using Target Registration Errors (TREs) computed on anatomical landmarks located on the resection side and along the midline, and DICE scores comparing predicted intraoperative masks with masks derived from intraoperative MRI. Results: Our experimental results show that our model can predict the global deformation of the brain (DICE of 0.97) with accurate local displacements (achieve landmark TRE as low as 1.12 mm), compensating for large brain shifts during temporal lobe removal neurosurgery. Conclusion: Our proposed model is capable of predicting the global deformation of the brain during temporal lobe resection using only preoperative images, providing potential opportunities to the surgical team to increase safety and efficiency of neurosurgery and better outcomes to patients. Our contributions will be publicly available after acceptance in https://github.com/SurgicalDataScienceKCL/NeuralShift.

cs.CV↗