arXiv subjects

William Freeman

Publications and source records attributed to William Freeman.

9 recordsLinked to original sources

arXiv2026

Modality-Invariant Coarse-to-Fine Retinal Image Registration

Retinal image registration is essential for ophthalmic diagnosis, longitudinal disease monitoring, and multimodal retinal image analysis. Existing retinal registration methods are typically modality-dependent: they are designed or optimized either for a single imaging modality in mono-modal registration or for a fixed pair of modalities in cross-modal registration. This limits their flexibility and applicability in practical scenarios involving diverse retinal imaging modalities and different combinations of them. In this work, we propose a generalizable two-stage, modality-invariant framework for retinal image registration. First, we introduce a sparse feature-matching model driven by a universal retinal vessel segmentation to achieve robust coarse global alignment across modalities. Second, we develop a modality-invariant optical flow estimation network, termed MI-RAFT, to refine the alignment through dense local registration. Extensive experiments demonstrate that the proposed method can handle diverse combinations of commonly used retinal imaging modalities, exhibiting strong modality invariance while outperforming state-of-the-art modality-dependent registration methods.

eess.IV↗

arXiv2025

Beyond I-Con: Exploring New Dimension of Distance Measures in Representation Learning

The Information Contrastive (I-Con) framework revealed that over 23 representation learning methods implicitly minimize KL divergence between data and learned distributions that encode similarities between data points. However, a KL-based loss may be misaligned with the true objective, and properties of KL divergence such as asymmetry and unboundedness may create optimization challenges. We present Beyond I-Con, a framework that enables systematic discovery of novel loss functions by exploring alternative statistical divergences. Key findings: (1) on unsupervised clustering of DINO-ViT embeddings, we achieve state-of-the-art results by modifying the PMI algorithm to use total variation (TV) distance; (2) supervised contrastive learning with Euclidean distance as the feature space metric is improved by replacing the standard loss function with Jenson-Shannon divergence (JSD); (3) on dimensionality reduction, we achieve superior qualitative results and better performance on downstream tasks than SNE by replacing KL with a bounded $f$-divergence. Our results highlight the importance of considering divergence choices in representation learning optimization.

cs.LG↗

arXiv2025

Tunable Wavelet Unit based Convolutional Neural Network in Optical Coherence Tomography Analysis Enhancement for Classifying Type of Epiretinal Membrane Surgery

In this study, we developed deep learning-based method to classify the type of surgery performed for epiretinal membrane (ERM) removal, either internal limiting membrane (ILM) removal or ERM-alone removal. Our model, based on the ResNet18 convolutional neural network (CNN) architecture, utilizes postoperative optical coherence tomography (OCT) center scans as inputs. We evaluated the model using both original scans and scans preprocessed with energy crop and wavelet denoising, achieving 72% accuracy on preprocessed inputs, outperforming the 66% accuracy achieved on original scans. To further improve accuracy, we integrated tunable wavelet units with two key adaptations: Orthogonal Lattice-based Wavelet Units (OrthLatt-UwU) and Perfect Reconstruction Relaxation-based Wavelet Units (PR-Relax-UwU). These units allowed the model to automatically adjust filter coefficients during training and were incorporated into downsampling, stride-two convolution, and pooling layers, enhancing its ability to distinguish between ERM-ILM removal and ERM-alone removal, with OrthLattUwU boosting accuracy to 76% and PR-Relax-UwU increasing performance to 78%. Performance comparisons showed that our AI model outperformed a trained human grader, who achieved only 50% accuracy in classifying the removal surgery types from postoperative OCT scans. These findings highlight the potential of CNN based models to improve clinical decision-making by providing more accurate and reliable classifications. To the best of our knowledge, this is the first work to employ tunable wavelets for classifying different types of ERM removal surgery.

eess.IV↗

arXiv2025

Graphics4Science: Computer Graphics for Scientific Impacts

Computer graphics, often associated with films, games, and visual effects, has long been a powerful tool for addressing scientific challenges--from its origins in 3D visualization for medical imaging to its role in modern computational modeling and simulation. This course explores the deep and evolving relationship between computer graphics and science, highlighting past achievements, ongoing contributions, and open questions that remain. We show how core methods, such as geometric reasoning and physical modeling, provide inductive biases that help address challenges in both fields, especially in data-scarce settings. To that end, we aim to reframe graphics as a modeling language for science by bridging vocabulary gaps between the two communities. Designed for both newcomers and experts, Graphics4Science invites the graphics community to engage with science, tackle high-impact problems where graphics expertise can make a difference, and contribute to the future of scientific discovery. Additional details are available on the course website: https://graphics4science.github.io

cs.GR↗

arXiv2025

Universal Vessel Segmentation for Multi-Modality Retinal Images

We identify two major limitations in the existing studies on retinal vessel segmentation: (1) Most existing works are restricted to one modality, i.e., the Color Fundus (CF). However, multi-modality retinal images are used every day in the study of the retina and diagnosis of retinal diseases, and the study of vessel segmentation on other modalities is scarce; (2) Even though a few works extended their experiments to new modalities such as the Multi-Color Scanning Laser Ophthalmoscopy (MC), these works still require fine-tuning a separate model for the new modality. The fine-tuning will require extra training data, which is difficult to acquire. In this work, we present a novel universal vessel segmentation model (URVSM) for multi-modality retinal images. In addition to performing the study on a much wider range of image modalities, we also propose a universal model to segment the vessels in all these commonly used modalities. While being much more versatile compared with existing methods, our universal model also demonstrates comparable performance to the state-of-the-art fine-tuned methods. To the best of our knowledge, this is the first work that achieves modality-agnostic retinal vessel segmentation and the first to study retinal vessel segmentation in several novel modalities.

eess.IV↗

arXiv2020

The MOSDEF Survey: An Improved Voronoi Binning Technique on Spatially Resolved Stellar Populations at z~2

We use a sample of 350 star-forming galaxies at $1.25<z<2.66$ from the MOSFIRE Deep Evolution Field survey to demonstrate an improved Voronoi binning technique that we use to study the properties of resolved stellar populations in $z\sim2$ galaxies. Stellar population and dust maps are constructed from the high-resolution CANDELS/3D-HST multi-band imaging. Rather than constructing the layout of resolved elements (i.e., Voronoi bins) from the S/N distribution of the $H_{160}$-band alone, we introduce a modified Voronoi binning method that additionally incorporates the S/N distribution of several resolved filters. The SED-derived resolved E(B-V)$_{\text{stars}}$, stellar population ages, SFRs, and stellar masses that are inferred from the Voronoi bins constructed from multiple filters are generally consistent with the properties inferred from the integrated photometry within the uncertainties, with the exception of the inferred E(B-V)$_{\text{stars}}$ from our $z\sim1.5$ sample due to their UV slopes being unconstrained by the resolved photometry. The results from our multi-filter Voronoi binning technique are compared to those derived from a "traditional" single-filter Voronoi binning approach. We find that single-filter binning produces inferred E(B-V)$_{\text{stars}}$ that are systematically redder by 0.02 mag on average, but could differ by up to 0.20 mag, and could be attributed to poorly constrained resolved photometry covering the UV slope. Overall, we advocate that our methodology produces more reliable SED-derived parameters due to the best-fit resolved SEDs being better constrained at all resolved wavelengths--particularly those covering the UV slope.

astro-ph.GA↗

arXiv2017

The MOSDEF survey: the prevalence and properties of galaxy-wide AGN-driven outflows at $z\sim 2$

Using observations from the first two years of the MOSFIRE Deep Evolution Field (MOSDEF) survey, we study 13 AGN-driven outflows detected from a sample of 67 X-ray, IR and/or optically-selected AGN at $z \sim 2$. The AGN have bolometric luminosities of $\sim10^{44}-10^{46} ~\mathrm{erg~s^{-1}}$, including both quasars and moderate-luminosity AGN. We detect blueshifted, ionized gas outflows in the H$\beta$ , [OIII], H$\alpha$ ~and/or [NII] emission lines of $19\%$ of the AGN, while only 1.8\% of the MOSDEF galaxies have similarly-detected outflows. The outflow velocities span $\sim$300 to 1000 km s$^{-1}$. Eight of the 13 outflows are spatially extended on similar scales as the host galaxies, with spatial extents of 2.5 to 11.0 kpc. Outflows are detected uniformly across the star-forming main sequence, showing little trend with the host galaxy SFR. Line ratio diagnostics indicate that the outflowing gas is photoionized by the AGN. We do not find evidence for positive AGN feedback, in either our small MOSDEF sample or a much larger SDSS sample, using the BPT diagram. Given that a galaxy with an AGN is ten times more likely to have a detected outflow, the outflowing gas is photoionzed by the AGN, and estimates of the mass and energy outflow rates indicate that stellar feedback is insufficient to drive at least some of these outflows, they are very likely to be AGN-driven. The outflows have mass-loading factors of the order of unity, suggesting that they help regulate star formation in their host galaxies, though they may be insufficient to fully quench it.

astro-ph.GA↗

arXiv2016

The MOSDEF Survey: Metallicity Dependence of the PAH Emission at High Redshift and Implications for 24 micron-inferred IR Luminosities and Star Formation Rates at z~2

We present results on the variation of 7.7 micron Polycyclic Aromatic Hydrocarbon (PAH) emission in galaxies spanning a wide range in metallicity at z ~ 2. For this analysis, we use rest-frame optical spectra of 476 galaxies at 1.37 < z < 2.61 from the MOSFIRE Deep Evolution Field (MOSDEF) survey to infer metallicities and ionization states. Spitzer/MIPS 24 micron and Herschel/PACS 100 and 160 micron observations are used to derive rest-frame 7.7 micron luminosities (L(7.7)) and total IR luminosities (L(IR)), respectively. We find significant trends between the ratio of L(7.7) to L(IR) (and to dust-corrected SFR) and both metallicity and [OIII]/[OII] (O32) emission-line ratio. The latter is an empirical proxy for the ionization parameter. These trends indicate a paucity of PAH emission in low metallicity environments with harder and more intense radiation fields. Additionally, L(7.7)/L(IR) is significantly lower in the youngest quartile of our sample (ages of 500 Myr) compared to older galaxies, which may be a result of the delayed production of PAHs by AGB stars. The relative strength of L(7.7) to L(IR) is also lower by a factor of ~ 2 for galaxies with masses $M_* < 10^{10}M_{\odot}$, compared to the more massive ones. We demonstrate that commonly-used conversions of L(7.7) (or 24 micron flux density; f(24)) to L(IR) underestimate the IR luminosity by more than a factor of 2 at $M_*$ ~ $10^{9.6-10.0} M_{\odot}$. We adopt a mass-dependent conversion of L(7.7) to L(IR) with L(7.7)/L(IR)= 0.09 and 0.22 for $M_* < 10^{10}$ and $> 10^{10} M_{\odot}$, respectively. Based on the new scaling, the SFR-$M_*$ relation has a shallower slope than previously derived. Our results also suggest a higher IR luminosity density at z ~ 2 than previously measured, corresponding to a ~ 30% increase in the SFR density.

astro-ph.GA↗

arXiv2015

Understanding Intra-Class Knowledge Inside CNN

Convolutional Neural Network (CNN) has been successful in image recognition tasks, and recent works shed lights on how CNN separates different classes with the learned inter-class knowledge through visualization. In this work, we instead visualize the intra-class knowledge inside CNN to better understand how an object class is represented in the fully-connected layers. To invert the intra-class knowledge into more interpretable images, we propose a non-parametric patch prior upon previous CNN visualization models. With it, we show how different "styles" of templates for an object class are organized by CNN in terms of location and content, and represented in a hierarchical and ensemble way. Moreover, such intra-class knowledge can be used in many interesting applications, e.g. style-based image retrieval and style-based object completion.

cs.CV↗