arXiv subjects

Ayodele Ore

Publications and source records attributed to Ayodele Ore.

13 recordsLinked to original sources

arXiv2026

Cross-simulator transfer with foundation model summaries: Towards robust SKA-era reionization inference

Simulation-based inference (SBI) for parameter estimation is vulnerable to model misspecification: neural summaries and density estimators trained on a specific forward model typically fail when applied to data drawn from another model, or from real observations, and no training simulator can capture the full observational pipeline of a real measurement exactly. We show that a self-supervised Vision Transformer (ViT), pretrained label-free on a fast approximate simulator, produces transferable data summaries that generalize across simulators. Without retraining, it can be reused as a frozen encoder to infer astrophysical parameters from a completely different simulator that resolves the radiative transfer explicitly, on which it has never seen either data or parameters. As a concrete use case in 21cm cosmology, SKATR, a ViT pretrained with a Joint Embedding Predictive Architecture (JEPA), serves as a foundation model for reionization inference from upcoming SKA measurements: SKATR is pretrained once on 67k low-cost, noiseless semi-numerical 21cmFAST lightcones, then frozen and applied to hydrodynamical Loreli II lightcones, where a lightweight conditional flow matching head infers five astrophysical parameters; the encoder is never shown Loreli data, its parameters, or any noise. In our comparison, SKATR yields the most precise and best-calibrated posteriors across all five parameters, matching the accuracy of the fully-supervised in-domain baseline while requiring 2.6x fewer radiative-transfer simulations. Under realistic SKA AA* noise, only SKATR remains simultaneously accurate, informative, and calibrated, outperforming even a supervised baseline retrained from scratch on noisy data. Self-supervised pretraining on computationally efficient semi-numerical simulations is therefore a viable route to calibrated, simulator- and noise-agnostic reionization inference for the SKA-era.

astro-ph.CO↗

arXiv2026

Unfolding without Iterations, Adversaries, or Surrogates

Correcting measurements for detector effects and constructing appropriate public data representations is a pressing problem in LHC physics. Current methods solve this inverse problem by relying on iterations, minimax optimization, or a surrogate forward mapping. We introduce Adversary-free Unfolding SanS Iteration or Emulation (AUSSIE), which dispenses with these mechanisms while remaining asymptotically correct. AUSSIE replaces the second OmniFold step with a new loss function that directly yields solutions with minimal dependence on the reference simulation. We showcase AUSSIE on various unfolding tasks, including full-phase-space jet substructure.

hep-ph↗

arXiv2025

Iterative HOMER with uncertainties

We present iHOMER, an iterative version of the HOMER method to extract Lund fragmentation functions from experimental data. Through iterations, we address the information gap between latent and observable phase spaces and systematically remove bias. To quantify uncertainties on the inferred weights, we use a combination of Bayesian neural networks and uncertainty-aware regression. We find that the combination of iterations and uncertainty quantification produces well-calibrated weights that accurately reproduce the data distribution. A parametric closure test shows that the iteratively learned fragmentation function is compatible with the true fragmentation function.

hep-ph↗

arXiv2025

Direct reconstruction of the Reionization history from 21cm 2D Power Spectra

The 21cm line from the spin-flip transition of neutral hydrogen (HI) provides a unique window into the Epoch of Reionization (EoR), the final phase transition of our Universe. The Square Kilometre Array (SKA) enables precise measurements of 21cm fluctuations that trace ionization, temperature, and density fluctuations of the intergalactic medium (IGM). Nevertheless, a direct reconstruction of the timeline of the EoR in terms of the progress of ionization remains an ongoing challenge due to the highly non-Gaussian nature and thus intractable likelihood of the 21cm signal. Here, we present EoRFlow, a simulation-based inference (SBI) framework for reconstructing the global neutral hydrogen fraction $x_{\mathrm{HI}}(z)$ directly from 2D cylindrically averaged power spectra (2DPS) of the 21cm signal. We validate our method on realistic mock datasets for SKA-Low. Bypassing the need for explicit likelihood formulations, our approach enables fast, unbiased posterior estimation of the $x_{\mathrm{HI}}$ evolution in narrow redshift slices, allowing for piecewise reconstruction of the global reionization history. By directly inferring the reionization history from 21cm power spectra, our framework provides a scalable and robust path forward for 21cm cosmology in the SKA era.

astro-ph.CO↗

arXiv2025

Large Language Models -- the Future of Fundamental Physics?

For many fundamental physics applications, transformers, as the state of the art in learning complex correlations, benefit from pretraining on quasi-out-of-domain data. The obvious question is whether we can exploit Large Language Models, requiring proper out-of-domain transfer learning. We show how the Qwen2.5 LLM can be used to analyze and generate SKA data, specifically 3D maps of the cosmological large-scale structure for a large part of the observable Universe. We combine the LLM with connector networks and show, for cosmological parameter regression and lightcone generation, that this Lightcone LLM (L3M) with Qwen2.5 weights outperforms standard initialization and compares favorably with dedicated networks of matching size.

astro-ph.CO↗

arXiv2024

Extrapolating Jet Radiation with Autoregressive Transformers

Generative networks are an exciting tool for fast LHC event fixed number of particles. Autoregressive transformers allow us to generate events containing variable numbers of particles, very much in line with the physics of QCD jet radiation, and offer the possibility to generalize to higher multiplicities. We show how transformers can learn a factorized likelihood for jet radiation and extrapolate in terms of the number of generated jets. For this extrapolation, bootstrapping training data and training with modifications of the likelihood loss can be used.

hep-ph↗

arXiv2024

CaloChallenge 2022: A Community Challenge for Fast Calorimeter Simulation

We present the results of the "Fast Calorimeter Simulation Challenge 2022" - the CaloChallenge. We study state-of-the-art generative models on four calorimeter shower datasets of increasing dimensionality, ranging from a few hundred voxels to a few tens of thousand voxels. The 31 individual submissions span a wide range of current popular generative architectures, including Variational AutoEncoders (VAEs), Generative Adversarial Networks (GANs), Normalizing Flows, Diffusion models, and models based on Conditional Flow Matching. We compare all submissions in terms of quality of generated calorimeter showers, as well as shower generation time and model size. To assess the quality we use a broad range of different metrics including differences in 1-dimensional histograms of observables, KPD/FPD scores, AUCs of binary classifiers, and the log-posterior of a multiclass classifier. The results of the CaloChallenge provide the most complete and comprehensive survey of cutting-edge approaches to calorimeter fast simulation to date. In addition, our work provides a uniquely detailed perspective on the important problem of how to evaluate generative models. As such, the results presented here should be applicable for other domains that use generative AI and require fast and faithful generation of samples in a large phase space.

physics.ins-det↗

arXiv2024

SKATR: A Self-Supervised Summary Transformer for SKA

The Square Kilometer Array will initiate a new era of radio astronomy by allowing 3D imaging of the Universe during Cosmic Dawn and Reionization. Modern machine learning is crucial to analyse the highly structured and complex signal. However, accurate training data is expensive to simulate, and supervised learning may not generalize. We introduce a self-supervised vision transformer, SKATR, whose learned encoding can be cheaply adapted for downstream tasks on 21cm maps. Focusing on regression and generative inference of astrophysical and cosmological parameters, we demonstrate that SKATR representations are maximally informative and that SKATR generalises out-of-domain to differently-simulated, noised, and higher-resolution datasets.

astro-ph.IM↗

arXiv2024

CaloDREAM -- Detector Response Emulation via Attentive flow Matching

Detector simulations are an exciting application of modern generative networks. Their sparse high-dimensional data combined with the required precision poses a serious challenge. We show how combining Conditional Flow Matching with transformer elements allows us to simulate the detector phase space reliably. Namely, we use an autoregressive transformer to simulate the energy of each layer, and a vision transformer for the high-dimensional voxel distributions. We show how dimension reduction via latent diffusion allows us to train more efficiently and how diffusion networks can be evaluated faster with bespoke solvers. We showcase our framework, CaloDREAM, on datasets 2 and 3 of the CaloChallenge.

hep-ph↗

arXiv2023

Quark-versus-gluon tagging in CMS Open Data with CWoLa and TopicFlow

We use the CMS Open Data to examine the performance of weakly-supervised learning for tagging quark and gluon jets at the LHC. We target $Z$+jet and dijet events as respective quark- and gluon-enriched mixtures and derive samples both from data taken in 2011 at 7 TeV, and from Monte Carlo. CWoLa and TopicFlow models are trained on real data and compared to fully-supervised classifiers trained on simulation. In order to obtain estimates for the discrimination power in real data, we consider three different estimates of the quark/gluon mixture fractions in the data. Compared to when the models are evaluated on simulation, we find reversed rankings for the fully- and weakly-supervised approaches. Further, these rankings based on data are robust to the estimate of the mixture fraction in the test set. Finally, we use TopicFlow to smooth statistical fluctuations in the small testing set, and to provide uncertainty on the performance in real data.

hep-ph↗

arXiv2022

TopicFlow: Disentangling quark and gluon jets with normalizing flows

The isolation of pure samples of quark and gluon jets is of key interest at hadron colliders. Recent work has employed topic modeling to disentangle the underlying distributions in mixed samples obtained from experiments. However, current implementations do not scale to high-dimensional observables as they rely on binning the data. In this work we introduce TopicFlow, a method based on normalizing flows to learn quark and gluon jet topic distributions from mixed datasets. These networks are as performant as the histogram-based approach, but since they are unbinned, they are efficient even in high dimension. The models can also be oversampled to alleviate the statistical limitations of histograms. As an example use case, we demonstrate how our models can improve the calibration accuracy of a classifier. Finally, we discuss how the flow likelihoods can be used to perform outlier-robust quark/gluon classification.

hep-ph↗

arXiv2021

Meta-learning and data augmentation for mass-generalised jet taggers

Deep neural networks trained for jet tagging are typically specific to a narrow range of transverse momenta or jet masses. Given the large phase space that the LHC is able to probe, the potential benefit of classifiers that are effective over a wide range of masses or transverse momenta is significant. In this work we benchmark the performance of a number of methods for achieving accurate classification at masses distant from those used in training, with a focus on algorithms that leverage meta-learning. We study the discrimination of jets from boosted $Z'$ bosons against a QCD background. We find that a simple data augmentation strategy that standardises the angular scale of jets with different masses is sufficient to produce strong generalisation. The meta-learning algorithms provide only a small improvement in generalisation when combined with this augmentation. We also comment on the relationship between mass generalisation and mass decorrelation, demonstrating that those models which generalise better than the baseline also sculpt the background to a smaller degree.

hep-ph↗

arXiv2020

Equivariant Energy Flow Networks for Jet Tagging

Jet tagging techniques that make use of deep learning show great potential for improving physics analyses at colliders. One such method is the Energy Flow Network (EFN) - a recently introduced neural network architecture that represents jets as permutation-invariant sets of particle momenta while maintaining infrared and collinear safety. We develop a variant of the Energy Flow Network architecture based on the Deep Sets formalism, incorporating permutation-equivariant layers. We derive conditions under which infrared and collinear safety can be maintained, and study the performance of these networks on the canonical example of W-boson tagging. We find that equivariant Energy Flow Networks have similar performance to Particle Flow Networks, which are superior to standard EFNs. However, equivariant Particle Flow Networks suffer from convergence and overfitting issues. Finally, we study how equivariant networks sculpt the jet mass and provide some initial results on decorrelation using planing.

hep-ph↗