arXiv subjects

Matteo Boschini

Publications and source records attributed to Matteo Boschini.

18 recordsLinked to original sources

arXiv2025

LHS in LHS: A new expansion strategy for Latin hypercube sampling in simulation design

Latin Hypercube Sampling (LHS) is a prominent tool in simulation design, with a variety of applications in high-dimensional and computationally expensive problems. LHS allows for various optimization strategies, most notably to ensure space-filling properties. However, LHS is a single-stage algorithm that requires a priori knowledge of the targeted sample size. In this work, we present LHS in LHS, a new expansion algorithm for LHS that enables the addition of new samples to an existing LHS-distributed set while (approximately) preserving its properties. In summary, the algorithm identifies regions of the parameter space that are far from the initial set, draws a new LHS within those regions, and then merges it with the original samples. As a by-product, we introduce a new metric, the LHS degree, which quantifies the deviation of a given design from an LHS distribution. Our public implementation is distributed via the Python package expandLHS.

stat.ME↗

arXiv2025

Non-adiabatic dynamics of eccentric black-hole binaries in post-Newtonian theory

Eccentric black-hole binaries are among the most awaited sources of gravitational waves, yet their dynamics lack a consistent framework that provides a detailed and physically robust evolutionary description due to gauge issues. We present a new set of non-orbit-averaged equations, free from radiation-reaction gauge ambiguities, that accurately describe the evolution of orbital elements for eccentric, non-spinning black-hole binaries. We derive these equations by mapping the Keplerian orbital elements to a new set of characteristic parameters using energy and angular momentum definitions combined with near-identity transformations. The resulting framework is valid for arbitrary eccentricities, including parabolic and hyperbolic limits. Using this framework, we demonstrate the strictly observable effects of the non-adiabatic emission of gravitational waves -- characteristic of eccentric binaries -- on the orbital parameters. Furthermore, we assess the regime of validity of the widely used orbit-averaged equations first derived by Peters in 1964. Importantly, their breakdown becomes evident at the first pericenter passage, implying that the validity of the orbit-averaged approximation cannot be inferred solely from binary initial conditions. The formalism we introduce, accurate up to 2.5 post-Newtonian order, aims to provide a robust tool for making reliable astrophysical predictions and accurately interpreting current and future gravitational wave data, paving the way for deeper insights into the dynamics of eccentric black hole binaries.

gr-qc↗

arXiv2025

A recoiling supermassive black hole in a powerful quasar

Supermassive black holes (SMBH) are thought to grow through accretion of matter and mergers. Models of SMBH mergers have long suffered the final parsec problem, where SMBH binaries may stall before energy loss from gravitational waves (GW) becomes significant, leaving the pair unmerged. Direct evidence of coalesced SMBH remains elusive. Theory predicts that GW recoiling black holes can occur following a black hole merger. Here we present new and conclusive spectroscopic evidence that both the accretion disk and the broad line region in the spatially offset quasar 3C 186 are blue-shifted by the same velocity relative to the host galaxy, with a line of sight velocity of (-1310 +- 21) km/s. This is best explained by the GW recoil super-kick scenario. This confirmation of the ejection process implies that the final parsec problem is resolved in nature, providing evidence that even the most massive black holes can merge.

astro-ph.GA↗

arXiv2024

Orbital eccentricity in general relativity from catastrophe theory

While the orbital eccentricity is a key feature of the gravitational two-body problem, providing an unambiguous definition in general relativity poses significant challenges. Despite such foundational issue, the eccentricity of binary black holes has important implications in gravitational-wave astronomy. We present a novel approach to consistently define the orbital eccentricity in general relativity, grounded in the mathematical field of catastrophe theory. Specifically, we identify the presence of catastrophes, i.e., breakdowns of the stationary-phase approximation, in numerical relativity waveforms and exploit them to develop a robust and fully gauge-invariant estimator of the eccentricity. Our procedure does not require orbital fitting and naturally satisfies the Newtonian limit. The proposed eccentricity estimator agrees with and generalizes a previous proposal, though with a fully independent derivation. We extract gauge-free eccentricity estimates from about 100 numerical relativity simulations and find that the resulting values are systematically lower compared to those reported alongside the simulations themselves.

gr-qc↗

arXiv2024

An Attention-based Representation Distillation Baseline for Multi-Label Continual Learning

The field of Continual Learning (CL) has inspired numerous researchers over the years, leading to increasingly advanced countermeasures to the issue of catastrophic forgetting. Most studies have focused on the single-class scenario, where each example comes with a single label. The recent literature has successfully tackled such a setting, with impressive results. Differently, we shift our attention to the multi-label scenario, as we feel it to be more representative of real-world open problems. In our work, we show that existing state-of-the-art CL methods fail to achieve satisfactory performance, thus questioning the real advance claimed in recent years. Therefore, we assess both old-style and novel strategies and propose, on top of them, an approach called Selective Class Attention Distillation (SCAD). It relies on a knowledge transfer technique that seeks to align the representations of the student network -- which trains continuously and is subject to forgetting -- with the teacher ones, which is pretrained and kept frozen. Importantly, our method is able to selectively transfer the relevant information from the teacher to the student, thereby preventing irrelevant information from harming the student's performance during online training. To demonstrate the merits of our approach, we conduct experiments on two different multi-label datasets, showing that our method outperforms the current state-of-the-art Continual Learning methods. Our findings highlight the importance of addressing the unique challenges posed by multi-label environments in the field of Continual Learning. The code of SCAD is available at https://github.com/aimagelab/SCAD-LOD-2024.

cs.CV↗

arXiv2024

Selective Attention-based Modulation for Continual Learning

We present SAM, a biologically-plausible selective attention-driven modulation approach to enhance classification models in a continual learning setting. Inspired by neurophysiological evidence that the primary visual cortex does not contribute to object manifold untangling for categorization and that primordial attention biases are still embedded in the modern brain, we propose to employ auxiliary saliency prediction features as a modulation signal to drive and stabilize the learning of a sequence of non-i.i.d. classification tasks. Experimental results confirm that SAM effectively enhances the performance (in some cases up to about twenty percent points) of state-of-the-art continual learning methods, both in class-incremental and task-incremental settings. Moreover, we show that attention-based modulation successfully encourages the learning of features that are more robust to the presence of spurious features and to adversarial attacks than baseline methods. Code is available at: https://github.com/perceivelab/SAM.

cs.CV↗

arXiv2024

Semantic Residual Prompts for Continual Learning

Prompt-tuning methods for Continual Learning (CL) freeze a large pre-trained model and train a few parameter vectors termed prompts. Most of these methods organize these vectors in a pool of key-value pairs and use the input image as query to retrieve the prompts (values). However, as keys are learned while tasks progress, the prompting selection strategy is itself subject to catastrophic forgetting, an issue often overlooked by existing approaches. For instance, prompts introduced to accommodate new tasks might end up interfering with previously learned prompts. To make the selection strategy more stable, we leverage a foundation model (CLIP) to select our prompts within a two-level adaptation mechanism. Specifically, the first level leverages a standard textual prompt pool for the CLIP textual encoder, leading to stable class prototypes. The second level, instead, uses these prototypes along with the query image as keys to index a second pool. The retrieved prompts serve to adapt a pre-trained ViT, granting plasticity. In doing so, we also propose a novel residual mechanism to transfer CLIP semantics to the ViT layers. Through extensive analysis on established CL benchmarks, we show that our method significantly outperforms both state-of-the-art CL approaches and the zero-shot CLIP test. Notably, our findings hold true even for datasets with a substantial domain gap w.r.t. the pre-training knowledge of the backbone model, as showcased by experiments on satellite imagery and medical datasets. The codebase is available at https://github.com/aimagelab/mammoth.

cs.LG↗

arXiv2024

Astrophysical and relativistic modeling of the recoiling black-hole candidate in quasar 3C 186

The compact object in quasar 3C 186 is one of the most promising recoiling black-hole candidates, exhibiting both an astrometric displacement between the quasar and the host galaxy as well as a spectroscopic shift between broad and narrow lines. 3C 186 also presents a radio jet which, when projected onto the plane of the sky, appears to be perpendicular to the quasar/galaxy displacement. Assuming a gravitational-wave kick is indeed responsible for the properties of 3C 186 and using state-of-the-art relativistic modeling, we show that current observations allow for exquisite modeling of the recoiling black hole. Most notably, we find that the kick velocity, the black-hole spin, and the line of sight are almost collinear and the former appear perpendicular to each other only because of a strong projection effect. The targeted configuration requires substantial fine-tuning: while there exists a region in the black-hole binary parameter space that is compatible with 3C 186, the observed system appears to be a rare occurrence. Using archival radio observations, we explore different strategies that could potentially confirm or rule out our interpretation. In particular, we develop two observational tests that rely on the brightness ratio between the approaching and receding jet as well as the asymmetry of the jet lobes. While the available radio data provide loose constraints, deeper observations have the unique potential of unveiling the nature of 3C 186.

astro-ph.GA↗

arXiv2023

Extending black-hole remnant surrogate models to extreme mass ratios

Numerical-relativity surrogate models for both black-hole merger waveforms and remnants have emerged as important tools in gravitational-wave astronomy. While producing very accurate predictions, their applicability is limited to the region of the parameter space where numerical-relativity simulations are available and computationally feasible. Notably, this excludes extreme mass ratios. We present a machine-learning approach to extend the validity of existing and future numerical-relativity surrogate models toward the test-particle limit, targeting in particular the mass and spin of post-merger black-hole remnants. Our model is trained on both numerical-relativity simulations at comparable masses and analytical predictions at extreme mass ratios. We extend the gaussian-process-regression model NRSur7dq4Remnant, validate its performance via cross validation, and test its accuracy against additional numerical-relativity runs. Our fit, which we dub NRSur7dq4EmriRemnant, reaches an accuracy that is comparable to or higher than that of existing remnant models while providing robust predictions for arbitrary mass ratios.

gr-qc↗

arXiv2023

On the Effectiveness of Equivariant Regularization for Robust Online Continual Learning

Humans can learn incrementally, whereas neural networks forget previously acquired information catastrophically. Continual Learning (CL) approaches seek to bridge this gap by facilitating the transfer of knowledge to both previous tasks (backward transfer) and future ones (forward transfer) during training. Recent research has shown that self-supervision can produce versatile models that can generalize well to diverse downstream tasks. However, contrastive self-supervised learning (CSSL), a popular self-supervision technique, has limited effectiveness in online CL (OCL). OCL only permits one iteration of the input dataset, and CSSL's low sample efficiency hinders its use on the input data-stream. In this work, we propose Continual Learning via Equivariant Regularization (CLER), an OCL approach that leverages equivariant tasks for self-supervision, avoiding CSSL's limitations. Our method represents the first attempt at combining equivariant knowledge with CL and can be easily integrated with existing OCL methods. Extensive ablations shed light on how equivariant pretext tasks affect the network's information flow and its impact on CL dynamics.

cs.LG↗

arXiv2023

Latent Spectral Regularization for Continual Learning

While biological intelligence grows organically as new knowledge is gathered throughout life, Artificial Neural Networks forget catastrophically whenever they face a changing training data distribution. Rehearsal-based Continual Learning (CL) approaches have been established as a versatile and reliable solution to overcome this limitation; however, sudden input disruptions and memory constraints are known to alter the consistency of their predictions. We study this phenomenon by investigating the geometric characteristics of the learner's latent space and find that replayed data points of different classes increasingly mix up, interfering with classification. Hence, we propose a geometric regularizer that enforces weak requirements on the Laplacian spectrum of the latent space, promoting a partitioning behavior. Our proposal, called Continual Spectral Regularizer for Incremental Learning (CaSpeR-IL), can be easily combined with any rehearsal-based CL approach and improves the performance of SOTA methods on standard benchmarks.

cs.LG↗

arXiv2022

On the Effectiveness of Lipschitz-Driven Rehearsal in Continual Learning

Rehearsal approaches enjoy immense popularity with Continual Learning (CL) practitioners. These methods collect samples from previously encountered data distributions in a small memory buffer; subsequently, they repeatedly optimize on the latter to prevent catastrophic forgetting. This work draws attention to a hidden pitfall of this widespread practice: repeated optimization on a small pool of data inevitably leads to tight and unstable decision boundaries, which are a major hindrance to generalization. To address this issue, we propose Lipschitz-DrivEn Rehearsal (LiDER), a surrogate objective that induces smoothness in the backbone network by constraining its layer-wise Lipschitz constants w.r.t. replay examples. By means of extensive experiments, we show that applying LiDER delivers a stable performance gain to several state-of-the-art rehearsal CL methods across multiple datasets, both in the presence and absence of pre-training. Through additional ablative experiments, we highlight peculiar aspects of buffer overfitting in CL and better characterize the effect produced by LiDER. Code is available at https://github.com/aimagelab/LiDER

cs.LG↗

arXiv2022

Effects of Auxiliary Knowledge on Continual Learning

In Continual Learning (CL), a neural network is trained on a stream of data whose distribution changes over time. In this context, the main problem is how to learn new information without forgetting old knowledge (i.e., Catastrophic Forgetting). Most existing CL approaches focus on finding solutions to preserve acquired knowledge, so working on the past of the model. However, we argue that as the model has to continually learn new tasks, it is also important to put focus on the present knowledge that could improve following tasks learning. In this paper we propose a new, simple, CL algorithm that focuses on solving the current task in a way that might facilitate the learning of the next ones. More specifically, our approach combines the main data stream with a secondary, diverse and uncorrelated stream, from which the network can draw auxiliary knowledge. This helps the model from different perspectives, since auxiliary data may contain useful features for the current and the next tasks and incoming task classes can be mapped onto auxiliary classes. Furthermore, the addition of data to the current task is implicitly making the classifier more robust as we are forcing the extraction of more discriminative features. Our method can outperform existing state-of-the-art models on the most common CL Image Classification benchmarks.

cs.LG↗

arXiv2022

Transfer without Forgetting

This work investigates the entanglement between Continual Learning (CL) and Transfer Learning (TL). In particular, we shed light on the widespread application of network pretraining, highlighting that it is itself subject to catastrophic forgetting. Unfortunately, this issue leads to the under-exploitation of knowledge transfer during later tasks. On this ground, we propose Transfer without Forgetting (TwF), a hybrid approach building upon a fixed pretrained sibling network, which continuously propagates the knowledge inherent in the source domain through a layer-wise loss term. Our experiments indicate that TwF steadily outperforms other CL methods across a variety of settings, averaging a 4.81% gain in Class-Incremental accuracy over a variety of datasets and different buffer sizes.

cs.LG↗

arXiv2022

Class-Incremental Continual Learning into the eXtended DER-verse

The staple of human intelligence is the capability of acquiring knowledge in a continuous fashion. In stark contrast, Deep Networks forget catastrophically and, for this reason, the sub-field of Class-Incremental Continual Learning fosters methods that learn a sequence of tasks incrementally, blending sequentially-gained knowledge into a comprehensive prediction. This work aims at assessing and overcoming the pitfalls of our previous proposal Dark Experience Replay (DER), a simple and effective approach that combines rehearsal and Knowledge Distillation. Inspired by the way our minds constantly rewrite past recollections and set expectations for the future, we endow our model with the abilities to i) revise its replay memory to welcome novel information regarding past data ii) pave the way for learning yet unseen classes. We show that the application of these strategies leads to remarkable improvements; indeed, the resulting method - termed eXtended-DER (X-DER) - outperforms the state of the art on both standard benchmarks (such as CIFAR-100 and miniImagenet) and a novel one here introduced. To gain a better understanding, we further provide extensive ablation studies that corroborate and extend the findings of our previous research (e.g. the value of Knowledge Distillation and flatter minima in continual learning setups).

cs.LG↗

arXiv2021

Continual Semi-Supervised Learning through Contrastive Interpolation Consistency

Continual Learning (CL) investigates how to train Deep Networks on a stream of tasks without incurring forgetting. CL settings proposed in literature assume that every incoming example is paired with ground-truth annotations. However, this clashes with many real-world applications: gathering labeled data, which is in itself tedious and expensive, becomes infeasible when data flow as a stream. This work explores Continual Semi-Supervised Learning (CSSL): here, only a small fraction of labeled input examples are shown to the learner. We assess how current CL methods (e.g.: EWC, LwF, iCaRL, ER, GDumb, DER) perform in this novel and challenging scenario, where overfitting entangles forgetting. Subsequently, we design a novel CSSL method that exploits metric learning and consistency regularization to leverage unlabeled examples while learning. We show that our proposal exhibits higher resilience to diminishing supervision and, even more surprisingly, relying only on 25% supervision suffices to outperform SOTA methods trained under full supervision.

stat.ML↗

arXiv2020

Rethinking Experience Replay: a Bag of Tricks for Continual Learning

In Continual Learning, a Neural Network is trained on a stream of data whose distribution shifts over time. Under these assumptions, it is especially challenging to improve on classes appearing later in the stream while remaining accurate on previous ones. This is due to the infamous problem of catastrophic forgetting, which causes a quick performance degradation when the classifier focuses on learning new categories. Recent literature proposed various approaches to tackle this issue, often resorting to very sophisticated techniques. In this work, we show that naive rehearsal can be patched to achieve similar performance. We point out some shortcomings that restrain Experience Replay (ER) and propose five tricks to mitigate them. Experiments show that ER, thus enhanced, displays an accuracy gain of 51.2 and 26.9 percentage points on the CIFAR-10 and CIFAR-100 datasets respectively (memory buffer size 1000). As a result, it surpasses current state-of-the-art rehearsal-based methods.

cs.LG↗

arXiv2020

Dark Experience for General Continual Learning: a Strong, Simple Baseline

Continual Learning has inspired a plethora of approaches and evaluation settings; however, the majority of them overlooks the properties of a practical scenario, where the data stream cannot be shaped as a sequence of tasks and offline training is not viable. We work towards General Continual Learning (GCL), where task boundaries blur and the domain and class distributions shift either gradually or suddenly. We address it through mixing rehearsal with knowledge distillation and regularization; our simple baseline, Dark Experience Replay, matches the network's logits sampled throughout the optimization trajectory, thus promoting consistency with its past. By conducting an extensive analysis on both standard benchmarks and a novel GCL evaluation setting (MNIST-360), we show that such a seemingly simple baseline outperforms consolidated approaches and leverages limited resources. We further explore the generalization capabilities of our objective, showing its regularization being beneficial beyond mere performance.

stat.ML↗