arXiv Science⌕ Search

arXiv subjects

Omer Haq

Publications and source records attributed to Omer Haq.

6 recordsLinked to original sources

Recursive State-Augmented Language Models: Persistent Latent-State Adaptation of Pretrained Language Models

Language models rely on finite token contexts, limiting access to earlier interactions. We introduce Recursive State-Augmented Language Models (Recursive SALM), a framework in which a persistent latent state evolves with each observation and generates adaptations of a pretrained language model. A modular read-transition-write architecture separates observation encoding, latent dynamics, and predictor adaptation. We develop deterministic and stochastic formulations. The experiments evaluate only deterministic SALM with frozen GPT-2 small and exclude earlier evidence from its decoder context. On a four-turn hypothesis-inference task, latent-gated LoRA reaches 89.1% accuracy, comparable to 88.0% for a trained full-context LoRA baseline, with earlier evidence available solely through a 64-dimensional state. Zeroing or shuffling the state substantially reduces accuracy. In a two-turn task with codewords unseen in training, several interfaces improve candidate ranking over chance, but rarely generate the exact codeword under the evaluated training schedule. These experiments show that persistent latent-conditioned adaptation can support sequential inference when earlier evidence is absent from the decoder context.

cs.LG↗

EviTrack: Selection over Sampling for Delayed Disambiguation

Sequential prediction is challenging in regimes of delayed disambiguation, where early observations are ambiguous and multiple latent explanations remain plausible until sufficient evidence accumulates. Standard approaches based on marginal inference struggle in this setting, either collapsing uncertainty prematurely or failing to recover once informative evidence arrives. We introduce EviTrack, a test-time inference framework that operates over latent trajectories rather than marginal states. EviTrack maintains a set of competing trajectory hypotheses and applies evidence- and likelihood-ratio-based selection to delay commitment until supported by data, drawing inspiration from hypothesis management in multiple hypothesis tracking and track-before-detect. To evaluate this setting, we construct a controlled synthetic benchmark with known latent ground truth that explicitly exhibits delayed disambiguation. At matched inference budget, EviTrack substantially outperforms sampling-based baselines, achieving faster post-disambiguation recovery. These results show that, in delayed disambiguation regimes, moderate trajectory-level selection is more effective than increasing sampling coverage, highlighting selection over sampling as a key principle for reliable sequential inference.

cs.LG↗

Functional Distribution Networks (FDN)

Modern probabilistic regressors often remain overconfident under distribution shift. Functional Distribution Networks (FDN) place input-conditioned distributions over network weights, producing predictive mixtures whose dispersion adapts to the input; we train them with a Monte Carlo beta-ELBO objective. We pair FDN with an evaluation protocol that separates interpolation from extrapolation and emphasizes simple OOD sanity checks. On controlled 1D tasks and small/medium UCI-style regression benchmarks, FDN remains competitive in accuracy with strong Bayesian, ensemble, dropout, and hypernetwork baselines, while providing strongly input-dependent, shift-aware uncertainty and competitive calibration under matched parameter and update budgets.

cs.LG↗

LatentTrack: Sequential Weight Generation via Latent Filtering

We introduce LatentTrack (LT), a sequential neural architecture for online probabilistic prediction under nonstationary dynamics. LT performs causal Bayesian filtering in a low-dimensional latent space and uses a lightweight hypernetwork to generate predictive model parameters at each time step, enabling constant-time online adaptation without per-step gradient updates. At each time step, a learned latent model predicts the next latent distribution, which is updated via amortized inference using new observations, yielding a predict--generate--update filtering framework in function space. The formulation supports both structured (Markovian) and unstructured latent dynamics within a unified objective, while Monte Carlo inference over latent trajectories produces calibrated predictive mixtures with fixed per-step cost. Evaluated on long-horizon online regression using the Jena Climate benchmark, LT consistently achieves lower negative log-likelihood and mean squared error than stateful sequential and static uncertainty-aware baselines, with competitive calibration, demonstrating that latent-conditioned function evolution is an effective alternative to traditional latent-state modeling under distribution shift.

cs.LG↗

Elastic Wave Scattering off a Single and Double Array of Periodic Defects

Elastic waves scattering off a periodic single and double array of thin cylindrical defects is considered for isotropic materials. An analytical expression for the scattering matrix is obtained by means of the Lippmann-Schwinger formalism and analyzed in the long wavelength limit using Schloemilch series in order to obtain explicit expressions for the poles of the scattering matrix. The latter is then used to prove that for a specific curve in the space of physical and geometric parameters, the scattering is dominated by resonances, and the width of the resonances in the shear mode parallel to the cylinders has a global minimum in parameter space. This a feature is not observed in similar photonic or acoustic systems. The resonances in shear and compression modes that are coupled in the plane perpendicular to the cylinders due to the normal traction boundary condition are studied for the double array. The analytical dependence of the width of these resonances on physical and geometrical parameters is exploited to prove the existence of resonances with the vanishing width, known as Bound States in the Continuum (BSC). Spectral characteristics of BSC are explicitly found in terms of the Bloch phase and group velocities of elastic modes.

physics.class-ph↗

Bound States in the Continuum in Elasticity

Diffraction of elastic waves is considered for a system consisting of two parallel arrays of thin (subwavelength) cylinders that are arranged periodically. The embedding media supports waves with all polarizations, one longitudinal and two transverse, having different dispersion relations. An interaction with scatters mixes longitudinal and one of the transverse modes. It is shown that the system supports bound states in the continuum (BSC) that have no specific polarization, that is, there are standing waves localized in the scattering structure whose wave numbers lies in the first open diffraction channels for both longitudinal and transverse modes. BSCs are shown to exists only for specific distances between the arrays and for specific values of the wave vector component along the array. An analytic solution is obtained for such BSCs. For distances between the parallel arrays much larger than the wavelength, BSCs is proved to exist due to destructive interference of the far field resonance radiation, similar to the interference in a Fabry-Perot interferometer, that can occur simultaneously for both propagating modes.

physics.class-ph↗