arXiv Science⌕ Search

arXiv subjects

Xiao Yin

Publications and source records attributed to Xiao Yin.

7 recordsLinked to original sources

Implementing Binary Transformers with Approximate Memories with Permutation Invariance Enabled Selective (PIES) Protection

Quantization is an effective technique to reduce memory storage, computation complexity and power for Transformers, and the binarization of the weights is important for meeting these requirements. Approximate memory can further reduce the power for memory, but it also introduces errors. Current research for binary Transformers focuses on task performance, but the model dependable operation against errors on the memory has not been well studied. In this paper, we first evaluate the dependability of binary Transformers with the Bidirectional Encoder Representations from Transformers (BERT) as a case study. A large accuracy loss occurs only at high Bit Error Rates (e.g. BER = 5% or 10%). Then we propose a novel Permutation Invariance Enabled Selective (PIES) scheme to protect the binary weights at very low overhead. This scheme exploits the invariance of Transformers to permutations in the rows and columns to group the critical weights and perform selective protection on those weights. Simulation results show that PIES can recover most of the performance loss caused by bit errors at BERs as high as 10%, at an overhead significantly lower than traditional error correction schemes. These results confirm the feasibility of running binary Transformers using approximate memory with aggressive voltage scaling on resource-limited low power edge devices. The code for PIES is released on github (https://github.com/YangZhutian/binary-transformer-prefix-repair).

cs.LG↗

Improving Ads Ranking Models with Auxiliary Supervision: Techniques and Insights

Accurate prediction of post-click Conversion Rate (CVR) is essential for personalized ad delivery in the advertising industry. Traditional ads ranking models rely on conversion events (e.g., a user purchasing an item) as ground truth labels for training. However, these conversion events are often sparse or missing, which limits the effectiveness of model learning. The primary contributions of this paper are: (1) a case study on the use of auxiliary supervision from a diverse set of user events and metadata signals beyond traditional conversion events, and (2) advanced modeling techniques designed to maximize the value of these signals. We detail architectural innovations such as linearized labels for improved auxiliary signal precision, a split architecture to facilitate training task decomposition, and a novel Label Combination (LaCo) approach for robustly managing noisy auxiliary data. Empirical results demonstrate that these methods deliver measurable improvements in ranking models, underscoring their versatility and effectiveness across recommendation systems and related domains.

cs.IR↗

Physics-Constrained Deep Learning Model for Contactless Blood Pressure Monitoring from Triaxial Bodyseismography

Ballistocardiography (BCG) is promising for unobtrusive long-term blood pressure (BP) monitoring in laboratory settings, but traditional BCG signals are vulnerable to the variations in body-bed interaction with shifted fiducial points in temporal or amplitude axis, and BP varies with personal hemodynamic changes, causing misaligned representations that affect model generalizability and robustness. In this work, we propose Phy-BP, a physics-constrained BP estimation framework based on triaxial bodyseismography (BSG) acquired using bed-mounted sensors as an extension of single-axis BCG. Firstly, we design an adaptive quality-control algorithm combining neighboring beat patterns and universal cardiogenic templates to retain reliable cardiac components. Secondly, we propose a physical model of wave propagation through the body-bed system to constrain latent feature evolution, aligning triaxial representations generated by a shared cardiogenic excitation. These mechanical constraints connect multi-axis feature learning to body-bed dynamics and complement data-driven regression from vibration signals. Evaluation on a 162-hour hospital dataset from 21 subjects against invasive arterial BP yields a mean absolute error of 5.07 mmHg, a prediction-error standard deviation of 7.24 mmHg, and a Pearson correlation coefficient of 0.86 for mean arterial pressure. This framework is intended for unobtrusive monitoring during rest and sleep, with potential applications in overnight hospital and home monitoring.

eess.SP↗

Learning Individual Intrinsic Reward in Multi-Agent Reinforcement Learning via Incorporating Generalized Human Expertise

Efficient exploration in multi-agent reinforcement learning (MARL) is a challenging problem when receiving only a team reward, especially in environments with sparse rewards. A powerful method to mitigate this issue involves crafting dense individual rewards to guide the agents toward efficient exploration. However, individual rewards generally rely on manually engineered shaping-reward functions that lack high-order intelligence, thus it behaves ineffectively than humans regarding learning and generalization in complex problems. To tackle these issues, we combine the above two paradigms and propose a novel framework, LIGHT (Learning Individual Intrinsic reward via Incorporating Generalized Human experTise), which can integrate human knowledge into MARL algorithms in an end-to-end manner. LIGHT guides each agent to avoid unnecessary exploration by considering both individual action distribution and human expertise preference distribution. Then, LIGHT designs individual intrinsic rewards for each agent based on actionable representational transformation relevant to Q-learning so that the agents align their action preferences with the human expertise while maximizing the joint action value. Experimental results demonstrate the superiority of our method over representative baselines regarding performance and better knowledge reusability across different sparse-reward tasks on challenging scenarios.

cs.LG↗

Quench dynamics of spin-imbalanced Fermi-Hubbard model in one dimension

We study a nonequilibrium dynamics of a one-dimensional spin-imbalanced Fermi-Hubbard model following a quantum quench of on-site interaction, realizable, for example, in Feshbach-resonant atomic Fermi gases. We focus on the post-quench evolution starting from the initial BCS and FuldeFerrell-Larkin-Ovchinnikov (FFLO) ground states and analyze the corresponding spin-singlet, spin-triplet, density-density, and magnetization-magnetization correlation functions. We find that beyond a light-cone crossover time, rich post-quench dynamics leads to thermalized and pre-thermalized stationary states that display strong dependence on the initial ground state. For initially gapped BCS state, the long-time stationary state resembles thermalization with the effective temperature set by the initial value of the Hubbard interaction. In contrast, while the initial gapless FFLO state reaches a stationary pre-thermalized form, it remains far from equilibrium. We suggest that such post-quench dynamics can be used as a fingerprint for identification and study of the FFLO phase.

cond-mat.quant-gas↗

Post-quench dynamics and pre-thermalization in a resonant Bose gas

We explore the dynamics of a resonant Bose gas following its quench to a strongly interacting regime near a Feshbach resonance. For such deep quenches, we utilize a self-consistent dynamic field approximation and find that after an initial regime of many-body Rabi-like oscillations between the condensate and finite-momentum quasiparticle pairs, at long times, the gas reaches a pre-thermalized nonequilibrium steady state. We explore the resulting state through its broad stationary momentum distribution function, that exhibits a power-law high momentum tail. We study the dynamics and steady-state form of the associated enhanced depletion, quench-rate dependent excitation energy, Tan's contact, structure function and radio frequency spectroscopy. We find these predictions to be in a qualitative agreement with recent experiments.

cond-mat.quant-gas↗

Quench dynamics of a strongly interacting resonant Bose gas

We explore the dynamics of a Bose gas following its quench to a strongly interacting regime near a Feshbach resonance. Within a self-consistent Bogoliubov analysis we find that after the initial condensate-quasiparticle Rabi oscillations, at long time scales the gas is characterized by a nonequilibrium steady-state momentum distribution function, with depletion, condensate density and contact that deviate strongly from their corresponding equilibrium values. These are in a qualitative agreement with recent experiments on Rb85 by Makotyn, et al. Our analysis also suggests that for sufficiently deep quenches close to the resonance the nonequilibrium state undergoes a phase transition to a fully depleted state, characterized by a vanishing condensate density.

cond-mat.quant-gas↗