arXiv subjects

Wenhao Zhao

Publications and source records attributed to Wenhao Zhao.

21 records · Page 2Linked to original sources

arXiv2025

Properties of QCD axion in two-flavor color superconductive matter with massive quarks

We investigate the properties of QCD axion at low temperature and moderate density in the Nambu-Jona-Lasinio model with instanton induced interactions by simultaneously considering the scalar and pseudo-scalar condensates in both quark-antiquark and diquark channels. We derive the analytical dispersion relations of quarks with four-type condensates at nonzero theta angle $θ=a/f_a$. The axion mass, quartic self-coupling, and the axion potential are calculated in both the chiral symmetry breaking and two-flavor color superconducting phases. Using the commonly adopted model parameters, we find that due to the emergence of color superconductivity, the chiral phase transition not only does not lead to a significant decrease in axion mass and self-coupling, but rather results in an obvious enhancement of them. As a $θ$ function, the axion potential exhibits an appropriate period of $π$, which is quite different from the case without considering the color superconductivity. The surface tension of axion domain wall is also calculated in the presence of color superconductivity.

hep-ph↗

arXiv2024

Representation Collapsing Problems in Vector Quantization

Vector quantization is a technique in machine learning that discretizes continuous representations into a set of discrete vectors. It is widely employed in tokenizing data representations for large language models, diffusion models, and other generative models. Despite its prevalence, the characteristics and behaviors of vector quantization in generative models remain largely underexplored. In this study, we investigate representation collapse in vector quantization - a critical degradation where codebook tokens or latent embeddings lose their discriminative power by converging to a limited subset of values. This collapse fundamentally compromises the model's ability to capture diverse data patterns. By leveraging both synthetic and real datasets, we identify the severity of each type of collapses and triggering conditions. Our analysis reveals that restricted initialization and limited encoder capacity result in tokens collapse and embeddings collapse. Building on these findings, we propose potential solutions aimed at mitigating each collapse. To the best of our knowledge, this is the first comprehensive study examining representation collapsing problems in vector quantization.

cs.LG↗

arXiv2024

Locally Differentially Private In-Context Learning

Large pretrained language models (LLMs) have shown surprising In-Context Learning (ICL) ability. An important application in deploying large language models is to augment LLMs with a private database for some specific task. The main problem with this promising commercial use is that LLMs have been shown to memorize their training data and their prompt data are vulnerable to membership inference attacks (MIA) and prompt leaking attacks. In order to deal with this problem, we treat LLMs as untrusted in privacy and propose a locally differentially private framework of in-context learning(LDP-ICL) in the settings where labels are sensitive. Considering the mechanisms of in-context learning in Transformers by gradient descent, we provide an analysis of the trade-off between privacy and utility in such LDP-ICL for classification. Moreover, we apply LDP-ICL to the discrete distribution estimation problem. In the end, we perform several experiments to demonstrate our analysis results.

cs.CR↗