arXiv subjects

Zining Fan

Publications and source records attributed to Zining Fan.

4 recordsLinked to original sources

arXiv2026

OPERA: Aligning Open-Ended Reasoning via Objective Perplexity-based Reinforcement Learning

Reinforcement Learning (RL) has enabled LLMs to excel in objective reasoning tasks such as mathematics and code generation. However, applying RL to open-ended tasks, such as creative writing, remains challenging because LLM-as-a-judge reward models often exhibit stylistic biases and positional inconsistencies, leading to unstable supervision. To address this, we propose OPERA (Objective Perplexity-based Reflective Alignment), which replaces unreliable external judges with intrinsic rewards derived from perplexity dynamics. Specifically, we derive an intrinsic reward signal from perplexity dynamics, quantifying uncertainty reduction at critical reflective states. During the cold-start phase, we introduce a data synthesis method that leverages carefully designed guiding words to generate diverse reasoning traces, along with perplexity-prioritized rollouts that utilize internal log-probabilities to identify logically consistent reasoning branches. This pipeline yields a large-scale dataset comprising 20,000 high-quality reasoning trajectories. Empirical evaluations consistently demonstrate the scalability and efficacy of our approach in alignment for open-ended tasks. Implementing OPERA on Qwen3-8B establishes a new state-of-the-art among open-source models, achieving parity with or surpassing proprietary models like Gemini2.5 and MiniMax-M2.5 in some open-ended tasks. The code is available at https://github.com/pangpang-xuan/OPERA.

cs.CL↗

arXiv2026

TR-ICRL: Test-Time Rethinking for In-Context Reinforcement Learning

In-Context Reinforcement Learning (ICRL) enables Large Language Models (LLMs) to learn online from external rewards directly within the context window. However, a central challenge in ICRL is reward estimation, as models typically lack access to ground-truths during inference. To address this limitation, we propose Test-Time Rethinking for In-Context Reinforcement Learning (TR-ICRL), a novel ICRL framework designed for both reasoning and knowledge-intensive tasks. TR-ICRL operates by first retrieving the most relevant instances from an unlabeled evaluation set for a given query. During each ICRL iteration, LLM generates a set of candidate answers for every retrieved instance. Next, a pseudo-label is derived from this set through majority voting. This label then serves as a proxy to give reward messages and generate formative feedbacks, guiding LLM through iterative refinement. In the end, this synthesized contextual information is integrated with the original query to form a comprehensive prompt, with the answer determining through a final round of majority voting. TR-ICRL is evaluated on mainstream reasoning and knowledge-intensive tasks, where it demonstrates significant performance gains. Remarkably, TR-ICRL improves Qwen2.5-7B by 21.23% on average on MedQA and even 137.59% on AIME2024. Extensive ablation studies and analyses further validate the effectiveness and robustness of our approach. Our code is available at https://github.com/pangpang-xuan/TR_ICRL.

cs.CL↗

arXiv2025

Detecting Causality with Conjugation Quandles over Dihedral Groups

We study whether quandle colorings can detect causality of events for links realized as skies in a $(2+1)$-dimensional globally hyperbolic spacetime $X$. Building off the Allen--Swenberg paper in which their $2$-sky link was conjectured to be causally related, they showed that the Alexander--Conway polynomial does not distinguish that link from the connected sum of two Hopf links, corresponding to two causally unrelated events. We ask whether the Alexander--Conway polynomial together with different types of quandle invariants suffice. We show that the conjugation quandle of the dihedral group $D_5$, together with the Alexander--Conway polynomial, does distinguish the two links and hence likely does detect causality in $X$. The $2$-sky link shares the same Alexander--Conway polynomial but has different $D_5$ conjugation-quandle counting invariants. Moreover, for $D_5$ the counting invariant alone already separates the pair, whereas for other small dihedral groups $D_3$, $D_4$, $D_6$, $D_7$ even the enhanced counting polynomial fails to detect causality. In fact we prove more that the conjugation quandle over D5 distinguishes all the infinitely many Allen-Swenberg links from the connected sum of two Hopf links. These results present an interesting reality where only the conjugation quandle over $D_5$ coupled with the Alexander--Conway polynomial can detect causality in $(2+1)$ dimensions. This results in a simple, computable quandle that can determine causality via the counting invariant alone, rather than reaching for more complicated counting polynomials and cocycles.

math.GT↗

arXiv2024

Seamless Website Fingerprinting in Multiple Environments

Website fingerprinting (WF) attacks identify the websites visited over anonymized connections by analyzing patterns in network traffic flows, such as packet sizes, directions, or interval times using a machine learning classifier. Previous studies showed WF attacks achieve high classification accuracy. However, several issues call into question whether existing WF approaches are realizable in practice and thus motivate a re-exploration. Due to Tor's performance issues and resulting poor browsing experience, the vast majority of users opt for Virtual Private Networking (VPN) despite VPNs weaker privacy protections. Many other past assumptions are increasingly unrealistic as web technology advances. Our work addresses several key limitations of prior art. First, we introduce a new approach that classifies entire websites rather than individual web pages. Site-level classification uses traffic from all site components, including advertisements, multimedia, and single-page applications. Second, our Convolutional Neural Network (CNN) uses only the jitter and size of 500 contiguous packets from any point in a TCP stream, in contrast to prior work requiring heuristics to find page boundaries. Our seamless approach makes eavesdropper attack models realistic. Using traces from a controlled browser, we show our CNN matches observed traffic to a website with over 90% accuracy. We found the training traffic quality is critical as classification accuracy is significantly reduced when the training data lacks variability in network location, performance, and clients' computational capability. We enhanced the base CNN's efficacy using domain adaptation, allowing it to discount irrelevant features, such as network location. Lastly, we evaluate several defensive strategies against seamless WF attacks.

cs.CR↗