arXiv subjects

Nakyung Lee

Publications and source records attributed to Nakyung Lee.

12 recordsLinked to original sources

arXiv2026

Towards Understanding Pause Token Fine-Tuning Dynamics: A Mode Retention Perspective

Pause-token methods improve LLM reasoning by inserting special tokens into sequences. Prior work explains these gains through computational expressivity. However, there is relatively little investigation into the training dynamics of pause tokens. We explore how pause tokens reshape the training dynamics of fine-tuning. Two controlled pilots expose distinct asymmetries. On a synthetic continual-learning task, masked pauses overwrite a previously-learned distribution roughly 4x less at matched final adaptation (H1, mode retention); on a synthetic math-reasoning probe, the boundary-adjacent token comes to encode substantially more downstream-step information (H2, non-myopic compression). We formalize a training rule consistent with both - Masked Boundary Pause (MBP), pause tokens placed at reasoning-step boundaries with their loss masked. Across 1B-8B Qwen and Llama models, MBP consistently improves reasoning, achieving gains of up to 6 points on math and 2.5 points on code, while preserving general language understanding abilities. We further demonstrate that this mode-preserving strategy extend gains to GRPO. These results recast pause tokens as a training-dynamics intervention on the retention-adaptation trade-off, rather than merely an inference-time computation device.

cs.CL↗

arXiv2026

DCGC: Draft-Conditioned Global Correction for Complex Reasoning with Masked Diffusion Models

Correcting flawed reasoning traces remains a significant challenge for Large Language Models (LLMs), whose autoregressive generation can propagate early mistakes into subsequent reasoning. We introduce DCGC, a Masked Diffusion Model (MDM) framework for global correction that uses an imperfect solution draft from an upstream solver as auxiliary context. DCGC combines task-specific Supervised Fine-Tuning (SFT) with a novel inference-time mechanism called Dynamic Dual-CFG. This mechanism separates problem-only and joint problem-draft branches and scales the draft-conditioned residual using a relative confidence gap. Across math, code, and knowledge reasoning benchmarks, DCGC outperforms standard sampling and simpler CFG variants, with additional results suggesting transfer to different diffusion backbones. In test-time setting where ground-truth failure labels are unavailable, DCGC improves full test set accuracy by correcting low-consensus upstream outputs, highlighting its utility as a verifier-free global correction module for difficult reasoning instances.

cs.CL↗

arXiv2026

Mechanistic Circuit Identification for Controllable Data Generation

While recent advances in data synthesis aim to curate high-quality datasets, most generation pipelines still rely on heuristic prompt-based control. This black-box paradigm provides limited insight into how individual samples interact with a model's underlying learning dynamics. To bridge this gap, we propose a circuit-grounded framework that connects training-dynamics-based data valuation with mechanistic interpretability (MI). Specifically, we conceptualize data quality along three complementary utility axes, learnability, challenge, and alignment. First, we uncover specialized model-internal circuits that causally govern these utility signals. Then, moving beyond heuristic prompting toward mechanistic control, we leverage these circuits as controllable interfaces, actively steering generation to produce utility-targeted data. Building on this capability, we introduce SAMS (Stage-Aware Mechanistic Scheduling), which schedules circuit-steered data according to the model's evolving optimization needs. Experiments on multiple-choice QA tasks demonstrate that our approach yields precisely controlled data with greater diversity than prompt-based baselines, consistently improving downstream performance and calibration. Ultimately, this work establishes a principled white-box paradigm for interpretable data generation, pioneering the use of MI not just as an analytical tool, but as a practical, controllable interface.

cs.LG↗

arXiv2026

Database Normalization via Dual-LLM Self-Refinement

Database normalization is crucial to preserving data integrity. However, it is time-consuming and error-prone, as it is typically performed manually by data engineers. To this end, we present Miffie, a database normalization framework that leverages the capability of large language models. Miffie enables automated data normalization without human effort while preserving high accuracy. The core of Miffie is a dual-model self-refinement architecture that combines the best-performing models for normalized schema generation and verification, respectively. The generation module eliminates anomalies based on the feedback of the verification module until the output schema satisfies the requirement for normalization. We also carefully design task-specific zero-shot prompts to guide the models for achieving both high accuracy and cost efficiency. Experimental results show that Miffie can normalize complex database schemas while maintaining high accuracy.

cs.DB↗

arXiv2026

Efficient Process Reward Modeling via Contrastive Mutual Information

Recent research has devoted considerable effort to verifying the intermediate reasoning steps of chain-of-thought (CoT) trajectories using process reward models (PRMs) and other verifier models. However, training a PRM typically requires human annotators to assign reward scores to each reasoning step, which is both costly and time-consuming. Existing automated approaches, such as Monte Carlo (MC) estimation, also demand substantial computational resources due to repeated LLM rollouts. To overcome these limitations, we propose contrastive pointwise mutual information (CPMI), a novel automatic reward labeling method that leverages the model's internal probability to infer step-level supervision while significantly reducing the computational burden of annotating dataset. CPMI quantifies how much a reasoning step increases the mutual information between the step and the correct target answer relative to hard-negative alternatives. This contrastive signal serves as a proxy for the step's contribution to the final solution and yields a reliable reward. The experimental results show that CPMI-based labeling reduces dataset construction time by 84% and token generation by 98% compared to MC estimation, while achieving higher accuracy on process-level evaluations and mathematical reasoning benchmarks.

cs.CL↗

arXiv2026

GateANN: I/O-Efficient Filtered Vector Search on SSDs

We present GateANN, an I/O-efficient SSD-based graph ANNS system that supports filtered vector search on an unmodified graph index. Existing SSD-based systems either waste I/O by post-filtering, or require expensive filter-aware index rebuilds. GateANN avoids both by decoupling graph traversal from vector retrieval. Our key insight is that traversing a node requires only its neighbor list and an approximate distance, neither of which needs the full-precision vector on SSD. Based on this, GateANN introduces graph tunneling. It checks each node's filter predicate in memory before issuing I/O and routes through non-matching nodes entirely in memory, preserving graph connectivity without any SSD read for non-matching nodes. Our experimental results show that it reduces SSD reads by up to 10x and improves throughput by up to 7.6x.

cs.OS↗

arXiv2026

RAISE: Enhancing Scientific Reasoning in LLMs via Step-by-Step Retrieval

Scientific reasoning requires not only long-chain reasoning processes, but also knowledge of domain-specific terminologies and adaptation to updated findings. To deal with these challenges for scientific reasoning, we introduce RAISE, a step-by-step retrieval-augmented framework which retrieves logically relevant documents from in-the-wild corpus. RAISE is divided into three steps: problem decomposition, logical query generation, and logical retrieval. We observe that RAISE consistently outperforms other baselines on scientific reasoning benchmarks. We analyze that unlike other baselines, RAISE retrieves documents that are not only similar in terms of the domain knowledge, but also documents logically more relevant.

cs.CL↗

arXiv2026

Enumeration of multiplex juggling card sequences using generalized q-derivatives

In 2019, Butler, Choi, Kim, and Seo introduced a new type of juggling card that represents multiplex juggling patterns in a natural bijective way. They conjectured a formula for the generating function for the number of multiplex juggling cards with capacity 2. In this paper we prove their conjecture. More generally, we find an explicit formula for the generating function with any capacity. We also find an expression for the generating function for multiplex juggling card sequences by introducing a generalization of the q-derivative operator. As a consequence, we show that this generating function is a rational function.

math.CO↗

arXiv2025

Blocker-Aware Beamforming and Dynamic Power Allocation for Multicarrier ISAC-NOMA Systems

This paper proposes a blocker-aware multicarrier integrated sensing and communication (ISAC)-non orthogonal multiple access (NOMA) system, leveraging hybrid beamforming and dynamic power allocation to enhance spectrum efficiency in 6G networks. Recognizing the performance degradation caused by environmental blockers, the system introduces a joint waveform design that ensures robust operation under varying channel conditions. A channel switching mechanism is deployed to reroute communication through alternative non-line-of-sight paths when the primary line-of-sight links are obstructed. Moreover, a dynamic power allocation strategy enforces a minimum rate constraint for the weak NOMA user, ensuring consistent quality of service. Extensive simulations over multiple blockage scenarios and signal to noise (SNR) conditions validate the effectiveness of the proposed solution. Notably, under severe blockage, the system achieves up to a 400% sensing rate enhancement at 15 dB SNR, with only a 20% reduction in communication rate. These results corroborate the system's ability to adapt and optimize joint sensing-communication performance in practical deployment environments.

eess.SP↗

arXiv2025

Mitigating Attention Localization in Small Scale: Self-Attention Refinement via One-step Belief Propagation

Transformer-based self-attention mechanism serves as the core of modern language models, yet it often suffers from localization, where attentions collapse onto a limited subset of tokens and fail to capture long-range dependencies. To address this issue, we propose Self-Attention One-step Belief Propagation (SAOBP), a refinement framework that injects multi-hop relationships through a belief propagation process. To interpret and quantify these interactions, we introduce Global Token Dependency (GTD) that captures the relative contribution of multihop connections within the attention graph. Empirical results indicate that SAOBP helps prevent entropy collapse in deeper layers and adaptively maintains GTD at task-appropriate levels, thereby supporting improvements in model performance. Importantly, we observe competitive gains in small-scale models, highlighting its potential for improving inference quality in resource-constrained scenarios.

cs.CL↗

arXiv2025

Modulating Optical Properties through Cation Substitution: Composition-Property Relationships in $M^I_3$$M^{III}$P$_3$O$_9$N:Eu$^{2+}$ ($M^I$=Na, K; $M^{III}$=Al, Ga, In)

Developing phosphors with narrow photoluminescence emission peaks and high chromatic stability holds significant importance in light-emitting diode (LED) display technologies, where a wide color gamut is essential to achieve the Rec. 2020 specifications. This research focuses on the optical properties of a solid solution: $M^I_{2.97}$Eu$_{0.015}$$M^{III}$P$_3$O$_9$N [$M^I$=Na, K; $M^{III}$=Al, (Al$_{0.75}$Ga$_{0.25}$), (Al$_{0.5}$Ga$_{0.5}$), (Al$_{0.25}$Ga$_{0.75}$), Ga, (Ga$_{0.75}$In$_{0.25}$), (Ga$_{0.5}$In$_{0.5}$)] to understand how the narrow-emitting photoluminescence in K$_3$AlP$_3$O$_9$N:Eu$^{2+}$ can evolve during host structure cation substitution. Photoluminescence measurements at low temperature (15 K) support that Eu$^{2+}$ replaces three crystallographically independent Na$^+$ sites in Na$_{2.97}$Eu$_{0.015}$AlP$_3$O$_9$N, similar to the parent K$^+$ phosphor, but substituting Ga$^{3+}$ and In$^{3+}$ for Al$^{3+}$ leads to a change in Eu$^{2+}$ site preference, narrowing the full-width-at-half-maximum (fwhm) of the emission peak. The chromatic stability and photoluminescence quantum yield are also enhanced with higher Ga$^{3+}$ content in the host but not with In$^{3+}$. Thermoluminescence analysis indicates the relationship between trap states and the enhanced quantum yield with Ga$^{3+}$ leads to the series' best performance. The analysis of the $M^I_{2.97}$Eu$_{0.015}$$M^{III}$P$_3$O$_9$N series offers insight into the potential method for modulating optical properties with cation substitution in the host structure.

cond-mat.mtrl-sci↗

arXiv2025

Machine Learning a Phosphor's Excitation Band Position

Creating superior lanthanide-activated inorganic phosphors is pivotal for advancing energy-efficient LED lighting and backlit flat panel displays. The most fundamental property these luminescent materials must possess is effective absorption/excitation by a blue InGaN LED for practical conversion into white light. The 5$d_1$ excited state energy level of lanthanides, which determines the excitation peak position, is influenced by the inorganic host structure, including the local environment, crystal structure, and composition, making it challenging to predict in advance. This study introduces a new extreme gradient boosting machine learning method that quantitatively determines a phosphor's longest (lowest energy) excitation wavelength. We focus on the Ce$^{3+}$ 4$f$ $\rightarrow$ 5$d$ transition due to its well-defined 5$d_1$ energy level observed in excitation and diffuse reflectance spectra. The model was trained on experimental data for 357 Ce$^{3+}$ cation substitution sites sourced from literature and in-house measurements and ultimately experimentally validated through the successful synthesis of a novel, blue-excited, green-emitting phosphor: Ca$_2$SrSc$_6$O$_{12}$:Ce$^{3+}$. This compound's excitation under commercial blue LED wavelength aligned remarkably well with the model's predictions. These results highlight the transformative potential of data-driven approaches in expediting the discovery of blue-absorbing phosphors for next-generation LED lighting.

cond-mat.mtrl-sci↗