arXiv subjects

Tianyu Zhao

Publications and source records attributed to Tianyu Zhao.

At least 55 records · Page 3Linked to original sources

arXiv2025

Reinforcement Learning Teachers of Test Time Scaling

Training reasoning language models (LMs) with reinforcement learning (RL) for one-hot correctness inherently relies on the LM being able to explore and solve its task with some chance at initialization. Furthermore, a key use case of reasoning LMs is to act as teachers for distilling new students and cold-starting future RL iterations rather than being deployed themselves. From these considerations, we introduce a new framework that avoids RL's exploration challenge by training a new class of Reinforcement-Learned Teachers (RLTs) focused on yielding the most effective downstream distillation. RLTs are prompted with both the question and solution to each problem, and tasked to simply "connect-the-dots" with detailed explanations tailored for their students. We train RLTs with dense rewards obtained by feeding each explanation to the student and testing its understanding of the problem's solution. In practice, the raw outputs of a 7B RLT provide higher final performance on competition and graduate-level tasks than existing distillation and cold-starting pipelines that collect and postprocess the reasoning traces of orders of magnitude larger LMs. Furthermore, RLTs maintain their effectiveness when training larger students and when applied zero-shot to out-of-distribution tasks, unlocking new levels of efficiency and re-usability for the RL reasoning framework. Code available at: https://github.com/SakanaAI/RLT

cs.LG↗

arXiv2025

Reinforcement Learning with Rubric Anchors

Reinforcement Learning from Verifiable Rewards (RLVR) has emerged as a powerful paradigm for enhancing Large Language Models (LLMs), exemplified by the success of OpenAI's o-series. In RLVR, rewards are derived from verifiable signals-such as passing unit tests in code generation or matching correct answers in mathematical reasoning. While effective, this requirement largely confines RLVR to domains with automatically checkable outcomes. To overcome this, we extend the RLVR paradigm to open-ended tasks by integrating rubric-based rewards, where carefully designed rubrics serve as structured, model-interpretable criteria for automatic scoring of subjective outputs. We construct, to our knowledge, the largest rubric reward system to date, with over 10,000 rubrics from humans, LLMs, or a hybrid human-LLM collaboration. Implementing rubric-based RL is challenging; we tackle these issues with a clear framework and present an open-sourced Qwen-30B-A3B model with notable gains: 1) With only 5K+ samples, our system improves by +5.2% on open-ended benchmarks (especially humanities), outperforming a 671B DeepSeek-V3 model by +2.4%, while preserving general and reasoning abilities. 2) Our method provides fine-grained stylistic control, using rubrics as anchors to mitigate the "AI-like" tone and produce more human-like, expressive responses. We share key lessons in rubric construction, data selection, and training, and discuss limitations and future releases.

cs.AI↗

arXiv2025

TransEvalnia: Reasoning-based Evaluation and Ranking of Translations

We present TransEvalnia, a prompting-based translation evaluation and ranking system that uses reasoning in performing its evaluations and ranking. This system presents fine-grained evaluations based on a subset of the Multidimensional Quality Metrics (https://themqm.org/), returns an assessment of which translation it deems the best, and provides numerical scores for the various dimensions and for the overall translation. We show that TransEvalnia performs as well as or better than the state-of-the-art MT-Ranker (Moosa et al. 2024) on our own English-Japanese data as well as several language pairs from various WMT shared tasks. Using Anthropic's Claude-3.5-Sonnet and Qwen-2.5-72B-Instruct as the evaluation LLMs, we show that the evaluations returned are deemed highly acceptable to human raters, and that the scores assigned to the translations by Sonnet, as well as other LLMs, correlate well with scores assigned by the human raters. We also note the sensitivity of our system -- as well as MT-Ranker -- to the order in which the translations are presented, and we propose methods to address this position bias. All data, including the system's evaluation and reasoning, human assessments, as well as code is released.

cs.CL↗

arXiv2025

On the mean values of the error terms in Mertens' theorems

For $i\in \{1,2,3\}$, let $E_i(x)$ denote the error term in each of the three theorems of Mertens on the asymptotic distribution of prime numbers. We show that for $i\in \{1,2\}$ the Riemann hypothesis is equivalent to the condition $\int_2^X E_i(x) \:\mathrm{d}x>0$ for all $X>2$, and we examine assumptions under which the equivalence also holds for $i=3$. In addition, we extend our results to analogues of Mertens' theorems concerning prime sums twisted by quadratic Dirichlet characters or restricted to arithmetic progressions.

math.NT↗

arXiv2025

HybHuff: Lossless Compression for Hypergraphs via Entropy-Guided Huffman-Bitwise Coordination

Hypergraphs provide a natural representation for many-to-many relationships in data-intensive applications, yet their scalability is often hindered by high memory consumption. While prior work has improved computational efficiency, reducing the space overhead of hypergraph representations remains a major challenge. This paper presents a hybrid compression framework for integer-based hypergraph adjacency formats, which adaptively combines Huffman encoding and bitwise encoding to exploit structural redundancy. We provide a theoretical analysis showing that an optimal encoding ratio exists between the two schemes, and introduce an empirical strategy to approximate this ratio for practical use. Experiments on real-world hypergraphs demonstrate that our method consistently outperforms standard compressors such as Zip and ZFP in compression rate by up to 2.3x with comparable decoding overhead. To assess practical utility, we integrate our framework with three common hypergraph workloads: breadth-first search, PageRank, and k-core label propagation, and show that compression incurs negligible performance loss. Extensive evaluations across four benchmark datasets confirm the efficiency and applicability of our approach.

cs.DS↗

arXiv2025

Large Language Models to Diffusion Finetuning

We propose a new finetuning method to provide pre-trained large language models (LMs) the ability to scale test-time compute through the diffusion framework. By increasing the number of diffusion steps, we show our finetuned models achieve monotonically increasing accuracy, directly translating to improved performance across downstream tasks. Furthermore, our finetuned models can expertly answer questions on specific topics by integrating powerful guidance techniques, and autonomously determine the compute required for a given problem by leveraging adaptive ODE solvers. Our method is universally applicable to any foundation model pre-trained with a cross-entropy loss and does not modify any of its original weights, fully preserving its strong single-step generation capabilities. We show our method is more effective and fully compatible with traditional finetuning approaches, introducing an orthogonal new direction to unify the strengths of the autoregressive and diffusion frameworks.

cs.CL↗

arXiv2025

CosyVoice 3: Towards In-the-wild Speech Generation via Scaling-up and Post-training

In our prior works, we introduced a scalable streaming speech synthesis model, CosyVoice 2, which integrates a large language model (LLM) and a chunk-aware flow matching (FM) model, and achieves low-latency bi-streaming speech synthesis and human-parity quality. Despite these advancements, CosyVoice 2 exhibits limitations in language coverage, domain diversity, data volume, text formats, and post-training techniques. In this paper, we present CosyVoice 3, an improved model designed for zero-shot multilingual speech synthesis in the wild, surpassing its predecessor in content consistency, speaker similarity, and prosody naturalness. Key features of CosyVoice 3 include: 1) A novel speech tokenizer to improve prosody naturalness, developed via supervised multi-task training, including automatic speech recognition, speech emotion recognition, language identification, audio event detection, and speaker analysis. 2) A new differentiable reward model for post-training applicable not only to CosyVoice 3 but also to other LLM-based speech synthesis models. 3) Dataset Size Scaling: Training data is expanded from ten thousand hours to one million hours, encompassing 9 languages and 18 Chinese dialects across various domains and text formats. 4) Model Size Scaling: Model parameters are increased from 0.5 billion to 1.5 billion, resulting in enhanced performance on our multilingual benchmark due to the larger model capacity. These advancements contribute significantly to the progress of speech synthesis in the wild. We encourage readers to listen to the demo at https://funaudiollm.github.io/cosyvoice3.

cs.SD↗

arXiv2025

Sudoku-Bench: Evaluating creative reasoning with Sudoku variants

Existing reasoning benchmarks for large language models (LLMs) frequently fail to capture authentic creativity, often rewarding memorization of previously observed patterns. We address this shortcoming with Sudoku-Bench, a curated benchmark of challenging and unconventional Sudoku variants specifically selected to evaluate creative, multi-step logical reasoning. Sudoku variants form an unusually effective domain for reasoning research: each puzzle introduces unique or subtly interacting constraints, making memorization infeasible and requiring solvers to identify novel logical breakthroughs (``break-ins''). Despite their diversity, Sudoku variants maintain a common and compact structure, enabling clear and consistent evaluation. Sudoku-Bench includes a carefully chosen puzzle set, a standardized text-based puzzle representation, and flexible tools compatible with thousands of publicly available puzzles -- making it easy to extend into a general research environment. Baseline experiments show that state-of-the-art LLMs solve fewer than 15\% of puzzles unaided, highlighting significant opportunities to advance long-horizon, strategic reasoning capabilities.

cs.AI↗

arXiv2025

Compact Binary Coalescence Gravitational Wave Signals Counting and Separation

As next-generation gravitational-wave (GW) observatories approach unprecedented sensitivities, the need for robust methods to analyze increasingly complex, overlapping signals becomes ever more pressing. Existing matched-filtering approaches and deep-learning techniques can typically handle only one or two concurrent signals, offering limited adaptability to more varied and intricate superimposed waveforms. To overcome these constraints, we present the UnMixFormer, an attention-based architecture that not only identifies the unknown number of concurrent compact binary coalescence GW events but also disentangles their individual waveforms through a multi-decoder architecture, even when confronted with five overlapping signals. Our UnMixFormer is capable of capturing both short- and long-range dependencies by modeling them in a dual-path manner, while also enhancing periodic feature representation by incorporating Fourier Analysis Networks. Our approach adeptly processes binary black hole, binary neutron star, and neutron star-black hole systems over extended time series data (16,384 samples). When evaluating on synthetic data with signal-to-noise ratios (SNR) ranging from 10 to 50, our method achieves 99.89% counting accuracy, a mean overlap of 0.9831 between separated waveforms and templates, and robust generalization ability to waveforms with spin precession, orbital eccentricity, and higher modes, marking a substantial advance in the precision and versatility of GW data analysis.

gr-qc↗

arXiv2025

GWAI: Artificial Intelligence Platform for Enhanced Gravitational Wave Data Analysis

Gravitational wave (GW) astronomy has opened new frontiers in understanding the cosmos, while the integration of artificial intelligence (AI) in science promises to revolutionize data analysis methodologies. However, a significant gap exists, as there is currently no dedicated platform that enables scientists to develop, test, and evaluate AI algorithms efficiently for GW data analysis. To address this gap, we introduce GWAI, a pioneering AI-centered software platform designed for GW data analysis. GWAI contains a three-layered architecture that emphasizes simplicity, modularity, and flexibility, covering the entire analysis pipeline. GWAI aims to accelerate scientific discoveries, bridging the gap between advanced AI techniques and astrophysical research.

gr-qc↗

arXiv2025

Dawning of a New Era in Gravitational Wave Data Analysis: Unveiling Cosmic Mysteries via Artificial Intelligence -- A Systematic Review

Gravitational wave data analysis (GWDA) faces significant challenges due to high-dimensional parameter spaces and non-Gaussian, non-stationary artifacts in the interferometer background, which traditional methods have made significant progress in addressing but continue to face limitations. Artificial intelligence (AI), particularly deep learning (DL) algorithms, offers potential advantages, including computational efficiency, scalability, and adaptability, which may complement traditional approaches in tackling these challenges more effectively. In this review, we explore AI-driven approaches to GWDA, covering every stage of the pipeline and presenting first explorations in waveform modeling and parameter estimation. This work represents the most comprehensive review to date, integrating the latest AI advancements with practical GWDA applications. Our meta-analysis reveals insights and trends, highlighting the transformative potential of AI in revolutionizing gravitational wave research and paving the way for future discoveries.

gr-qc↗

arXiv2025

PluralLLM: Pluralistic Alignment in LLMs via Federated Learning

Ensuring Large Language Models (LLMs) align with diverse human preferences while preserving privacy and fairness remains a challenge. Existing methods, such as Reinforcement Learning from Human Feedback (RLHF), rely on centralized data collection, making them computationally expensive and privacy-invasive. We introduce PluralLLM a federated learning-based approach that enables multiple user groups to collaboratively train a transformer-based preference predictor without sharing sensitive data, which can also serve as a reward model for aligning LLMs. Our method leverages Federated Averaging (FedAvg) to aggregate preference updates efficiently, achieving 46% faster convergence, a 4% improvement in alignment scores, and nearly the same group fairness measure as in centralized training. Evaluated on a Q/A preference alignment task, PluralLLM demonstrates that federated preference learning offers a scalable and privacy-preserving alternative for aligning LLMs with diverse human values.

cs.LG↗

arXiv2025

An Evolved Universal Transformer Memory

Prior methods propose to offset the escalating costs of modern foundation models by dropping specific parts of their contexts with hand-designed rules, while attempting to preserve their original performance. We overcome this trade-off with Neural Attention Memory Models (NAMMs), introducing a learned network for memory management that improves both the performance and efficiency of transformers. We evolve NAMMs atop pre-trained transformers to provide different latent contexts focusing on the most relevant information for individual layers and attention heads. NAMMs are universally applicable to any model using self-attention as they condition exclusively on the values in the produced attention matrices. Learning NAMMs on a small set of problems, we achieve substantial performance improvements across multiple long-context benchmarks while cutting the model's input contexts up to a fraction of the original sizes. We show the generality of our conditioning enables zero-shot transfer of NAMMs trained only on language to entirely new transformer architectures even across input modalities, with their benefits carrying over to vision and reinforcement learning.

cs.LG↗

arXiv2025

MinMo: A Multimodal Large Language Model for Seamless Voice Interaction

Recent advancements in large language models (LLMs) and multimodal speech-text models have laid the groundwork for seamless voice interactions, enabling real-time, natural, and human-like conversations. Previous models for voice interactions are categorized as native and aligned. Native models integrate speech and text processing in one framework but struggle with issues like differing sequence lengths and insufficient pre-training. Aligned models maintain text LLM capabilities but are often limited by small datasets and a narrow focus on speech tasks. In this work, we introduce MinMo, a Multimodal Large Language Model with approximately 8B parameters for seamless voice interaction. We address the main limitations of prior aligned multimodal models. We train MinMo through multiple stages of speech-to-text alignment, text-to-speech alignment, speech-to-speech alignment, and duplex interaction alignment, on 1.4 million hours of diverse speech data and a broad range of speech tasks. After the multi-stage training, MinMo achieves state-of-the-art performance across various benchmarks for voice comprehension and generation while maintaining the capabilities of text LLMs, and also facilitates full-duplex conversation, that is, simultaneous two-way communication between the user and the system. Moreover, we propose a novel and simple voice decoder that outperforms prior models in voice generation. The enhanced instruction-following capabilities of MinMo supports controlling speech generation based on user instructions, with various nuances including emotions, dialects, and speaking rates, and mimicking specific voices. For MinMo, the speech-to-text latency is approximately 100ms, full-duplex latency is approximately 600ms in theory and 800ms in practice. The MinMo project web page is https://funaudiollm.github.io/minmo, and the code and models will be released soon.

cs.CL↗

arXiv2025

Universal Online Temporal Calibration for Optimization-based Visual-Inertial Navigation Systems

6-Degree of Freedom (6DoF) motion estimation with a combination of visual and inertial sensors is a growing area with numerous real-world applications. However, precise calibration of the time offset between these two sensor types is a prerequisite for accurate and robust tracking. To address this, we propose a universal online temporal calibration strategy for optimization-based visual-inertial navigation systems. Technically, we incorporate the time offset td as a state parameter in the optimization residual model to align the IMU state to the corresponding image timestamp using td, angular velocity and translational velocity. This allows the temporal misalignment td to be optimized alongside other tracking states during the process. As our method only modifies the structure of the residual model, it can be applied to various optimization-based frameworks with different tracking frontends. We evaluate our calibration method with both EuRoC and simulation data and extensive experiments demonstrate that our approach provides more accurate time offset estimation and faster convergence, particularly in the presence of noisy sensor data.

cs.RO↗

arXiv2024

CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Models

In our previous work, we introduced CosyVoice, a multilingual speech synthesis model based on supervised discrete speech tokens. By employing progressive semantic decoding with two popular generative models, language models (LMs) and Flow Matching, CosyVoice demonstrated high prosody naturalness, content consistency, and speaker similarity in speech in-context learning. Recently, significant progress has been made in multi-modal large language models (LLMs), where the response latency and real-time factor of speech synthesis play a crucial role in the interactive experience. Therefore, in this report, we present an improved streaming speech synthesis model, CosyVoice 2, which incorporates comprehensive and systematic optimizations. Specifically, we introduce finite-scalar quantization to improve the codebook utilization of speech tokens. For the text-speech LM, we streamline the model architecture to allow direct use of a pre-trained LLM as the backbone. In addition, we develop a chunk-aware causal flow matching model to support various synthesis scenarios, enabling both streaming and non-streaming synthesis within a single model. By training on a large-scale multilingual dataset, CosyVoice 2 achieves human-parity naturalness, minimal response latency, and virtually lossless synthesis quality in the streaming mode. We invite readers to listen to the demos at https://funaudiollm.github.io/cosyvoice2.

cs.SD↗

arXiv2024

On-demand Quick Metasurface Design with Neighborhood Attention Transformer

Metasurfaces are reshaping traditional optical paradigms and are increasingly required in complex applications that demand substantial computational resources to numerically solve Maxwell's equations-particularly for large-scale systems, inhomogeneous media, and densely packed metadevices. Conventional forward design using electromagnetic solvers is based on specific approximations, which may not effectively address complex problems. In contrast, existing inverse design methods are a stepwise process that is often time-consuming and involves repetitive computations. Here, we present an inverse design approach utilizing a surrogate Neighborhood Attention Transformer, MetaE-former, to predict the performance of metasurfaces with ultrafast speed and high accuracy. This method achieves global solutions for hundreds of nanostructures simultaneously, providing up to a 250,000-fold speedup compared with solving for individual meta-atoms based on the FDTD method. As examples, we demonstrate a binarized high-numerical-aperture (about 1.31) metalens and several optimized structured-light meta-generators. Our method significantly improves the beam shaping adaptability with metasurfaces and paves the way for fast designing of large-scale metadevices for shaping extreme light fields with high accuracy.

physics.optics↗

arXiv2024

High SNR 3D Imaging from Millimeter-scale Thick Tissues to Cellular Dynamics via Structured Illumination Microscopy

Three-dimensional (3D) fluorescence imaging provides a vital approach for study of biological tissues with intricate structures, and optical sectioning structured illumination microscopy (OS-SIM) stands out for its high imaging speed, low phototoxicity and high spatial resolution. However, OS-SIM faces the problem of low signal-to-noise ratio (SNR) when using traditional decoding algorithms, especially in thick tissues. Here we propose a Hilbert-transform decoding and space domain based high-low (HT-SHiLo) algorithm for noise suppression in OS-SIM. We demonstrate HT-SHiLo algorithm can significantly improve the SNR of optical sectioning images at rapid processing speed, and double the imaging depth in thick tissues. With our OS-SIM system, we achieve high quality 3D images of various biological samples including mouse brains, Drosophila clock neurons, organoids, and live cells. We anticipate that this approach will render OS-SIM a powerful technique for research of cellular organelles or thick tissues in 3D morphology.

physics.optics↗