arXiv Science⌕ Search
Home/Archive/arXiv

arXiv · 2610.23670

OwlSight: A Panoramic Monitor for Chain-of-Thought Reasoning

Lewen Yan·Yanxu Zhu·Jie Zhang·Jing Shao
View original source ↗Read source document ↗Export metadata ↓

Abstract

Chain-of-Thought (CoT) monitoring offers actionable evidence for oversight AI systems, with the potential to enable early detection and intervention. However, existing approaches are focused on specific tasks, whereas the regulation of AI, particularly reasoning models, requires monitoring a broader range of tasks. We introduce Factor for CoT monitoring, a strategy that supports customizing monitoring requirements in the prompt, which enables a single monitor to cover different tasks by relying on different factors. Furthermore, we introduce two types of factors, fine-grained and coarse-grained (e.g., monitoring whether a specific email address has been leaked versus monitoring privacy leakage), to accommodate diverse user monitoring needs. Based on this, we develop a CoT monitor named OwlSight that relies solely on factor to monitor behaviors related to multiple dimensions, including capability, safety, and trustworthiness. Experiments across 15,971 reasoning trajectories show that OwlSight surpasses general-purpose LLMs by 1.32-30.57 and 11.71-27.81 points at fine and coarse granularity, respectively, and achieves state-of-the-art results on OpenAI's CoT monitorability benchmark. It also generalizes to unseen models' CoTs and remains robust to Factor rephrasings, CoT compression, and obfuscation, offering a promising paradigm for CoT monitoring that can contribute to the oversight of frontier AI.

Explore related subjects

cs.AI

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Lewen Yan, Yanxu Zhu, Jie Zhang, Jing Shao. 2026-10-16. OwlSight: A Panoramic Monitor for Chain-of-Thought Reasoning. https://arxiv.org/abs/2610.23670

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Python Tool for Visualizing Variability of Pareto Fronts over Multiple Runs

Hyperparameter optimization is crucial to achieving high performance in deep learning. On top of the performance, other criteria such as inference time or memory requirement often need to be optimized due to some practical reasons. This motivates research on multi-objective optimization (MOO). However, Pareto fronts of MOO methods are often shown without considering the variability caused by random seeds and this makes the performance stability evaluation difficult. Although there is a concept named \emph{empirical attainment surface} to enable the visualization with uncertainty over multiple runs, there is no major Python package for empirical attainment surface. We, therefore, develop a Python package for this purpose and describe the usage. The package is available at https://github.com/nabenabe0928/empirical-attainment-func. OptunaHub now provides our implementation at https://hub.optuna.org/visualization/plot_empirical_attainment_surface/.

cs.AI↗
arXiv2026

From Language Models to Control Policies: Search, Selection, and Verification

Industrial process control demands policies that are interpretable and auditable, requirements that black-box neural policies struggle to meet. LLM-driven heuristic synthesis offers an alternative: a language model iteratively proposes and refines human-readable Python controllers using rich behavioral feedback from a simulator. We study this form of programmatic policy search on hot steel rolling, a physics-simulated, multi-pass control problem with competing objectives and hard equipment constraints, and address three questions that arise once the search is run at scale: how to split a fixed budget of LLM iterations between restarting and refining, how to deploy the resulting pool of heuristics, and how to verify it. Treating each search run as a randomized algorithm, we find across 52 independent runs that runs of moderate length match the hindsight-optimal budget allocation, and that Luby-style universal restarts reach this regime without problem-specific tuning. Because each heuristic is cheap to simulate, a small portfolio selected per job outperforms the best single heuristic, also under cross-validation and on out-of-distribution operating conditions on which the best heuristic stalls. Finally, an automated audit pipeline combining static analysis, Z3 proofs, and property-based testing proves key safety properties for all 174 synthesized heuristics; the stall, which neither the search reward nor these properties reveal, shows that the specifications must also cover progress.

cs.AI↗
arXiv2026

Virtual Speech Therapist: A Clinician-in-the-Loop AI Agent for Speech Therapy Planning

This paper presents a proof of concept of the Virtual Speech Therapist (VST), an intelligent multi-agent based system designed to support stuttering assessment and individualized therapy planning through artificial intelligence (AI) driven workflows. VST integrates deep learning (DL) based stuttering detection fSD with large language model (LLM) based agentic system to assist clinicians in developing evidence informed therapy plans. The VST system begins with the acquisition and feature extraction of given speech samples, followed by the detection of stuttering types. Building on these speech driven information S, VST initiates an agentic reasoning process in which specialized LLM agents autonomously generate, critique, and iteratively refine individualized therapy plans P. A dedicated CriticAgent πca acts as a structured self-refinement mechanism, reviewing the generated therapy plans P and identifying areas for potential improvement in consistency, methodological soundness, and alignment with available evidence and professional guidelines. The resulting therapy plan P is presented then as a draft for clinician review. Clinician feedback can then be incorporated to produce a revised plan P, thereby maintaining a clinician-in-the-loop (CITL) paradigm. An initial evaluation, involving expert speech language pathologist (SLP) examines the clinical interpretability and feasibility of the generated therapy plans. Taken together, the results show the potential of VST to augment clinician-led therapy planning and provide a promising basis for future studies evaluating its integration into real world clinical settings. An interactive user interface for the proposed prototype is available online at: https://vocametrix.com/ai/stuttering-therapy-planning-agent, facilitating interactive stuttering assessment and stuttering therapy planning.

cs.AI↗