arXiv subjects

Ning Zhang

Publications and source records attributed to Ning Zhang.

At least 91 records · Page 5Linked to original sources

arXiv2026

AgentSys: Secure and Dynamic LLM Agents Through Explicit Hierarchical Memory Management

Indirect prompt injection threatens LLM agents by embedding malicious instructions in external content, enabling unauthorized actions and data theft. LLM agents maintain working memory through their context window, which stores interaction history for decision-making. Conventional agents indiscriminately accumulate all tool outputs and reasoning traces in this memory, creating two critical vulnerabilities: (1) injected instructions persist throughout the workflow, granting attackers multiple opportunities to manipulate behavior, and (2) verbose, non-essential content degrades decision-making capabilities. Existing defenses treat bloated memory as given and focus on remaining resilient, rather than reducing unnecessary accumulation to prevent the attack. We present AgentSys, a framework that defends against indirect prompt injection through explicit memory management. Inspired by process memory isolation in operating systems, AgentSys organizes agents hierarchically: a main agent spawns worker agents for tool calls, each running in an isolated context and able to spawn nested workers for subtasks. External data and subtask traces never enter the main agent's memory; only schema-validated return values can cross boundaries through deterministic JSON parsing. Ablations show isolation alone cuts attack success to 2.19%, and adding a validator/sanitizer further improves defense with event-triggered checks whose overhead scales with operations rather than context length. On AgentDojo and ASB, AgentSys achieves 0.78% and 4.25% attack success while slightly improving benign utility over undefended baselines. It remains robust to adaptive attackers and across multiple foundation models, showing that explicit memory management enables secure, dynamic LLM agent architectures. Our code is available at: https://github.com/ruoyaow/agentsys-memory.

cs.CR↗

arXiv2026

DiMo: Discrete Diffusion Modeling for Motion Generation and Understanding

Prior masked modeling motion generation methods predominantly study text-to-motion. We present DiMo, a discrete diffusion-style framework, which extends masked modeling to bidirectional text--motion understanding and generation. Unlike GPT-style autoregressive approaches that tokenize motion and decode sequentially, DiMo performs iterative masked token refinement, unifying Text-to-Motion (T2M), Motion-to-Text (M2T), and text-free Motion-to-Motion (M2M) within a single model. This decoding paradigm naturally enables a quality-latency trade-off at inference via the number of refinement steps. We further improve motion token fidelity with residual vector quantization (RVQ) and enhance alignment and controllability with Group Relative Policy Optimization (GRPO). Experiments on HumanML3D and KIT-ML show strong motion quality and competitive bidirectional understanding under a unified framework. In addition, we demonstrate model ability in text-free motion completion, text-guided motion prediction and motion caption correction without architectural change. Additional qualitative results are available on our project page: https://animotionlab.github.io/DiMo/.

cs.CV↗

arXiv2026

NECromancer: Breathing Life into Skeletons via BVH Animation

Motion tokenization is a key component of generalizable motion models, yet most existing approaches are restricted to species-specific skeletons, limiting their applicability across diverse morphologies. We propose NECromancer (NEC), a universal motion tokenizer that operates directly on arbitrary BVH skeletons. NEC consists of three components: (1) an Ontology-aware Skeletal Graph Encoder (OwO) that encodes structural priors from BVH files, including joint semantics, rest-pose offsets, and skeletal topology, into skeletal embeddings; (2) a Topology-Agnostic Tokenizer (TAT) that compresses motion sequences into a universal, topology-invariant discrete representation; and (3) the Unified BVH Universe (UvU), a large-scale dataset aggregating BVH motions across heterogeneous skeletons. Experiments show that NEC achieves high-fidelity reconstruction under substantial compression and effectively disentangles motion from skeletal structure. The resulting token space supports cross-species motion transfer, composition, denoising, generation with token-based models, and text-motion retrieval, establishing a unified framework for motion analysis and synthesis across diverse morphologies. Demo page: https://animotionlab.github.io/NECromancer/

cs.CV↗

arXiv2026

ShapeGaussian: High-Fidelity 4D Human Reconstruction in Monocular Videos via Vision Priors

We introduce ShapeGaussian, a high-fidelity, template-free method for 4D human reconstruction from casual monocular videos. Generic reconstruction methods lacking robust vision priors, such as 4DGS, struggle to capture high-deformation human motion without multi-view cues. While template-based approaches, primarily relying on SMPL, such as HUGS, can produce photorealistic results, they are highly susceptible to errors in human pose estimation, often leading to unrealistic artifacts. In contrast, ShapeGaussian effectively integrates template-free vision priors to achieve both high-fidelity and robust scene reconstructions. Our method follows a two-step pipeline: first, we learn a coarse, deformable geometry using pretrained models that estimate data-driven priors, providing a foundation for reconstruction. Then, we refine this geometry using a neural deformation model to capture fine-grained dynamic details. By leveraging 2D vision priors, we mitigate artifacts from erroneous pose estimation in template-based methods and employ multiple reference frames to resolve the invisibility issue of 2D keypoints in a template-free manner. Extensive experiments demonstrate that ShapeGaussian surpasses template-based methods in reconstruction accuracy, achieving superior visual quality and robustness across diverse human motions in casual monocular videos.

cs.CV↗

arXiv2026

Unified MPI Parallelization of Wave Function Methods: iCIPT2 as a Showcase

The integration of quantum chemical methods with high-performance computing is indispensable for handling large systems with modest accuracy or even small systems but with high accuracy. Continuing with the unified implementation of non-relativistic and relativistic wave functions methods within the MetaWave platform (J. Phys. Chem. A. 2025, 129, 5170), we present here a unified MPI parallelization of the methods by abstracting ever computational step of a method as a dynamically-scheduled loop via ghost process, followed by a global reduction of local results from each node. The algorithmic abstraction enables the use of a single MPI template in various steps of different methods. Taking iCIPT2 [J. Chem. Theory Comput. 2021, 17, 949] as a showcase, the parallel efficiencies achieve 94% and 89% on 16 nodes (1024 cores) for the perturbation and whole calculations, respectively. Further combined with an improved algorithm for the matrix-vector product in the matrix diagonalization and an orbital-configuration-based semi-stochastic estimator for the perturbation correction, this renders large active space calculations possible, so as to obtain benchmarks for the automerization of cyclobutadiene, ground state energy of benzene and potential energy profile of ozone. It is also shown that the error of iCIPT2 follows a power law with respect to the number of configuration state functions.

physics.chem-ph↗

arXiv2026

Ab Initio Many Body Quantum Embedding and Local Correlation in Crystalline Materials using Interpolative Separable Density Fitting

We present an efficient implementation of ab initio many-body quantum embedding and local correlation methods for infinite periodic systems through translational symmetry adapted interpolative separable density fitting, an approach which reduces the scaling of the calculations to only linear with the number of k-points. Employing this methodology, we compute correlated ground-state coupled cluster energies within density matrix embedding and local natural orbital correlation frameworks for both weakly and strongly correlated solids, using up to 1000 k-points. By extrapolating the local correlation domains and k-point sampling we further obtain estimates of the full coupled cluster with singles, doubles, and perturbative triples ground-state energies in the thermodynamic limit.

cond-mat.str-el↗

arXiv2026

ReasAlign: Reasoning Enhanced Safety Alignment against Prompt Injection Attack

Large Language Models (LLMs) have enabled the development of powerful agentic systems capable of automating complex workflows across various fields. However, these systems are highly vulnerable to indirect prompt injection attacks, where malicious instructions embedded in external data can hijack agent behavior. In this work, we present ReasAlign, a model-level solution to improve safety alignment against indirect prompt injection attacks. The core idea of ReasAlign is to incorporate structured reasoning steps to analyze user queries, detect conflicting instructions, and preserve the continuity of the user's intended tasks to defend against indirect injection attacks. To further ensure reasoning logic and accuracy, we introduce a test-time scaling mechanism with a preference-optimized judge model that scores reasoning steps and selects the best trajectory. Comprehensive evaluations across various benchmarks show that ReasAlign maintains utility comparable to an undefended model while consistently outperforming Meta SecAlign, the strongest prior guardrail. On the representative open-ended CyberSecEval2 benchmark, which includes multiple prompt-injected tasks, ReasAlign achieves 94.6% utility and only 3.6% ASR, far surpassing the state-of-the-art defensive model of Meta SecAlign (56.4% utility and 74.4% ASR). These results demonstrate that ReasAlign achieves the best trade-off between security and utility, establishing a robust and practical defense against prompt injection attacks in real-world agentic systems. Our code and experimental results could be found at https://github.com/leolee99/ReasAlign.

cs.CR↗

arXiv2026

Halpern Acceleration of the Inexact Proximal Point Method of Rockafellar

This paper investigates a Halpern acceleration of the inexact proximal point method for solving maximal monotone inclusion problems in Hilbert spaces. The proposed Halpern inexact proximal point method (HiPPM) is shown to be globally convergent, and a unified framework is developed to analyze its worst-case convergence behavior. Under mild conditions on the inexactness tolerances, HiPPM achieves an $\mathcal{O}(1/k^{2})$ convergence rate in terms of the squared fixed-point residual. Moreover, under additional well-studied regularity conditions, the method attains a fast linear convergence rate. Building on this framework, we further extend the Halpern acceleration to the inexact augmented Lagrangian method for constrained convex optimization. In the spirit of Rockafellar's classical results, the resulting accelerated inexact augmented Lagrangian method inherits the convergence rate and iteration complexity guarantees of HiPPM. Numerical experiments are provided to support the theoretical findings.

math.OC↗

arXiv2025

SWiT-4D: Sliding-Window Transformer for Lossless and Parameter-Free Temporal 4D Generation

Despite significant progress in 4D content generation, the conversion of monocular videos into high-quality animated 3D assets with explicit 4D meshes remains considerably challenging. The scarcity of large-scale, naturally captured 4D mesh datasets further limits the ability to train generalizable video-to-4D models from scratch in a purely data-driven manner. Meanwhile, advances in image-to-3D generation, supported by extensive datasets, offer powerful prior models that can be leveraged. To better utilize these priors while minimizing reliance on 4D supervision, we introduce SWiT-4D, a Sliding-Window Transformer for lossless, parameter-free temporal 4D mesh generation. SWiT-4D integrates seamlessly with any Diffusion Transformer (DiT)-based image-to-3D generator, adding spatial-temporal modeling across video frames while preserving the original single-image forward process, enabling 4D mesh reconstruction from videos of arbitrary length. To recover global translation, we further introduce an optimization-based trajectory module tailored for static-camera monocular videos. SWiT-4D demonstrates strong data efficiency: with only a single short (<10s) video for fine-tuning, it achieves high-fidelity geometry and stable temporal consistency, indicating practical deployability under extremely limited 4D supervision. Comprehensive experiments on both in-domain zoo-test sets and challenging out-of-domain benchmarks (C4D, Objaverse, and in-the-wild videos) show that SWiT-4D consistently outperforms existing baselines in temporal smoothness. Project page: https://animotionlab.github.io/SWIT4D/

cs.CV↗

arXiv2025

A Model and Data Dual-driven Approach for Multitargets Detection under Mainlobe Jamming

In modern radar systems, target detection and parameter estimation face significant challenges when confronted with mainlobe jamming. This paper presents a Diffusion-based Model and Data Dual-driven (DMDD) approach to estimate and detect multitargets and suppress structured jamming. In DMDD, the jamming prior is modeled through a score-based diffusion process with its score learned from the pure jamming data, enabling posterior sampling without requiring detailed knowledge of jamming. Meanwhile, the target signal is usually sparse in the range space, which can be modeled via a sparse Bayesian learning (SBL) framework, and hyperparameter is updated through the expectation-maximization (EM) algorithm. A single diffusion process is constructed for the jamming, while the state of targets are estimated through direct posterior inference, enhancing computational efficiency. The noise variance is also estimated through EM algorithm. Numerical experiments demonstrate the effectiveness of the proposed method in structured jamming scenarios. The proposed DMDD algorithm achieves superior target detection performance, compared with existing methods.

eess.SP↗

arXiv2025

DarkMind: Latent Chain-of-Thought Backdoor in Customized LLMs

With the rapid rise of personalized AI, customized large language models (LLMs) equipped with Chain of Thought (COT) reasoning now power millions of AI agents. However, their complex reasoning processes introduce new and largely unexplored security vulnerabilities. We present DarkMind, a novel latent reasoning level backdoor attack that targets customized LLMs by manipulating internal COT steps without altering user queries. Unlike prior prompt based attacks, DarkMind activates covertly within the reasoning chain via latent triggers, enabling adversarial behaviors without modifying input prompts or requiring access to model parameters. To achieve stealth and reliability, we propose dual trigger types instant and retrospective and integrate them within a unified embedding template that governs trigger dependent activation, employ a stealth optimization algorithm to minimize semantic drift, and introduce an automated conversation starter for covert activation across domains. Comprehensive experiments on eight reasoning datasets spanning arithmetic, commonsense, and symbolic domains, using five LLMs, demonstrate that DarkMind consistently achieves high attack success rates. We further investigate defense strategies to mitigate these risks and reveal that reasoning level backdoors represent a significant yet underexplored threat, underscoring the need for robust, reasoning aware security mechanisms.

cs.CR↗

arXiv2025

Towards Automating Data Access Permissions in AI Agents

As AI agents attempt to autonomously act on users' behalf, they raise transparency and control issues. We argue that permission-based access control is indispensable in providing meaningful control to the users, but conventional permission models are inadequate for the automated agentic execution paradigm. We therefore propose automated permission management for AI agents. Our key idea is to conduct a user study to identify the factors influencing users' permission decisions and to encode these factors into an ML-based permission management assistant capable of predicting users' future decisions. We find that participants' permission decisions are influenced by communication context but importantly individual preferences tend to remain consistent within contexts, and align with those of other participants. Leveraging these insights, we develop a permission prediction model achieving 85.1% accuracy overall and 94.4% for high-confidence predictions. We find that even without using permission history, our model achieves an accuracy of 66.9%, and a slight increase of training samples (i.e., 1-4) can substantially increase the accuracy by 10.8%.

cs.CR↗

arXiv2025

EcoLoRA: Communication-Efficient Federated Fine-Tuning of Large Language Models

To address data locality and privacy restrictions, Federated Learning (FL) has recently been adopted to fine-tune large language models (LLMs), enabling improved performance on various downstream tasks without requiring aggregated data. However, the repeated exchange of model updates in FL can result in prohibitively high communication costs, hindering the distributed learning process. To address this challenge, we propose EcoLoRA, a novel communication-efficient federated fine-tuning framework for LLMs. Leveraging the modular structure, we propose a round-robin segment sharing scheme, where each client uploads only a complementary LoRA segment per round to reduce network bandwidth. It is further combined with adaptive sparsification methods tailored to LoRA's training dynamics and lossless encoding techniques. We conduct extensive evaluations on both question-answering and value-alignment tasks across multiple datasets and models. The results show that EcoLoRA significantly reduces communication overhead without compromising performance. For instance, it reduces communication time by up to 79% and total training time by up to 65%.

cs.DC↗

arXiv2025

Compressing Multi-Task Model for Autonomous Driving via Pruning and Knowledge Distillation

Autonomous driving systems rely on panoptic perception to jointly handle object detection, drivable area segmentation, and lane line segmentation. Although multi-task learning is an effective way to integrate these tasks, its increasing model parameters and complexity make deployment on on-board devices difficult. To address this challenge, we propose a multi-task model compression framework that combines task-aware safe pruning with feature-level knowledge distillation. Our safe pruning strategy integrates Taylor-based channel importance with gradient conflict penalty to keep important channels while removing redundant and conflicting channels. To mitigate performance degradation after pruning, we further design a task head-agnostic distillation method that transfers intermediate backbone and encoder features from a teacher to a student model as guidance. Experiments on the BDD100K dataset demonstrate that our compressed model achieves a 32.7% reduction in parameters while segmentation performance shows negligible accuracy loss and only a minor decrease in detection (-1.2% for Recall and -1.8% for mAP50) compared to the teacher. The compressed model still runs at 32.7 FPS in real-time. These results show that combining pruning and knowledge distillation provides an effective compression solution for multi-task panoptic perception.

cs.CV↗

arXiv2025

On the Stability of Graph Convolutional Neural Networks: A Probabilistic Perspective

Graph convolutional neural networks (GCNNs) have emerged as powerful tools for analyzing graph-structured data, achieving remarkable success across diverse applications. However, the theoretical understanding of the stability of these models, i.e., their sensitivity to small changes in the graph structure, remains in rather limited settings, hampering the development and deployment of robust and trustworthy models in practice. To fill this gap, we study how perturbations in the graph topology affect GCNN outputs and propose a novel formulation for analyzing model stability. Unlike prior studies that focus only on worst-case perturbations, our distribution-aware formulation characterizes output perturbations across a broad range of input data. This way, our framework enables, for the first time, a probabilistic perspective on the interplay between the statistical properties of the node data and perturbations in the graph topology. We conduct extensive experiments to validate our theoretical findings and demonstrate their benefits over existing baselines, in terms of both representation stability and adversarial attacks on downstream tasks. Our results demonstrate the practical significance of the proposed formulation and highlight the importance of incorporating data distribution into stability analysis.

cs.LG↗

arXiv2025

Data-driven Communication and Control Design for Distributed Frequency Regulation with Black-box Inverters

The increasing penetration of inverter-based resources into the power grid, with often only black-box models available, challenges long-standing frequency control methods. Most recent works take a decentralized approach without online device coordination via communication. This paper considers both dynamic behavior and communication within secondary frequency control on an intermediate timescale. We develop a distributed data-driven approach that utilizes peer-to-peer communication between inverters to avoid the need for a central control center. To enable a trade off between communication network requirements and control performance, we present a framework to guide communication topology design for secondary frequency regulation. Following design of the inter-agent information exchange scheme, we design a controller that is structured according to the communication topology with a closed-loop stability guarantee. Case studies on the IEEE 39-bus system validate the framework and illustrate the trade-off between communication requirements and control performance that is enabled by our approach.

eess.SY↗

arXiv2025

Automated Glaucoma Report Generation via Dual-Attention Semantic Parallel-LSTM and Multimodal Clinical Data Integration

Generative AI for automated glaucoma diagnostic report generation faces two predominant challenges: content redundancy in narrative outputs and inadequate highlighting of pathologically significant features including optic disc cupping, retinal nerve fiber layer defects, and visual field abnormalities. These limitations primarily stem from current multimodal architectures' insufficient capacity to extract discriminative structural-textural patterns from fundus imaging data while maintaining precise semantic alignment with domain-specific terminology in comprehensive clinical reports. To overcome these constraints, we present the Dual-Attention Semantic Parallel-LSTM Network (DA-SPL), an advanced multimodal generation framework that synergistically processes both fundus imaging and supplementary visual inputs. DA-SPL employs an Encoder-Decoder structure augmented with the novel joint dual-attention mechanism in the encoder for cross-modal feature refinement, the parallelized LSTM decoder architecture for enhanced temporal-semantic consistency, and the specialized label enhancement module for accurate disease-relevant term generation. Rigorous evaluation on standard glaucoma datasets demonstrates DA-SPL's consistent superiority over state-of-the-art models across quantitative metrics. DA-SPL exhibits exceptional capability in extracting subtle pathological indicators from multimodal inputs while generating diagnostically precise reports that exhibit strong concordance with clinical expert annotations.

cs.CE↗

arXiv2025

ThermalGen: Style-Disentangled Flow-Based Generative Models for RGB-to-Thermal Image Translation

Paired RGB-thermal data is crucial for visual-thermal sensor fusion and cross-modality tasks, including important applications such as multi-modal image alignment and retrieval. However, the scarcity of synchronized and calibrated RGB-thermal image pairs presents a major obstacle to progress in these areas. To overcome this challenge, RGB-to-Thermal (RGB-T) image translation has emerged as a promising solution, enabling the synthesis of thermal images from abundant RGB datasets for training purposes. In this study, we propose ThermalGen, an adaptive flow-based generative model for RGB-T image translation, incorporating an RGB image conditioning architecture and a style-disentangled mechanism. To support large-scale training, we curated eight public satellite-aerial, aerial, and ground RGB-T paired datasets, and introduced three new large-scale satellite-aerial RGB-T datasets--DJI-day, Bosonplus-day, and Bosonplus-night--captured across diverse times, sensor types, and geographic regions. Extensive evaluations across multiple RGB-T benchmarks demonstrate that ThermalGen achieves comparable or superior translation performance compared to existing GAN-based and diffusion-based methods. To our knowledge, ThermalGen is the first RGB-T image translation model capable of synthesizing thermal images that reflect significant variations in viewpoints, sensor characteristics, and environmental conditions. Project page: http://xjh19971.github.io/ThermalGen

cs.CV↗