arXiv subjects

Jun Huang

Publications and source records attributed to Jun Huang.

At least 19 recordsLinked to original sources

arXiv2026

Beyond Token Representations: Explicit Visual Object Grounding for Video Reasoning Segmentation

Video Reasoning Segmentation (VRS) aims to infer and segment target objects specified by reasoning queries in videos. Existing methods typically first infer the target and generate a few segmentation tokens for it, which are then fed into a mask decoder for mask prediction. However, such segmentation tokens tend to be both semantically monotonous and spatially ambiguous, making them largely insufficient to represent the target to guide mask prediction. In this paper, we propose an explicit visual object grounding framework for VRS. Specifically, we first propose a spatial-aware prompt generation and refinement scheme, in which we query a multimodal large language model to generate explicit spatial prompts (i.e., bounding box and point set) for the target and then refine them through multi-step dialogue. Furthermore, we introduce a query diversification and alignment module to generate auxiliary queries that describe the same target from different perspectives. We then enforce cross-query consistency between their predicted spatial prompts to alleviate the training bias caused by query limitations. Finally, we select several reliable frames using a prompt-based keyframe selection strategy to support mask decoding and propagation. Extensive experiments on eight datasets demonstrate that our method significantly outperforms state-of-the-art methods. The code is available at https://github.com/viscom-tongji/EVOG.

cs.CV↗

arXiv2026

Pod-wise Multimodal Federated Learning for Heterogeneous IoT

Federated learning (FL) is well suited to Internet of Things (IoT) sensing. However, devices carry different sensors and come online in periodic patterns, and as such, the sensors among the participants change from round to round. Under naive averaging, devices without a sensor overwrite its parameters, and a single-sensor device never learns how its sensor relates to the others. Existing methods either redesign local training for mismatched sensors (assuming always-available devices) or select clients without regard to their sensors. We propose Tier-aware Intra-pod Decoupled Encoders (TIDE). Specifically, each sensor has its own encoder and auxiliary head, whose parameters are merged only within the "pod" (i.e., the devices that hold the sensor), while the fusion layer and the task head remain shared by all devices. On three multimodal activity recognition datasets against eight baselines, TIDE reaches 78.06% on ActionSense, 7.63 points above the next method, is the most accurate on all three datasets at half its own communication, and at equal communication leads on ActionSense and is ahead within seed variation on the other two.

cs.NI↗

arXiv2026

AdaGround: Token-Normalized Adaptive Zooming with Verifier-in-the-Loop Refinement for GUI Grounding

GUI grounding on high-resolution screens fails mostly on small targets, and zooming in is a common training-free remedy. However, most existing zooming pipelines do not match the zoom to the target's size, accept the zoomed answer unconditionally and never revisit a wrong answer. To address these issues, we propose AdaGround, a training-free framework with three components: (1) Token-Normalized Adaptive Zooming, which sets the magnification, field of view and zoom depth of each sample from the model's own estimate of the target size; (2) Verifier-in-the-Loop Refinement, which uses one marked-click Yes/No verifier to keep or reject each zoom, choose between zoom chains, trigger re-grounding and, with an existence query, decide when to abstain; and (3) Compositional Structure Grounding, which composes spreadsheet cells from their headers. On Qwen3-VL models, AdaGround surpasses state-of-the-art training-free zooming methods and many trained GUI models, improving accuracy over the backbone by 7.4 to 20.6 points on OSWorld-G and ScreenSpot-Pro, and its gains carry over to held-out UI-Vision. Code is available at https://github.com/bcwang-sjtu/AdaGround

cs.CV↗

arXiv2026

GUI-JEPA: Candidate-Conditioned JEPA Latents as a Fast Action Policy for Mobile GUI Agents

Mobile GUI agents are typically fine-tuned 7B-scale vision-language models (VLMs) that generate each action as text. However, these VLMs decode every action token by token, and even agents that score candidate actions keep a large language model or VLM in the loop, making them slow and memory-hungry. To address these issues, we propose GUI-JEPA, to our knowledge the first decoder-free GUI agent policy built on a Joint-Embedding Predictive Architecture (JEPA), which selects actions in the latent space of a JEPA encoder pretrained on unlabelled GUI screenshots. Its action- and region-conditioned latent predictor turns the screen latent into one latent per candidate action, letting lightweight heads score all candidates against the instruction in a single batched pass, without decoding a token. We further show that the scorer relies on this latent pathway and that the predictor is trained for the decision itself: a randomly initialised predictor does just as well as one pretrained on screen dynamics. On AndroidControl, GUI-JEPA is 2.8-4.0x faster on the same GPU than a Qwen2.5-VL-7B agent we fine-tune with the same inputs, while reaching 86-92% of its in-distribution accuracy with 6% of its parameters. Code is available at https://github.com/bcwang-sjtu/GUI-JEPA

cs.AI↗

arXiv2026

Relational Geometry Forcing for World Action Models

World Action Models (WAMs) have shown strong potential for robot manipulation by jointly learning visual dynamics and executable actions. However, their video and action objectives do not explicitly constrain the 3D geometry that is crucial for precise manipulation. Recently, geometry-aware WAMs introduce depth or 4D prediction branches, whereas representation alignment methods supervise intermediate WAM features with pretrained 3D foundation models. Yet we find that such pointwise alignment benefits the video features but degrades action learning, since dense geometric feature targets overly restrict compact action features. To address this limitation, we propose Relational Geometry Forcing (RGF), which transfers geometric priors from a frozen 3D foundation model by aligning pairwise feature relations instead of pointwise features. Specifically, RGF applies three complementary relational objectives, where intra-frame and cross-view relations transfer scene geometry and multi-view correspondence to the video features, while action-temporal relations couple action dynamics to geometric scene evolution. Applied only during training, RGF requires no explicit geometry branch or extra inference module. Extensive experiments in simulation and the real world demonstrate the effectiveness of Relational Geometry Forcing, showing consistent gains across diverse manipulation scenarios and WAM paradigms.

cs.RO↗

arXiv2026

Initiating Removal from Operational Symptoms in Federated Vehicular Networks

Connected vehicles train a driving model through federated learning (FL) and drive with it, so a faulty contribution often surfaces only as a delayed symptom on the road. Federated unlearning (FU) assumes that a removal request names what to forget. A symptom names no vehicle or round, and aggregation has already spread its cause across the fleet. We propose \underline{A}ggregation-\underline{T}ime \underline{O}perational-evidence \underline{N}eutralization of \underline{E}ffects (\method{}), which makes unlearning a term of the FL aggregation rule. \method{} projects everyday driving errors and reported symptoms onto a subspace spanned by recent global increments. It propagates the surviving effect of every contribution with curvature estimated from these increments and scores all contributions in one backward pass. A two-tier correction suppresses suspected effects in every round and eliminates confirmed ones after a one-round probe. We prove that backward scoring matches forward scoring and that the elimination is exact when training is flat outside the subspace. On three driving datasets, \method{} leaves 43\% to 61\% less residual harm than the strongest baseline in eight of nine settings while staying within 0.36 points of full retraining in utility.

cs.LG↗

arXiv2026

Structuring and Merging Deletion Demands on Crowdsourced Channel Knowledge Maps

A base station can build a channel knowledge map (CKM) from the uplink pilots of its terminals and sweep only the beams that the map ranks highest. A fault at a terminal or at the base station corrupts the measurements taken while it lasts, and the map must then forget them. Removing the whole terminal discards its good measurements, and the raw reports needed to remove samples are gone. We define the structured deletion unit (SDU), which names the terminals, rounds, grids, and bands to forget. We propose \underline{S}tructured \underline{W}iping with \underline{E}nvelope-bounded \underline{E}xecution \underline{P}artitioning (\method{}), which executes an SDU by clearing blocks of an aggregated ledger and re-sounding the affected grids. \method{} merges pending demands so that one campaign serves several, and it bounds what the smallest SDU containing a group clears beyond the demands. We prove that a merged execution leaves no named measurement in the map, that demands differing only in their intervals are best merged in contiguous runs, and that the greedy merge is within a factor $1+\ln Q$ of the best grouping over its candidates. In simulation and on a USRP testbed, \method{} clears at most 5.1\% of the good measurements, against up to 86.8\% when whole terminals are removed or the map is rolled back to an earlier block.

cs.NI↗

arXiv2026

What Matters for World Modeling in Vision-Language-Action Models?

Integrating world models into Vision-Language-Action models (VLAs) has emerged as a promising direction, enabling policies to anticipate future dynamics beyond reactive observation-to-action mapping. However, existing approaches vary simultaneously in their world representations, integration paradigms, and policy backbones, making it difficult to determine what truly matters for effective world model integration. To disentangle these factors, we categorize existing efforts into four representative paradigms, namely Implicit Future Alignment, Explicit Future Query, MoT Future Expert, and External World Predictor. We instantiate each within a unified framework and systematically evaluate these paradigms with diverse world representations and policy backbones across in-domain, out-of-domain, long-horizon, dynamic, few-shot, and real-world settings. Our study yields four main findings: 1) world representation effectiveness is paradigm-dependent, whereas 2D semantic representations offer robust, cross-paradigm gains; 2) no single paradigm is universally optimal, yet the decoupled MoT Future Expert offers competitive overall performance, particularly in OOD robustness; 3) world modeling consistently benefits policies built on standard VLMs, while decoupled designs exhibit greater compatibility with pretrained VLAs; and 4) world modeling improves performance over action-only baselines across multiple settings, with notable gains on long-horizon and dynamic tasks. Project page and video demonstrations are available at https://dylanmqs.github.io/WM-in-VLAs/.

cs.RO↗

arXiv2026

When Unlearning Fails: Reliable Data Deletion under Post-Training in Agent Networks

Embedded AI agents on edge devices can learn as a federation and keep improving after deployment. Each agent collects new trajectories with the current policy, and the edge server aggregates them in later rounds. In such self-improving agent networks, deleting data is harder than a one-time model repair. The data to be deleted may have already shaped later retained trajectories, so retraining or model-side unlearning leaves an influence echo that returns as the network keeps learning. We propose Muting Unlearned Trajectories' Echoes (MUTE) for reliable deletion over edge networks. MUTE traces the echo from a lightweight ledger at the edge server, without moving raw trajectories off the devices. It erases the current residue with a forget-retain update, contains high-influence retained trajectories, and audits later behavior to schedule further erasure under the uplink budget and the response deadline. On LIBERO, MUTE keeps the regenerated share of forgotten behavior at or below 0.401 for every request type, against at least 0.466 for five recent methods.

cs.NI↗

arXiv2026

Tracing Knowledge Lineage for Federated Unlearning under Mutual Distillation

An edge server and nearby devices can train language models of different sizes together by mutual distillation. Federated unlearning (FU) and unlearning for large language models (LLMs) execute a removal request on one model. Under mutual distillation, however, the effect of the removed data has been copied into many students, and their uploads teach it back to the corrected teacher. We propose Holder-level Erasure via Replication-weighted Delta-closure (HERD), which decides which holders must be corrected. HERD records a knowledge lineage graph from the transfers that the server already performs and weights each edge by its transfer type. It computes how many times the residual at each holder will be copied and selects the cheapest set of holders whose correction bounds the cumulative residual after the request. A transfer guard and a rejoin rule cover the holders left uncorrected or offline. We prove that partial correction suffices only when the copy number of the lineage graph is below one, that averaging alone never reaches one, and that the correction set remains valid under link outages. On TOFU, HERD leaves 18% to 52% less cumulative residual than the strongest baseline at 6% to 12% of its correction cost.

cs.NI↗

arXiv2026

Federated Unlearning of Quantum Data via Classical Shadows

Quantum federated learning trains a quantum neural network (QNN) across nodes that hold quantum data, such as the states of quantum sensors. To forget data, unlearning methods for quantum models replay them, and federated unlearning from stored updates removes whole nodes only. Quantum data cannot be replayed, since a state is consumed when measured and cannot be cloned. We propose Shadow-based Data Erasure (SHADE), in which each node records a classical shadow of every training state, a few randomized single-qubit measurements taken at training time. The gradient and curvature of a QNN on a state are expectations of observables, so SHADE estimates them for the forget data from the shadows at any later parameters and removes their effect by a federated Newton correction. Only classical estimates cross the network. We prove that the shadow size needed for a target unlearning error grows exponentially in the size of the backward light cone of the measured observable and not in the number of qubits. In simulation with 50 nodes and on edge boards, SHADE matches the forget accuracy of retraining within 0.5 points, without replay and in under 0.2 s.

quant-ph↗

arXiv2026

SparKV: Overhead-Aware KV Cache Loading for Efficient On-Device LLM Inference

Efficient inference for on-device Large Language Models (LLMs) remains challenging due to limited hardware resources and the high cost of the prefill stage, which processes the full input context to construct Key-Value (KV) caches. We present SparKV, an adaptive KV loading framework that combines cloud-based KV streaming with on-device computation. SparKV models the cost of individual KV chunks and decides whether each chunk should be streamed or computed locally, while overlapping the two execution paths to reduce latency. To handle fluctuations in wireless connectivity and edge resource availability, SparKV further refines offline-generated schedules at runtime to rebalance communication and computation costs. Experiments across diverse datasets, LLMs, and edge devices show that SparKV reduces Time-to-First-Token by 1.3$x-5.1x with negligible impact on response quality, while lowering per-request energy consumption by 1.5x to 3.3x, demonstrating its robustness and practicality for real-world on-device deployment.

cs.NI↗

arXiv2026

Hardware-Software Co-Design for Energy-Efficient Neural Recording in Wireless Implantable BCIs

Implantable Brain-Computer Interfaces (iBCIs) are increasingly pivotal in clinical and daily applications. However, wireless iBCIs face severe constraints in power consumption and data throughput. To mitigate these bottlenecks, we propose a wireless iBCI headstage featuring adaptive ADC sampling and spike detection. Distinguishing our design from traditional application-layer compression, we employ a server-driven architecture that achieves source-level efficiency. Specifically, the server learns an optimal, electrode-specific sample rate vector to dynamically reconfigure the ADC hardware. This strategy reduces data volume directly at the acquisition layer (ADC and amplifier) rather than relying on computationally intensive post-digitization processing. Extensive experiments across diverse subjects and arrays demonstrate a power reduction of up to 40 mW and a 3.2x decrease in FPGA resource utilization, all while maintaining or exceeding decoding accuracy in both motor and visual tasks. This design offers a highly practical solution for long-term in-vivo recording.

cs.NI↗

arXiv2026

ViDAL: A Visual Dynamics-Grounded Action Latent Space for Vision-Language-Action Models

Vision-Language-Action (VLA) models have become a central paradigm for robot policy learning, which predict actions in three forms: raw action chunks, discrete action tokens, or continuous action latents. However, existing action representations primarily model action trajectories, with limited consideration of the visual dynamics induced by these actions. We introduce ViDAL, a Visual Dynamics-grounded Action Latent Space that anchors continuous action latents in the future visual dynamics of the scene. Specifically, ViDAL learns action latent space by training an Action Variational Autoencoder (Action VAE) to reconstruct action chunks while aligning its latent with future scene dynamics. When integrated into downstream robot policies, the proposed Action VAE serves as a plug-in action interface compatible with multiple VLA architectures and enables optional future-video prediction as an additional capability. Empirically, ViDAL outperforms competitive baselines on LIBERO with 98.1% average success, improves a multi-task $π_{0.5}$ policy on RoboTwin 2.0 from 54.3% to 65.5% (Clean) and from 33.2% to 43.1% (Random) success rates over 50 dual-arm tasks, and yields 20.0% and 23.4% absolute success-rate gains on real-world single-arm Franka and dual-arm ARX robot platforms.

cs.RO↗

arXiv2026

Not All Experience Belongs in the Weights: Component Routing for Self-Improving GUI Agents

Self-improving GUI agents keep the trajectories they produce and return them to the agent, by fine-tuning or by retrieval into the prompt, and studies that compare the two destinations disagree. We attribute this to the unit of experience: a trajectory bundles items with different properties, so a conclusion about the bundle depends on its mix. To address this, (i) we introduce component routing, which splits the experience into locators, procedures, state facts and lessons and sends each component to the context or to the weights, compared on the same items across three backbone families, two environments and three seeds. One pool has two destinations: locators and lessons win in the weights, procedures and state facts in the context. (ii) We fit a rule in two properties measured before any training, recurrence and state-conditionality; it recovers the destination of a held-out backbone family in 24 of 24 cells, two interventions move a component toward the boundary, and routing by the rule beats every whole-trajectory baseline and, by +3.5 points on average, the better single destination of each backbone. (iii) We identify how training and producer-consumer differences change the value of the two destinations: note readout decreases after the same component is written into the weights, most for the items that recur most, context gains increase with the information gap, and weights gains decrease with the policy gap. Code and data will be released.

cs.AI↗

arXiv2026

Improving OCR Faithfulness via Gated and Attenuated On-Policy Distillation

Vision-language models may rewrite anomalous text in images into linguistically plausible expressions, compromising OCR transcription faithfulness. Sequence-level task rewards and local teacher guidance are complementary, but guidance from the same teacher may not remain equally effective as the student improves. Offline analysis shows that supervision from a fixed teacher becomes progressively less favorable as the student improves, both across training checkpoints and across response groups with different task rewards. Motivated by this observation, we introduce GAD-RL, which adaptively regulates teacher supervision during joint post-training according to the student's current task performance and local distributions. A frozen teacher conditions on reference transcriptions and student-generated prefixes. GAD-RL disables distillation for response groups containing an output with task reward at least 0.95 and continuously attenuates distillation strength as group-mean reward increases. It also weights forward KL by the student's probability of the teacher's Top-1 token, moderating local auxiliary updates when student support for that candidate is low. On Qwen3.5-2B, GAD-RL achieves 59.92% Micro Recall on CHAOS-Bench, surpassing GRPO and GRPO+OPD (fixed-weight) by 8.45 and 4.43 percentage points, respectively, while achieving an Overall score of 91.18 on OmniDocBench v1.6.

cs.AI↗

arXiv2026

DART: Distillation-Aware Reparameterization for Training-Free LoRA Reuse in Few-Step Video Diffusion Models

Few-step distillation reduces the inference cost of image-to-video generation, but directly reusing LoRA adapters trained for long denoising trajectories can weaken their intended effects and degrade video quality. We observe that adapters with similar measured static parameter geometry can behave differently under the shortened target schedule, motivating response-aware transfer. We propose DART, a training-free reparameterization method that transports source LoRAs into aligned coordinates through a low-rank distillation bridge. Paired forward evaluations measure channel-level incremental responses under the target schedule to fit coefficients that calibrate response direction, source-relative magnitude, and timestep allocation. Coordinate transport establishes update directions, while calibration adapts their contributions, requiring neither source training videos nor backpropagation. Experiments across multiple distilled I2V models demonstrate improved generation quality and aggregate functional retention over direct reuse. Further analyses show that coordinate transport complements response calibration, with adapter-level benefits encompassing both functional preservation and reduced negative transfer.

cs.AI↗

arXiv2026

REBASE: Device-Cloud Experience Coherence for GUI Agents Across App Updates

A graphical user interface (GUI) agent that ships on a phone runs a small model and reuses experience: action paths recorded on earlier runs and cached from the cloud. When an app updates, part of this experience becomes silently wrong. On nine real app version pairs, an agent carrying experience recorded on the old version succeeds 10.1 points less often than one carrying none, and it does not notice the mismatch until 3 steps after acting on it. We propose REBASE, a protocol that keeps the device copy and the cloud copy of the experience coherent across updates. When a version changes, the cloud replays its copy on the new version and the device verifies each recorded step before executing it; when a step still fails, the device sends the cloud evidence in increasing size, starting from an accessibility-subtree diff, until the cloud re-derives an executable patch, keyed by version so that one repair serves the whole fleet. On two Jetson devices, REBASE restores the success rate of stale experience to that of fresh experience recorded on the new version within one episode, at 708x fewer bytes and 47.3x fewer cloud calls than uploading screenshots to the cloud, and 2.5x less device energy than running without experience.

cs.DC↗