FocusOPD: Structure-Guided On-Policy Distillation for Diffusion Models

Xinrui Lin; Shumin Wang; Suqi Song; Fan Li; Yanyong Zhang

Home/Archive/arXiv

arXiv · 2610.21250

FocusOPD: Structure-Guided On-Policy Distillation for Diffusion Models

Xinrui Lin·Shumin Wang·Suqi Song·Fan Li·Yanyong Zhang

View original source ↗Read source document ↗Export metadata ↓

Abstract

Multi-teacher on-policy distillation enables rectified-flow models with diffusion-transformer architectures to combine task-specific capabilities within a single student. However, matching model outputs does not guarantee that their internal representations are aligned, and it remains unclear where and when additional feature supervision is most useful. We analyze the teacher-student feature mismatch across Transformer blocks and denoising timesteps, and find that the mismatch is concentrated in deeper Transformer blocks during early denoising and persists after OPD distillation, indicating ineffective distillation. Motivated by this observation, we introduce FocusOPD, which augments the base OPD objective with token-wise directional feature alignment restricted to deep layers and early denoising steps. Across two rectified-flow backbones and multiple OPD objectives, FocusOPD improves capability transfer in compositional generation, text rendering, and preference-based evaluation. On the primary benchmark, it improves over DiffusionOPD from 0.953 to 0.967 on GenEval, from 23.797 to 23.944 on PickScore, and from 6.052 to 6.199 on LAION-Aesthetic. Ablations show that selective deep and early supervision is more effective and efficient than alignment over the full network and trajectory.

Explore related subjects

cs.CV

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Xinrui Lin, Shumin Wang, Suqi Song, Fan Li, Yanyong Zhang. 2026-10-15. FocusOPD: Structure-Guided On-Policy Distillation for Diffusion Models. https://arxiv.org/abs/2610.21250

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Text-to-Image Synthesis: A Decade Survey

Text-to-image synthesis (T2I) focuses on generating high-quality images from textual descriptions In this survey, we review 490 papers on T2I spanning the past decade, with representative advances covered through 2026. We start by briefly introducing how GANs, autoregressive models, and diffusion models have been used for image generation, and we additionally examine the shared components whose evolution has driven recent progress. Building on this foundation, we discuss the development of these models for T2I and organize cutting-edge research by problem rather than by chronology alone, covering controllability, personalized and identity-preserving generation, instruction-based image editing, preference alignment and post-training with human and AI feedback, efficiency, safety, and consistency in content and spatial relationships. We further review unified multimodal understanding-and-generation models, summarize the datasets and evaluation metrics commonly used in T2I research, and provide a comparative discussion of the three paradigms together with their respective trade-offs. Finally, we discuss the potential applications of T2I within AIGC, along with the challenges and forward-looking research opportunities in this field.

cs.CV↗

arXiv2026

Towards Accurate State Estimation: Motion Dynamics Kalman Filter for 3D Multi-Object Tracking

Robust 3D state estimation in multi-object tracking is crucial for object localization in self-driving vehicles. Misalignment between the Kalman filter (KF) motion model and an object's motion can cause state drift, particularly in distant and occlusion cases. The static motion model assumption in the conventional KF is widely used in 3D MOT due to its low latency; however, its estimation accuracy degrades as the object's motion disaligns with the chosen model due to traffic. Although recent work introduces a multimodel KF that runs multiple motion models simultaneously to improve state robustness, it incurs significant computational overhead, so many studies still favor the constant-motion KF to maintain efficiency. This work addresses this trade-off by introducing a motion dynamics Kalman filter (MD-KF) that accounts for motion variation through temporal modeling of object motion and adaptively scales the contribution of the motion components in the motion model to correct state-prediction drift caused by this variation, while maintaining computational efficiency by retaining a single-model structure. Evaluations on KITTI and the Waymo Open Dataset, complemented by vehicle experiments, show that MD-KF achieves tracking performance comparable to multimodel KF with an average 1.72x speedup, while outperforming it in handling highly distant and prolonged occluded objects.

cs.CV↗

arXiv2026

RUSplatting: Robust 3D Gaussian Splatting for Sparse-View Underwater Scene Reconstruction

Reconstructing high-fidelity underwater scenes remains a challenging task due to light absorption, scattering, and limited visibility inherent in aquatic environments. This paper presents an enhanced Gaussian Splatting-based framework that improves both the visual quality and geometric accuracy of deep underwater rendering. We propose decoupled learning for RGB channels, guided by the physics of underwater attenuation, to enable more accurate colour restoration. To address sparse-view limitations and improve view consistency, we introduce a frame interpolation strategy with a novel adaptive weighting scheme. Additionally, we introduce a new loss function aimed at reducing noise while preserving edges, which is essential for deep-sea content. We also release a newly collected dataset, Submerged3D, captured specifically in deep-sea environments. Experimental results demonstrate that our framework consistently outperforms state-of-the-art methods with PSNR gains up to 1.90dB, delivering superior perceptual quality and robustness, and offering promising directions for marine robotics and underwater visual analytics. The code of RUSplatting is available at https://github.com/theflash987/RUSplatting and the dataset Submerged3D can be downloaded at https://zenodo.org/records/15482420.

cs.CV↗