Diagnose-to-Generate: Eliminating Structural Distortions in Flow Models via Patch-Level Grounded Alignment

Yifan Li; Lishuai Gao; Tong Zhang; Qilong Huang; Chen Zhao; Zhuoliang Kang; Jie Hu; Yujie Zhong; Fenghe Tang; S Kevin Zhou; Xiaoming Wei

arXiv · 2610.17882

Diagnose-to-Generate: Eliminating Structural Distortions in Flow Models via Patch-Level Grounded Alignment

Yifan Li·Lishuai Gao·Tong Zhang·Qilong Huang·Chen Zhao·Zhuoliang Kang·Jie Hu·Yujie Zhong·Fenghe Tang·S Kevin Zhou·Xiaoming Wei

View original source ↗Read source document ↗Export metadata ↓

Abstract

Flow-based image generators achieve strong semantic synthesis but still produce localized structural distortions, such as malformed hands and limbs. Existing post-training methods typically optimize global scalar rewards, which provide weak spatial credit assignment and may alter well-formed regions while correcting local defects. We present \textit{Diagnose-to-Generate (D2G)}, a grounded alignment framework that separates visual diagnosis from generative optimization. We construct \textbf{Achilles-600K}, a region-annotated dataset for structural anomalies, and train \textbf{ARGUS} through supervised cold-start and Region-Aware Group Relative Policy Optimization to predict anomaly categories and bounding boxes. These diagnoses are converted into confidence-weighted spatial rewards for our \textbf{Patch-level Flow-GRPO}. We further identify that global stochastic exploration can shift semantic regions across group rollouts, making patch-wise relative advantages spatially inconsistent. Our \textbf{Trajectory-Preserving Stochastic Sampling (TPSS)} anchors early denoising with deterministic ODE trajectories and introduces stochasticity only at later steps, thereby limiting intra-group spatial drift while retaining local exploration. Experiments on FLUX.1-dev and SD3.5 show improved structural fidelity with preserved FID and CLIP scores. Consistent gains under HumanRefiner, zero-shot GPT-4.1, and blinded human evaluation further demonstrate that the improvements extend beyond the training reward model.

Explore related subjects

cs.CV

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Yifan Li, Lishuai Gao, Tong Zhang, Qilong Huang, Chen Zhao, Zhuoliang Kang, Jie Hu, Yujie Zhong, Fenghe Tang, S Kevin Zhou, Xiaoming Wei. 2026-10-13. Diagnose-to-Generate: Eliminating Structural Distortions in Flow Models via Patch-Level Grounded Alignment. https://arxiv.org/abs/2610.17882

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Universal Incremental Learning: Mitigating Confusion from Inter- and Intra-task Distribution Randomness

Existing incremental learning (IL) methods rely on strong assumptions: incoming tasks either introduce exclusively new classes or new domains (i.e., Class IL and Domain IL), or increment by a fixed scale in a class- and domain-agnostic manner (i.e., Versatile IL, VIL). These assumptions severely limit their applicability in dynamic, unpredictable real-world scenarios. In this work, we investigate Universal Incremental Learning (UIL), a more realistic setting where the model has no prior knowledge of which new classes or domains will appear across sequential tasks, nor the scale of these increments within each task. This uncertainty hinders the model's ability to confidently acquire knowledge across varying task distributions and to learn uniformly from imbalanced intra-task distributions. Consequently, UIL introduces a more general IL scenario, where the model must cope with confusion stemming from both inter-task and intra-task distribution randomness. To mitigate both sources of confusion, we propose MiCo, a simple yet effective framework for UIL. At the inter-task level, we employ a multi-objective learning scheme to enforce accurate and deterministic predictions, further enhanced by a direction recalibration module that alleviates conflicting gradient updates. At the intra-task level, we introduce a magnitude recalibration module to address asymmetrical optimization caused by imbalanced class distributions. Extensive experiments on four benchmarks demonstrate that MiCo significantly outperforms existing state-of-the-art methods in both the proposed UIL and existing VIL scenarios.

cs.CV↗

arXiv2026

Panoramic Out-of-Distribution Segmentation

Panoramic imaging enables capturing 360° images with an ultra-wide Field-of-View (FoV) for dense omnidirectional perception, which is critical to applications such as autonomous driving and augmented reality. However, current panoramic semantic segmentation methods fail to identify outliers, and pinhole out-of-distribution segmentation (pinhole-OoS) models perform unsatisfactorily in the panoramic domain due to pixel distortions and background clutter. To address these issues, we introduce a new task, Panoramic Out-of-Distribution Segmentation (PanOoS), with the aim of achieving comprehensive and safe scene understanding. Furthermore, we propose the first solution, POS, which adapts to the characteristics of panoramic images through text-guided prompt distribution learning. Specifically, POS integrates a disentanglement strategy designed to materialize the cross-domain generalization capability of CLIP. The proposed Prompt-based Restoration Attention (PRA) optimizes semantic decoding by prompt guidance and self-adaptive correction, while Bilevel Prompt Distribution Learning (BPDL) refines the manifold of per-pixel mask embeddings via semantic prototype supervision. Besides, to compensate for the scarcity of PanOoS datasets, we establish two benchmarks: DenseOoS, which features diverse outliers in complex environments, and QuadOoS, captured by a quadruped robot with a panoramic annular lens system. Extensive experiments demonstrate the effectiveness of POS, with AuPRC increasing and FPR95 decreasing by 34.25 and 21.42 percentage points, respectively, compared with RbA on DenseOoS, outperforming state-of-the-art pinhole-OoS methods. Moreover, POS achieves leading closed-set segmentation capabilities and advances the development of panoramic understanding. Code and datasets will be available at https://github.com/MengfeiD/PanOoS.

cs.CV↗

arXiv2026

Towards Accurate State Estimation: Motion Dynamics Kalman Filter for 3D Multi-Object Tracking

Robust 3D state estimation in multi-object tracking is crucial for object localization in self-driving vehicles. Misalignment between the Kalman filter (KF) motion model and an object's motion can cause state drift, particularly in distant and occlusion cases. The static motion model assumption in the conventional KF is widely used in 3D MOT due to its low latency; however, its estimation accuracy degrades as the object's motion disaligns with the chosen model due to traffic. Although recent work introduces a multimodel KF that runs multiple motion models simultaneously to improve state robustness, it incurs significant computational overhead, so many studies still favor the constant-motion KF to maintain efficiency. This work addresses this trade-off by introducing a motion dynamics Kalman filter (MD-KF) that accounts for motion variation through temporal modeling of object motion and adaptively scales the contribution of the motion components in the motion model to correct state-prediction drift caused by this variation, while maintaining computational efficiency by retaining a single-model structure. Evaluations on KITTI and the Waymo Open Dataset, complemented by vehicle experiments, show that MD-KF achieves tracking performance comparable to multimodel KF with an average 1.72x speedup, while outperforming it in handling highly distant and prolonged occluded objects.

cs.CV↗