arXiv Science⌕ Search

arXiv subjects

Boyang Deng

Publications and source records attributed to Boyang Deng.

At least 19 recordsLinked to original sources

SearchChord: Prompt Preselection and Field Recomposition for One-Step Image Editing

Finding effective prompts and seeds for one-step image editors requires balancing edit fidelity, source preservation, and computational cost. To address this challenge, we explore a large collection of candidate edits, where each candidate is defined by a unique prompt-seed pair, and generate corresponding images on demand. We present SearchChord, a training-free framework that leverages a large pool of such candidates, combining text-based preselection, image verification, and latent-field recomposition. Our ablation studies across pool sizes up to 200 candidates show that a pool of 100 offers a cost-effective operating point. Compared with feeding all 100 candidates into recomposition, TextSim preselects a subset of 10 candidates. Rather than uniform averaging, we incorporate image-verifier scores to weight each candidate during recomposition. Experiments on PIE-Bench and the proposed CropWeed-EditBench reveal a trade-off: candidate selection improves target-text alignment, while recomposition better preserves unmodified source content. The oracle verifier achieves the best performance in both preservation and target-text alignment. Practical verification partially recovers semantic alignment, but recomposition still compromises edit fidelity, including in agricultural image edits. These results motivate text-first preselection and stronger verification to balance preservation, fidelity, and cost. Code and data will be made publicly available.

cs.CV↗

CityRAG: Stepping Into a City via Spatially-Grounded Video Generation

We address the problem of generating a 3D-consistent, navigable environment that is spatially grounded: a simulation of a real location. Existing video generative models can produce a plausible sequence that is consistent with a text (T2V) or image (I2V) prompt. However, the capability to reconstruct the real world under arbitrary weather conditions and dynamic object configurations is essential for downstream applications including autonomous driving and robotics simulation. To this end, we present CityRAG, a video generative model that leverages large corpora of geo-registered data as context to ground generation to the physical scene, while maintaining learned priors for complex motion and appearance changes. CityRAG relies on temporally unaligned training data, which teaches the model to semantically disentangle the underlying scene from its transient attributes. Our experiments demonstrate that CityRAG can generate coherent minutes-long, physically grounded video sequences, maintain weather and lighting conditions over thousands of frames, achieve loop closure, and navigate complex trajectories to reconstruct real-world geography.

cs.CV↗

FluidGen: Physics-Aware Fluid Video Generation via Motion-Guided Post-Training

Video generation models produce visually compelling motion but systematically fail on fluid phenomena, including smoke, fire, and explosion debris, where outputs lack coherent vortical structure and collapse to bland dynamics that violate the physics of real fluids. Classical solvers achieve high physical fidelity but require detailed 3D scenes and carefully tuned parameters, making them impractical for in-the-wild imagery and video generation. We address this gap by injecting a data-driven fluid prior into a pre-trained video model, trained on a mixture of high-fidelity Houdini simulation data with paired velocity-field ground truth and curated in-the-wild fluid video data for appearance diversity. The core of our approach is a dual stream architecture that predicts per-frame velocity fields alongside RGB, supervised by simulation ground truth. This auxiliary signal regularizes generation toward physically consistent motion and exposes an interpretable physical quantity that we further leverage as a reward for velocity-aware direct preference optimization. Our model produces substantially more realistic fluid dynamics sequences than strong video baselines and optimization-based methods. Project page: https://y-zheng18.github.io/FluidGen/

cs.CV↗

Multi-Speed Field Evaluation of a Semi-Autonomous Precision Sprayer for Weed Control in Vegetable Crops

Machine vision-guided precision spraying is promising for sustainable vegetable production by enabling site-specific weed targeting, reducing herbicide application while protecting crop yield and quality. However, the effects of operating speed of sprayer systems on the integrated detection-tracking-actuation pipeline remain insufficiently characterized under field conditions. This study evaluated a semi-autonomous, tractor-based spraying system for targeted weed control in radish and quantified performance across three operating speeds. A five-season dataset comprising 10,599 images and 84,311 annotated instances across seven classes was curated for model development. A YOLOv11-medium model was integrated with ByteTrack tracking and a customized 12-nozzle sprayer on a semi-autonomous electric tractor. Field trials were conducted at 0.13, 0.27, and 0.45 m/s (i.e., 0.3, 0.6, and 1.0 mph) to characterize speed-dependent changes in weed detection, targeting, actuation, and crop damage. At the lowest speed of 0.13 m/s, the system achieved 94.6% mAP@50 and 93.2% recall for weed and crop detection. However, increasing platform speed substantially reduced system performance, with weed hit rate declining from 78.3% to 35.0% and crop contact rates increasing from 20.1% to 28.7%. Consequently, the system did not meet predefined commercial-viability thresholds. These results demonstrate that practical deployment requires improvements beyond weed detection alone, particularly coordinated improvements in motion-robust plant perception, spray-timing compensation, and crop-adjacent actuation precision. The identified performance bottlenecks provide specific engineering targets for advancing vision-guided precision spraying toward practical, high-speed vegetable weed management. The software programs developed in this study have been made publicly available to support future precision weeding research.

cs.CV↗

Unbiased first-principles construction of complete tensorial spin Hamiltonians

Magnetic ground states are commonly predicted using spin Hamiltonians whose interaction terms are selected a priori, potentially overlooking the microscopic interactions that govern complex magnetic order. Here, we introduce a general framework for the unbiased first-principles construction of symmetry-complete tensorial spin Hamiltonians and its automated implementation in AMATIS. The framework constructs the Hamiltonian directly from density-functional theory while rigorously enforcing quantum spin algebra and crystallographic symmetry. Applied to representative two-dimensional van der Waals magnets, the framework reproduces established magnetic interactions and uncovers hidden physics beyond conventional spin models, including chiral interactions that stabilize metastable skyrmions, higher-rank tensorial interactions that reconstruct the magnetic phase diagram and establish stabilizing competing multi-Q phases, and an emergent p-wave altermagnetic electronic structure. Our results demonstrate that unbiased tensorial Hamiltonian construction provides a predictive alternative to the conventional practice of manually selecting spin-model interactions, enabling first-principles discovery of unconventional magnetic phases.

cond-mat.mtrl-sci↗

Automated Generation of Commensurate Magnetic Structures based on Spin Space Groups and Graph Theory

Magnetic structures with symmetry constraints are candidates for energetically favorable configurations. Enumerating these structures is essential for identifying experimental observations and provides unbiased, linearly stable, and optimally sampled reference configurations for energy fitting when extracting spin interactions. We present SpinGraph, an automated workflow generating symmetry-distinct magnetic configurations. SpinGraph is not only compatible with the more general spin space groups, but also allows precise control of the prescribed single- or multi-Q states superposition. For finite groups, we enumerate compatible subgroups directly. For spin space groups with a continuous or special spin-only part, the finite component is enumerated first and then combined with the compatible exact spin-only constraints. These actions are translated into graph constraints in real or Fourier space. The symmetry of every generated structure is re-evaluated after construction. Finally, we integrate SpinGraph with the magnetic analysis code AMATIS to perform a thorough calculation for the spin interactions of the insulating monolayer CrI3. SpinGraph complements the last part of AMATIS and realizes the fully automated workflow for the spin Hamiltonian construction, which is essential for studying phase transitions, spin textures, magnons, and spin dynamics.

cond-mat.mtrl-sci↗

Policy-based Foveated Imaging and Perception

Ultra-high-resolution image sensors offer the potential to capture fine spatial details critical for many visual perception tasks, but acquiring and processing all pixels at full resolution is often infeasible under realistic bandwidth, latency, and power constraints. Existing approaches address this challenge through acquisition strategies such as spatial or temporal downsampling, which irrevocably discard information before task relevance can be assessed. In this work, we introduce a real-time, predictive, and task-aware foveated imaging system that operates directly at image acquisition time. Leveraging emerging dual-stream sensor architectures, our method dynamically allocates limited pixel bandwidth to task-relevant regions of interest while maintaining a low-resolution global context. We formulate foveated acquisition as a sensor attention policy-learning problem, in which past observations guide actions that determine future measurements, closing the perception-acquisition loop. Through extensive simulation across multiple perception tasks, we demonstrate that our approach achieves high task performance under strict pixel budgets and significantly outperforms relevant baselines operating at the same bandwidth. We further validate our system on a 200-megapixel dual-stream sensor, capturing real-world videos under realistic bandwidth and latency constraints, demonstrating the practical feasibility of task-driven, acquisition-time foveated imaging.

cs.CV↗

GeoFlow: Enforcing Implicit Geometric Consistency in Video Generation

Generating geometrically consistent videos remains an open challenge: text-to-video diffusion models trained on web-scale data treat geometry only implicitly, leading to object deformation, texture drift, and non-rigid backgrounds under camera motion. Existing solutions either improve consistency as a byproduct, apply only to static scenes or realign the latent space of the model completely. We introduce a geometry-consistency reward that directly measures whether motion in a generated video is compatible with a coherent scene. Our key insight is that in physically consistent videos, background motion should be explainable by rigid camera-induced flow, while independently moving objects should preserve appearance identity along motion trajectories. We operationalize this using optical flow, depth--pose predictions, and feature-based correspondence to separate rigid and dynamic regions and evaluate their respective consistency. Integrating this reward with reinforcement fine-tuning transforms geometric consistency from an emergent property into an explicit optimization objective for video generators. The approach is model agnostic and applies to diverse dynamic scenes containing both camera and object motion. Experiments show substantial reductions in temporal geometric artifacts over strong baselines while preserving perceptual quality. Code and model weights are published.

cs.CV↗

DINOv3 Meets YOLO26 for Weed Detection in Vegetable Crops

Developing robust models for precision vegetable weeding is currently constrained by the scarcity of large-scale, annotated weed-crop datasets. To address this limitation, this study proposes a foundational crop-weed detection model by integrating heterogeneous datasets and leveraging self-supervised learning. A total of 618,642 crop-weed images were initially collected and subsequently refined to 199,388 filtered images for fine-tuning a DINOv3 vision transformer (ViT-small) through a sequential curation strategy. The fine-tuned DINOv3 backbone was then integrated into YOLO26, serving either as a primary backbone or part of a dual-backbone architecture. A feature alignment loss was introduced in the dual backbone framework to enhance feature fusion with minimal computational overhead. Experimental results show that the proposed DINOv3-finetuned ViT-small-based YOLO26-large achieved up to a +5.4% mAP50 gain on in-domain images collected in the 2025 season. Moreover, it demonstrated strong cross-domain generalization with mAP50 improvements of +14.0% on the 2021-2023 season dataset and +11.9% on the 2024 season dataset, compared to the standard YOLO26-large. Although the DINOv3-YOLO26-large model has 45.6% more parameters and a 2.9x increase in inference latency, it maintains real-time performance at ~28.5 frames per second (fps). The curated dataset and software programs developed in this study will be made publicly available.

cs.CV↗

A Comparative Benchmark of Real-time Detectors for Blueberry Detection towards Precision Orchard Management

Blueberry detection in natural environments remains challenging due to variable lighting, occlusions, and motion blur due to environmental factors and imaging devices. Deep learning-based object detectors promise to address these challenges, but they demand a large-scale, diverse dataset that captures the real-world complexities. Moreover, deploying these models in practical scenarios often requires the right accuracy/speed/memory trade-off in model selection. This study presents a novel comparative benchmark analysis of advanced real-time object detectors, including YOLO (You Only Look Once) (v8-v12) and RT-DETR (Real-Time Detection Transformers) (v1-v2) families, consisting of 36 model variants, evaluated on a newly curated dataset for blueberry detection. This dataset comprises 661 canopy images collected with smartphones during the 2022-2023 seasons, consisting of 85,879 labelled instances (including 36,256 ripe and 49,623 unripe blueberries) across a wide range of lighting conditions, occlusions, and fruit maturity stages. Among the YOLO models, YOLOv12m achieved the best accuracy with a mAP@50 of 93.3%, while RT-DETRv2-X obtained a mAP@50 of 93.6%, the highest among all the RT-DETR variants. The inference time varied with the model scale and complexity, and the mid-sized models appeared to offer a good accuracy-speed balance. To further enhance detection performance, all the models were fine-tuned using Unbiased Mean Teacher-based semi-supervised learning (SSL) on a separate set of 1,035 unlabeled images acquired by a ground-based machine vision platform in 2024. This resulted in accuracy gains ranging from -1.4% to 2.9%, with RT-DETR-v2-X achieving the best mAP@50 of 94.8%. More in-depth research into SSL is needed to better leverage cross-domain unlabeled data. Both the dataset and software programs of this study are made publicly available to support further research.

cs.CV↗

Visual Chronicles: Using Multimodal LLMs to Analyze Massive Collections of Images

We present a system using Multimodal LLMs (MLLMs) to analyze a large database with tens of millions of images captured at different times, with the aim of discovering patterns in temporal changes. Specifically, we aim to capture frequent co-occurring changes ("trends") across a city over a certain period. Unlike previous visual analyses, our analysis answers open-ended queries (e.g., "what are the frequent types of changes in the city?") without any predetermined target subjects or training labels. These properties cast prior learning-based or unsupervised visual analysis tools unsuitable. We identify MLLMs as a novel tool for their open-ended semantic understanding capabilities. Yet, our datasets are four orders of magnitude too large for an MLLM to ingest as context. So we introduce a bottom-up procedure that decomposes the massive visual analysis problem into more tractable sub-problems. We carefully design MLLM-based solutions to each sub-problem. During experiments and ablation studies with our system, we find it significantly outperforms baselines and is able to discover interesting trends from images captured in large cities (e.g., "addition of outdoor dining,", "overpass was painted blue," etc.). See more results and interactive demos at https://boyangdeng.com/visual-chronicles.

cs.CV↗

Adding New Categories in Object Detection Using Few-Shot Copy-Paste

Developing data-efficient instance detection models that can handle rare object categories remains a key challenge in computer vision. However, existing research often overlooks data collection strategies and evaluation metrics tailored to real-world scenarios involving neural networks. In this study, we systematically investigate data collection and augmentation techniques focused on object occlusion, aiming to mimic occlusion relationships observed in practical applications. Surprisingly, we find that even a simple occlusion mechanism is sufficient to achieve strong performance when introducing new object categories. Notably, by adding just 15 images of a new category to a large-scale training dataset containing over half a million images across hundreds of categories, the model achieves 95\% accuracy on an unseen test set with thousands of instances of the new category.

cs.CV↗

Semi-Supervised Weed Detection in Vegetable Fields: In-domain and Cross-domain Experiments

Robust weed detection remains a challenging task in precision weeding, requiring not only potent weed detection models but also large-scale, labeled data. However, the labeled data adequate for model training is practically difficult to come by due to the time-consuming, labor-intensive process that requires specialized expertise to recognize plant species. This study introduces semi-supervised object detection (SSOD) methods for leveraging unlabeled data for enhanced weed detection and proposes a new YOLOv8-based SSOD method, i.e., WeedTeacher. An experimental comparison of four SSOD methods, including three existing frameworks (i.e., DenseTeacher, EfficientTeacher, and SmallTeacher) and WeedTeacher, alongside fully supervised baselines, was conducted for weed detection in both in-domain and cross-domain contexts. A new, diverse weed dataset was created as the testbed, comprising a total of 19,931 field images from two differing domains, including 8,435 labeled (basic-domain) images acquired by handholding devices from 2021 to 2023 and 11,496 unlabeled (new-domain) images acquired by a ground-based mobile platform in 2024. The in-domain experiment with models trained using 10% of the labeled, basic-domain images and tested on the remaining 90% of the data, showed that the YOLOv8-basedWeedTeacher achieved the highest accuracy among all four SSOD methods, with an improvement of 2.6% mAP@50 and 3.1% mAP@50:95 over its supervised baseline (i.e., YOLOv8l). In the cross-domain experiment where the unlabeled new-domain data was incorporated, all four SSOD methods, however, resulted in no or limited improvements over their supervised counterparts. Research is needed to address the difficulty of cross-domain data utilization for robust weed detection.

cs.CV↗

Stable Diffusion for Data Augmentation in COCO and Weed Datasets

Generative models have increasingly impacted various tasks, from computer vision to interior design and beyond. Stable Diffusion, a powerful diffusion model, enables the creation of high-resolution images with intricate details from text prompts or reference images. An intriguing challenge lies in improving performance for small datasets with image-sparse categories. This study explores the effectiveness of Stable Diffusion by evaluating seven common categories and three widespread weed species. Synthetic images were generated using three Stable Diffusion-based techniques: Image-to-Image Translation, DreamBooth, and ControlNet, each with distinct focuses. Classification and detection tasks were then performed on these synthetic images, and their performance was compared to models trained on original images. Promising results were achieved for certain classes, demonstrating the potential of Stable Diffusion in enhancing image-sparse datasets. This foundational study may accelerate the adaptation of diffusion models across various domains.

cs.CV↗

An Improved CNN-based Neural Network Model for Fruit Sugar Level Detection

Artificial Intelligence (AI) is widely used in image classification, recognition, text understanding, and natural language processing, leading to significant advancements. In this paper, we introduce AI into the field of fruit quality detection. We designed a regression model for fruit sugar level estimation, utilizing an Artificial Neural Network (ANN) based on the visible/near-infrared (V/NIR) spectra of fruits. After analyzing the fruit spectra, we proposed an innovative neural network structure: the lower layers consist of a Multilayer Perceptron (MLP), a middle layer features a 2-dimensional correlation matrix, and the upper layers contain several Convolutional Neural Network (CNN) layers. Using fruit sugar levels as the detection target, we collected data from two fruit types, Gan Nan Navel and Tian Shan Pear, and conducted separate experiments to compare their results. To assess the reliability of our dataset, we first applied Analysis of Variance (ANOVA). We then explored various strategies for processing spectral data and evaluated their impact. Additionally, we employed Wavelet Decomposition (WD) for dimensionality reduction and a Genetic Algorithm (GA) to identify optimal features. We compared the performance of Neural Network models with traditional Partial Least Squares (PLS) models, and specifically evaluated our proposed MLP-CNN structure against other traditional neural network architectures. Finally, we introduced a novel evaluation metric based on the dataset's standard deviation (STD) to assess detection performance, demonstrating the feasibility of using an artificial neural network model for nondestructive fruit sugar level detection.

cs.AI↗

Robust Symmetry Detection via Riemannian Langevin Dynamics

Symmetries are ubiquitous across all kinds of objects, whether in nature or in man-made creations. While these symmetries may seem intuitive to the human eye, detecting them with a machine is nontrivial due to the vast search space. Classical geometry-based methods work by aggregating "votes" for each symmetry but struggle with noise. In contrast, learning-based methods may be more robust to noise, but often overlook partial symmetries due to the scarcity of annotated data. In this work, we address this challenge by proposing a novel symmetry detection method that marries classical symmetry detection techniques with recent advances in generative modeling. Specifically, we apply Langevin dynamics to a redefined symmetry space to enhance robustness against noise. We provide empirical results on a variety of shapes that suggest our method is not only robust to noise, but can also identify both partial and global symmetries. Moreover, we demonstrate the utility of our detected symmetries in various downstream tasks, such as compression and symmetrization of noisy shapes.

cs.CV↗

Streetscapes: Large-scale Consistent Street View Generation Using Autoregressive Video Diffusion

We present a method for generating Streetscapes-long sequences of views through an on-the-fly synthesized city-scale scene. Our generation is conditioned by language input (e.g., city name, weather), as well as an underlying map/layout hosting the desired trajectory. Compared to recent models for video generation or 3D view synthesis, our method can scale to much longer-range camera trajectories, spanning several city blocks, while maintaining visual quality and consistency. To achieve this goal, we build on recent work on video diffusion, used within an autoregressive framework that can easily scale to long sequences. In particular, we introduce a new temporal imputation method that prevents our autoregressive approach from drifting from the distribution of realistic city imagery. We train our Streetscapes system on a compelling source of data-posed imagery from Google Street View, along with contextual map data-which allows users to generate city views conditioned on any desired city layout, with controllable camera poses. Please see more results at our project page at https://boyangdeng.com/streetscapes.

cs.CV↗

GINA-3D: Learning to Generate Implicit Neural Assets in the Wild

Modeling the 3D world from sensor data for simulation is a scalable way of developing testing and validation environments for robotic learning problems such as autonomous driving. However, manually creating or re-creating real-world-like environments is difficult, expensive, and not scalable. Recent generative model techniques have shown promising progress to address such challenges by learning 3D assets using only plentiful 2D images -- but still suffer limitations as they leverage either human-curated image datasets or renderings from manually-created synthetic 3D environments. In this paper, we introduce GINA-3D, a generative model that uses real-world driving data from camera and LiDAR sensors to create realistic 3D implicit neural assets of diverse vehicles and pedestrians. Compared to the existing image datasets, the real-world driving setting poses new challenges due to occlusions, lighting-variations and long-tail distributions. GINA-3D tackles these challenges by decoupling representation learning and generative modeling into two stages with a learned tri-plane latent structure, inspired by recent advances in generative modeling of images. To evaluate our approach, we construct a large-scale object-centric dataset containing over 1.2M images of vehicles and pedestrians from the Waymo Open Dataset, and a new set of 80K images of long-tail instances such as construction equipment, garbage trucks, and cable cars. We compare our model with existing approaches and demonstrate that it achieves state-of-the-art performance in quality and diversity for both generated images and geometries.

cs.CV↗