arXiv subjects

Zhiwei Zhang

Publications and source records attributed to Zhiwei Zhang.

At least 91 records · Page 5Linked to original sources

arXiv2025

Can Large Multimodal Models Understand Agricultural Scenes? Benchmarking with AgroMind

Large Multimodal Models (LMMs) has demonstrated capabilities across various domains, but comprehensive benchmarks for agricultural remote sensing (RS) remain scarce. Existing benchmarks designed for agricultural RS scenarios exhibit notable limitations, primarily in terms of insufficient scene diversity in the dataset and oversimplified task design. To bridge this gap, we introduce AgroMind, a comprehensive agricultural remote sensing benchmark covering four task dimensions: spatial perception, object understanding, scene understanding, and scene reasoning, with a total of 13 task types, ranging from crop identification and health monitoring to environmental analysis. We curate a high-quality evaluation set by integrating eight public datasets and one private farmland plot dataset, containing 27,247 QA pairs and 19,615 images. The pipeline begins with multi-source data pre-processing, including collection, format standardization, and annotation refinement. We then generate a diverse set of agriculturally relevant questions through the systematic definition of tasks. Finally, we employ LMMs for inference, generating responses, and performing detailed examinations. We evaluated 20 open-source LMMs and 4 closed-source models on AgroMind. Experiments reveal significant performance gaps, particularly in spatial reasoning and fine-grained recognition, it is notable that human performance lags behind several leading LMMs. By establishing a standardized evaluation framework for agricultural RS, AgroMind reveals the limitations of LMMs in domain knowledge and highlights critical challenges for future work. Data and code can be accessed at https://rssysu.github.io/AgroMind/.

cs.CV↗

arXiv2025

The benefit of dose-exposure-response modeling in the estimation of dose-response relationship and dose optimization: some theoretical and simulation evidence

In randomized dose-finding trials, although drug exposure data form a part of key information for dose selection, the evaluation of the dose-response (DR) relationship often mainly uses DR data. We examine the benefit of dose-exposure-response (DER) modeling by sequentially modeling the dose-exposure (DE) and exposure-response (ER) relationships in parameter estimation and prediction, compared with direct DR modeling without PK data. We consider ER modeling approaches with control function (CF) that adjust for unobserved confounders in the ER relationship using randomization as an instrumental variable (IV). With both analytical derivation and a simulation study, we show that when the DE and ER models are linear, although the DER approach is moderately more efficient than the DR approach, with adjustment using CF, it has no efficiency gain (but also no loss). However, with some common ER models representing sigmoid curves, generally DER approaches with and without CF adjustment are more efficient than the DR approach. For response prediction at a given dose, the efficiency also depends on the dose level. Our simulation quantifies the benefit in multiple scenarios with different models and parameter settings. Our method can be used easily to assess the performance of randomized dose-finding trial designs.

stat.ME↗

arXiv2025

Robust estimation of causal dose-response relationship using exposure data with dose as an instrumental variable

An accurate estimation of the dose-response relationship is important to determine the optimal dose. For this purpose, a dose finding trial in which subjects are randomized to a few fixed dose levels is the most commonly used design. Often, the estimation uses response data only, although drug exposure data are often obtained during the trial. The use of exposure data to improve this estimation is difficult, as exposure-response relationships are typically subject to confounding bias even in a randomized trial. We propose a robust approach to estimate the dose-response relationship without assuming a true exposure-response model, using dose as an instrumental variable. Our approach combines the control variable approach in causal inference with unobserved confounding factors and the ANCOVA adjustment of randomized trials. The approach presented uses working models for dose-exposure-response data, but they are robust to model misspecification and remain consistent when the working models are far from correct. The asymptotic properties of the proposed approach are also examined. A simulation study is performed to evaluate the performance of the proposed approach. For illustration, the approach is used to a Car-T trial with randomized doses.

stat.ME↗

arXiv2025

Bradley-Terry and Multi-Objective Reward Modeling Are Complementary

Reward models trained on human preference data have demonstrated strong effectiveness in aligning Large Language Models (LLMs) with human intent under the framework of Reinforcement Learning from Human Feedback (RLHF). However, RLHF remains vulnerable to reward hacking, where the policy exploits imperfections in the reward function rather than genuinely learning the intended behavior. Although significant efforts have been made to mitigate reward hacking, they predominantly focus on and evaluate in-distribution scenarios, where the training and testing data for the reward model share the same distribution. In this paper, we empirically show that state-of-the-art methods struggle in more challenging out-of-distribution (OOD) settings. We further demonstrate that incorporating fine-grained multi-attribute scores helps address this challenge. However, the limited availability of high-quality data often leads to weak performance of multi-objective reward functions, which can negatively impact overall performance and become the bottleneck. To address this issue, we propose a unified reward modeling framework that jointly trains Bradley--Terry (BT) single-objective and multi-objective regression-based reward functions using a shared embedding space. We theoretically establish a connection between the BT loss and the regression objective and highlight their complementary benefits. Specifically, the regression task enhances the single-objective reward function's ability to mitigate reward hacking in challenging OOD settings, while BT-based training improves the scoring capability of the multi-objective reward function, enabling a 7B model to outperform a 70B baseline. Extensive experimental results demonstrate that our framework significantly improves both the robustness and the scoring performance of reward models.

cs.LG↗

arXiv2025

Thinking Out of the Box: Hybrid SAT Solving by Unconstrained Continuous Optimization

The Boolean satisfiability (SAT) problem lies at the core of many applications in combinatorial optimization, software verification, cryptography, and machine learning. While state-of-the-art solvers have demonstrated high efficiency in handling conjunctive normal form (CNF) formulas, numerous applications require non-CNF (hybrid) constraints, such as XOR, cardinality, and Not-All-Equal constraints. Recent work leverages polynomial representations to represent such hybrid constraints, but it relies on box constraints that can limit the use of powerful unconstrained optimizers. In this paper, we propose unconstrained continuous optimization formulations for hybrid SAT solving by penalty terms. We provide theoretical insights into when these penalty terms are necessary and demonstrate empirically that unconstrained optimizers (e.g., Adam) can enhance SAT solving on hybrid benchmarks. Our results highlight the potential of combining continuous optimization and machine-learning-based methods for effective hybrid SAT solving.

cs.LO↗

arXiv2025

Are You Using Reliable Graph Prompts? Trojan Prompt Attacks on Graph Neural Networks

Graph Prompt Learning (GPL) has been introduced as a promising approach that uses prompts to adapt pre-trained GNN models to specific downstream tasks without requiring fine-tuning of the entire model. Despite the advantages of GPL, little attention has been given to its vulnerability to backdoor attacks, where an adversary can manipulate the model's behavior by embedding hidden triggers. Existing graph backdoor attacks rely on modifying model parameters during training, but this approach is impractical in GPL as GNN encoder parameters are frozen after pre-training. Moreover, downstream users may fine-tune their own task models on clean datasets, further complicating the attack. In this paper, we propose TGPA, a backdoor attack framework designed specifically for GPL. TGPA injects backdoors into graph prompts without modifying pre-trained GNN encoders and ensures high attack success rates and clean accuracy. To address the challenge of model fine-tuning by users, we introduce a finetuning-resistant poisoning approach that maintains the effectiveness of the backdoor even after downstream model adjustments. Extensive experiments on multiple datasets under various settings demonstrate the effectiveness of TGPA in compromising GPL models with fixed GNN encoders.

cs.LG↗

arXiv2025

Optimal Treatment Allocations Accounting for Population Differences

The treatment allocation mechanism in a randomized clinical trial can be optimized by maximizing the nonparametric efficiency bound for a specific measure of treatment effect. Optimal treatment allocations which may or may not depend on baseline covariates have been derived for a variety of effect measures focusing on the trial population, the patient population represented by the trial participants. Frequently, clinical trial data are used to estimate treatment effects in a target population that is related to but different from the trial population. This article provides optimal treatment allocations that account for the impact of such population differences. We consider three cases with different data configurations: transportation, generalization, and post-stratification. Our results indicate that, for general effect measures, optimal treatment allocations may depend on the covariate distribution in the target population but not on the configuration of data or information that describes the target covariate distribution. For estimating average treatment effects, there is a unique covariate-dependent allocation that achieves maximal efficiency regardless of the target covariate distribution and the associated data configuration.

stat.ME↗

arXiv2025

Drive in Corridors: Enhancing the Safety of End-to-end Autonomous Driving via Corridor Learning and Planning

Safety remains one of the most critical challenges in autonomous driving systems. In recent years, the end-to-end driving has shown great promise in advancing vehicle autonomy in a scalable manner. However, existing approaches often face safety risks due to the lack of explicit behavior constraints. To address this issue, we uncover a new paradigm by introducing the corridor as the intermediate representation. Widely adopted in robotics planning, the corridors represents spatio-temporal obstacle-free zones for the vehicle to traverse. To ensure accurate corridor prediction in diverse traffic scenarios, we develop a comprehensive learning pipeline including data annotation, architecture refinement and loss formulation. The predicted corridor is further integrated as the constraint in a trajectory optimization process. By extending the differentiability of the optimization, we enable the optimized trajectory to be seamlessly trained within the end-to-end learning framework, improving both safety and interpretability. Experimental results on the nuScenes dataset demonstrate state-of-the-art performance of our approach, showing a 66.7% reduction in collisions with agents and a 46.5% reduction with curbs, significantly enhancing the safety of end-to-end driving. Additionally, incorporating the corridor contributes to higher success rates in closed-loop evaluations. Project page: https://zhiwei-pg.github.io/Drive-in-Corridors.

cs.RO↗

arXiv2025

A Unified Approach to Covariate Adjustment for Survival Endpoints in Randomized Clinical Trials

Covariate adjustment aims to improve the statistical efficiency of randomized trials by incorporating information from baseline covariates. Popular methods for covariate adjustment include analysis of covariance for continuous endpoints and standardized logistic regression for binary endpoints. For survival endpoints, while some covariate adjustment methods have been developed for specific effect measures, they are not commonly used in practice for various reasons, including high demands for theoretical and methodological sophistication as well as computational skills. This article describes an augmentation approach to covariate adjustment for survival endpoints that is relatively easy to understand and widely applicable to different effect measures. This approach involves augmenting a given treatment effect estimator in a way that preserves interpretation, consistency, and asymptotic normality. The optimal augmentation term, which minimizes asymptotic variance, can be estimated using various statistical and machine learning methods. Simulation results demonstrate that the augmentation approach can bring substantial gains in statistical efficiency. This approach has been implemented in an R package named \texttt{sleete}, which is described in detail and illustrated with real data.

stat.ME↗

arXiv2025

MediAug: Exploring Visual Augmentation in Medical Imaging

Data augmentation is essential in medical imaging for improving classification accuracy, lesion detection, and organ segmentation under limited data conditions. However, two significant challenges remain. First, a pronounced domain gap between natural photographs and medical images can distort critical disease features. Second, augmentation studies in medical imaging are fragmented and limited to single tasks or architectures, leaving the benefits of advanced mix-based strategies unclear. To address these challenges, we propose a unified evaluation framework with six mix-based augmentation methods integrated with both convolutional and transformer backbones on brain tumour MRI and eye disease fundus datasets. Our contributions are threefold. (1) We introduce MediAug, a comprehensive and reproducible benchmark for advanced data augmentation in medical imaging. (2) We systematically evaluate MixUp, YOCO, CropMix, CutMix, AugMix, and SnapMix with ResNet-50 and ViT-B backbones. (3) We demonstrate through extensive experiments that MixUp yields the greatest improvement on the brain tumor classification task for ResNet-50 with 79.19% accuracy and SnapMix yields the greatest improvement for ViT-B with 99.44% accuracy, and that YOCO yields the greatest improvement on the eye disease classification task for ResNet-50 with 91.60% accuracy and CutMix yields the greatest improvement for ViT-B with 97.94% accuracy. Code will be available at https://github.com/AIGeeksGroup/MediAug.

cs.CV↗

arXiv2025

Threshold for Fault-tolerant Quantum Advantage with the Quantum Approximate Optimization Algorithm

Optimization is often cited as a promising application of quantum computers. However, the low degree of provable quantum speedups has led prior rigorous end-to-end resource analyses to conclude that a quantum computer is unlikely to surpass classical state-of-the-art on optimization problems under realistic assumptions. In this work, we compile and analyze the Quantum Approximate Optimization Algorithm (QAOA) combined with Amplitude Amplification (AA) applied to random 8-SAT at the satisfiability threshold. Our compilation involves careful optimization of circuits for Hamiltonian simulation, which may be of independent interest. We use the analytical scaling of the time-to-solution for QAOA identified by PRX Quantum 5, 030348 (2024) and find that with QAOA depth $p=623$, QAOA+AA achieves a crossover with state-of-the-art classical heuristics at 179 variables and 14.99 hours of runtime when executed on a surface-code-based fault-tolerant quantum computer with 73.91 million physical qubits, a physical error rate of $10^{-3}$, and a $1~μ$s code cycle time. Notably, we allow the classical solver to be parallelized as long as its total energy consumption is equal to that required for decoding in the surface code. We further show that this restriction on classical solver energy consumption can be relaxed given optimistic but plausible reductions in physical error rates and fault-tolerance overheads, enabling a crossover of 2.94 hours using 8.88 million physical qubits against a classical solver running on a supercomputer with $725,760$ CPU cores. These findings support the hypothesis that large-scale fault-tolerant quantum computers will be useful for optimization.

quant-ph↗

arXiv2025

GEOcc: Geometrically Enhanced 3D Occupancy Network with Implicit-Explicit Depth Fusion and Contextual Self-Supervision

3D occupancy perception holds a pivotal role in recent vision-centric autonomous driving systems by converting surround-view images into integrated geometric and semantic representations within dense 3D grids. Nevertheless, current models still encounter two main challenges: modeling depth accurately in the 2D-3D view transformation stage, and overcoming the lack of generalizability issues due to sparse LiDAR supervision. To address these issues, this paper presents GEOcc, a Geometric-Enhanced Occupancy network tailored for vision-only surround-view perception. Our approach is three-fold: 1) Integration of explicit lift-based depth prediction and implicit projection-based transformers for depth modeling, enhancing the density and robustness of view transformation. 2) Utilization of mask-based encoder-decoder architecture for fine-grained semantic predictions; 3) Adoption of context-aware self-training loss functions in the pertaining stage to complement LiDAR supervision, involving the re-rendering of 2D depth maps from 3D occupancy features and leveraging image reconstruction loss to obtain denser depth supervision besides sparse LiDAR ground-truths. Our approach achieves State-Of-The-Art performance on the Occ3D-nuScenes dataset with the least image resolution needed and the most weightless image backbone compared with current models, marking an improvement of 3.3% due to our proposed contributions. Comprehensive experimentation also demonstrates the consistent superiority of our method over baselines and alternative approaches.

cs.CV↗

arXiv2025

Catastrophic Failure of LLM Unlearning via Quantization

Large language models (LLMs) have shown remarkable proficiency in generating text, benefiting from extensive training on vast textual corpora. However, LLMs may also acquire unwanted behaviors from the diverse and sensitive nature of their training data, which can include copyrighted and private content. Machine unlearning has been introduced as a viable solution to remove the influence of such problematic content without the need for costly and time-consuming retraining. This process aims to erase specific knowledge from LLMs while preserving as much model utility as possible. Despite the effectiveness of current unlearning methods, little attention has been given to whether existing unlearning methods for LLMs truly achieve forgetting or merely hide the knowledge, which current unlearning benchmarks fail to detect. This paper reveals that applying quantization to models that have undergone unlearning can restore the "forgotten" information. To thoroughly evaluate this phenomenon, we conduct comprehensive experiments using various quantization techniques across multiple precision levels. We find that for unlearning methods with utility constraints, the unlearned model retains an average of 21\% of the intended forgotten knowledge in full precision, which significantly increases to 83\% after 4-bit quantization. ... Our code is available at: \href{https://github.com/zzwjames/FailureLLMUnlearning}{https://github.com/zzwjames/FailureLLMUnlearning}.

cs.CL↗

arXiv2025

Robustness Inspired Graph Backdoor Defense

Graph Neural Networks (GNNs) have achieved promising results in tasks such as node classification and graph classification. However, recent studies reveal that GNNs are vulnerable to backdoor attacks, posing a significant threat to their real-world adoption. Despite initial efforts to defend against specific graph backdoor attacks, there is no work on defending against various types of backdoor attacks where generated triggers have different properties. Hence, we first empirically verify that prediction variance under edge dropping is a crucial indicator for identifying poisoned nodes. With this observation, we propose using random edge dropping to detect backdoors and theoretically show that it can efficiently distinguish poisoned nodes from clean ones. Furthermore, we introduce a novel robust training strategy to efficiently counteract the impact of the triggers. Extensive experiments on real-world datasets show that our framework can effectively identify poisoned nodes, significantly degrade the attack success rate, and maintain clean accuracy when defending against various types of graph backdoor attacks with different properties.

cs.LG↗

arXiv2025

LanP: Rethinking the Impact of Language Priors in Large Vision-Language Models

Large Vision-Language Models (LVLMs) have shown impressive performance in various tasks. However, LVLMs suffer from hallucination, which hinders their adoption in the real world. Existing studies emphasized that the strong language priors of LVLMs can overpower visual information, causing hallucinations. However, the positive role of language priors is the key to a powerful LVLM. If the language priors are too weak, LVLMs will struggle to leverage rich parameter knowledge and instruction understanding abilities to complete tasks in challenging visual scenarios where visual information alone is insufficient. Therefore, we propose a benchmark called LanP to rethink the impact of Language Priors in LVLMs. It is designed to investigate how strong language priors are in current LVLMs. LanP consists of 170 images and 340 corresponding well-designed questions. Extensive experiments on 25 popular LVLMs reveal that many LVLMs' language priors are not strong enough to effectively aid question answering when objects are partially hidden. Many models, including GPT-4 Turbo, exhibit an accuracy below 0.5 in such a scenario.

cs.CV↗

arXiv2025

Graph-based Molecular In-context Learning Grounded on Morgan Fingerprints

In-context learning (ICL) effectively conditions large language models (LLMs) for molecular tasks, such as property prediction and molecule captioning, by embedding carefully selected demonstration examples into the input prompt. This approach avoids the computational overhead of extensive pertaining and fine-tuning. However, current prompt retrieval methods for molecular tasks have relied on molecule feature similarity, such as Morgan fingerprints, which do not adequately capture the global molecular and atom-binding relationships. As a result, these methods fail to represent the full complexity of molecular structures during inference. Moreover, small-to-medium-sized LLMs, which offer simpler deployment requirements in specialized systems, have remained largely unexplored in the molecular ICL literature. To address these gaps, we propose a self-supervised learning technique, GAMIC (Graph-Aligned Molecular In-Context learning, which aligns global molecular structures, represented by graph neural networks (GNNs), with textual captions (descriptions) while leveraging local feature similarity through Morgan fingerprints. In addition, we introduce a Maximum Marginal Relevance (MMR) based diversity heuristic during retrieval to optimize input prompt demonstration samples. Our experimental findings using diverse benchmark datasets show GAMIC outperforms simple Morgan-based ICL retrieval methods across all tasks by up to 45%.

cs.LG↗

arXiv2025

A Connection Between Covariate Adjustment and Stratified Randomization in Randomized Clinical Trials

The statistical efficiency of randomized clinical trials can be improved by incorporating information from baseline covariates (i.e., pre-treatment patient characteristics). This can be done in the design stage using stratified (permutated block) randomization or in the analysis stage through covariate adjustment. This article makes a connection between covariate adjustment and stratified randomization in a general framework where all regular, asymptotically linear estimators are identified as augmented estimators. From a geometric perspective, covariate adjustment can be viewed as an attempt to approximate the optimal augmentation function, and stratified randomization improves a given approximation by moving it closer to the optimal augmentation function. The efficiency benefit of stratified randomization is asymptotically equivalent to attaching an optimal augmentation term based on the stratification factor. In designing a trial with stratified randomization, it is not essential to include all important covariates in the stratification, because their prognostic information can be incorporated through covariate adjustment. Under stratified randomization, adjusting for the stratification factor only in data analysis is not expected to improve efficiency, and the key to efficient estimation is incorporating prognostic information from all important covariates. These observations are confirmed in a simulation study and illustrated using real clinical trial data.

stat.ME↗

arXiv2025

A Likelihood Perspective on Dose-Finding Study Designs in Oncology

Dose-finding studies in oncology often include an up-and-down dose transition rule that assigns a dose to each cohort of patients based on accumulating data on dose-limiting toxicity (DLT) events. In making a dose transition decision, a key scientific question is whether the true DLT rate of the current dose exceeds the target DLT rate, and the statistical question is how to evaluate the statistical evidence in the available DLT data with respect to that scientific question. This article introduces generalized likelihood ratios (GLRs) that can be used to measure statistical evidence and support dose transition decisions. Applying this approach to a single-dose likelihood leads to a GLR-based interval design with three parameters: the target DLT rate and two GLR cut-points representing the levels of evidence required for dose escalation and de-escalation. This design gives a likelihood interpretation to each existing interval design and provides a unified framework for comparing different interval designs in terms of how much evidence is required for escalation and de-escalation. A GLR-based comparison of commonly used interval designs reveals important differences and motivates alternative designs that reduce over-treatment while maintaining MTD estimation accuracy. The GLR-based approach can also be applied to a joint likelihood based on a nonparametric (e.g., isotonic regression) model or a parametric model. Simulation results indicate that the isotonic GLR performs similarly to the single-dose GLR but the GLR based on a parsimonious model can improve MTD estimation when the underlying model is correct.

stat.ME↗