arXiv Science⌕ Search

arXiv subjects

Hongbo Yu

Publications and source records attributed to Hongbo Yu.

5 recordsLinked to original sources

PACT: Plan-Action Consistency Training by Preserving Correct Plans and Repairing Actions in GUI Agents

GUI agents can formulate a correct immediate plan in their reasoning yet emit an action that fails to execute it, creating an execution gap. This recurring mismatch undermines reliable task completion, as a single execution error can derail a multi-step workflow. In outcome-supervised reinforcement learning, broadcasting a negative response-level advantage across all generated tokens risks penalizing valid planning together with erroneous execution. To address this credit-assignment conflict, we introduce PACT (Plan-Action Consistency Training), a training framework that combines reference-aligned plan protection and plan-aligned action repair without altering the original scalar action reward. Using annotated reference actions to identify execution gaps, PACT protects the verified plan by masking negative policy-gradient contributions across its token span. Keeping the generated reasoning unchanged as context, it then favors the reference action over the emitted erroneous action through a preference surrogate with stop-gradient pair weights. Compared with the outcome-only baseline, PACT raises AndroidWorld task success from 60.34% to 68.97% (+8.63 pp), while improving rule-based plan-action consistency on AndroidControl High from 83.71% to 89.74% (+6.03 pp). Adapting PACT to desktop agents also improves task success from 40.92% to 44.17% (+3.25 pp) on the full 369-task OSWorld benchmark. Together, these results establish PACT as an effective credit-assignment framework for bridging execution gaps and improving task completion across mobile and desktop GUI environments.

cs.AI↗

GuidedRay: Diversity-Guided Direction Discovery for Targeted Hard-Label Black-Box Attacks

Deep neural networks are vulnerable to adversarial attacks. Among black-box attacks, targeted decision-based attacks are particularly difficult: the attacker observes only the target model's top-1 label and aims to make it predict a prespecified target class under a bounded perturbation. Before perturbation refinement, the attacker must discover a direction that reaches the prescribed target region. This initialization step can incur substantial query cost. We propose GuidedRay, a targeted decision-based attack based on diversity-guided direction discovery. GuidedRay builds on two observations: target-class reference samples provide useful target-conditioned direction priors, and diverse candidates increase the probability of discovering a targeted adversarial direction. GuidedRay generates varied candidates from one or multiple target-class references and uses a one-query Fast Test to screen their induced sign directions. Once a feasible direction is found, GuidedRay applies Ray Search to reduce its decision-boundary radius. Experiments on CIFAR-10, CIFAR-100, and ImageNet demonstrate that GuidedRay consistently outperforms five state-of-the-art decision-based attacks at four evaluated query budgets from 500 to 5,000, with particularly pronounced gains in direction discovery during initialization. Against models protected by adversarial training or TRADES, it likewise achieves the highest attack success rate at all four query budgets.

cs.CR↗

Cryptanalytic Extraction of Neural Networks Without Known Architecture Assumption

Cryptanalytic model extraction aims to reconstruct a functionally equivalent model through black-box interactions with the victim model. Under the fundamental assumption that the network architecture is completely known, existing attacks achieve the goal by recovering the model parameters. In this paper, we explore whether this assumption can be removed practically. Focusing on ReLU fully connected networks, which are widely studied in this field, we propose a guess-and-determine framework that jointly recovers the network architecture (including network depth and hidden-layer dimensions) and the model parameters. This framework is based on a simple yet effective high-level idea: after designing a parameter recovery attack under the known-architecture assumption, we can analyze the architecture-sensitive traces observed during parameter recovery to recover the network architecture. We identify two such traces in differential extraction attacks: (i) a \emph{zero suffix} in the merged weight vectors produced by signature recovery, whose length reveals the hidden layer dimension; and (ii) an \emph{equality pattern} in the preimage-based sign recovery, which occurs only under the true hidden layer dimension. These two signals give rise to two routes for network architecture recovery. For the second-to-last layer, we further propose two methods, one for identifying it, and one for recovering its dimension. Practical end-to-end attacks are implemented on a wide range of ReLU neural networks, including both expansive and non-expansive networks. To the best of our knowledge, this is the first time the feasibility of achieving functionally equivalent extraction on deep neural networks, after removing the known-architecture assumption, has been demonstrated in practice.

cs.CR↗

PhysSFI-Net: Physics-informed Geometric Learning of Skeletal and Facial Interactions for Orthognathic Surgical Outcome Prediction

Orthognathic surgery repositions jaw bones to restore occlusion and enhance facial aesthetics. Accurate simulation of postoperative facial morphology is essential for preoperative planning. This study aims to develop and validate a physics-informed geometric deep learning framework named PhysSFI-Net for precise prediction of soft tissue deformation following orthognathic surgery. The model integrates a hierarchical feature extraction module with attention mechanisms to capture skeletal-facial interactions, an LSTM-based sequential predictor for incremental deformation, and a biomechanics-inspired reconstruction module for high-resolution facial modeling. The model was trained on 135 patients and externally validated on an independent cohort of 33 patients. Model performance was assessed using point cloud shape error, surface deviation error and landmark error between predicted facial shapes with corresponding ground truths. Quantitative analysis demonstrated that PhysSFI-Net achieved a global shape error of 1.070 +/- 0.088 mm, a surface deviation error of 1.296 +/- 0.349 mm and a landmark error of 2.445 +/- 1.326 mm. Comparative experiments indicated that PhysSFI-Net outperformed the state-of-the-art method ACMT-Net and baseline models. External validation further confirmed its robustness with a global HD of 1.431 +/- 0.087 mm and consistently lower subregional and mesh-based errors. In conclusion, PhysSFI-Net enables interpretable, high-resolution prediction of postoperative facial morphology, showing strong potential for clinical application in orthognathic surgical planning.

cs.CV↗

Delving into Cryptanalytic Extraction of PReLU Neural Networks

The machine learning problem of model extraction was first introduced in 1991 and gained prominence as a cryptanalytic challenge starting with Crypto 2020. For over three decades, research in this field has primarily focused on ReLU-based neural networks. In this work, we take the first step towards the cryptanalytic extraction of PReLU neural networks, which employ more complex nonlinear activation functions than their ReLU counterparts. We propose a raw output-based parameter recovery attack for PReLU networks and extend it to more restrictive scenarios where only the top-m probability scores are accessible. Our attacks are rigorously evaluated through end-to-end experiments on diverse PReLU neural networks, including models trained on the MNIST dataset. To the best of our knowledge, this is the first practical demonstration of PReLU neural network extraction across three distinct attack scenarios.

cs.CR↗