arXiv subjects

Zhuoxi Wang

Publications and source records attributed to Zhuoxi Wang.

6 recordsLinked to original sources

arXiv2026

0%, 45%, or 99%: A Guardrail's Own Share of the Refusals It Is Credited With

A defended pipeline's refusals have two producers: the guardrail bolted in front of the model, and the model's own alignment. Recovering the split costs nothing, because a guard block replaces the model's response and the two counts are therefore disjoint. Holding the defense, the targets, the corpus and the judge fixed, the guardrail's own share of the refusals credited to it is 0%, 41-45%, or 99% across three settings that a results table would describe identically. Two choices move it, and neither belongs to the deployer who bought the guardrail. The attacker drives the share to zero by choosing which channel carries the payload: a plainly written request rendered as pixels, with nothing obfuscated, leaves a text guard's read covering none of it. The evaluator drives the share to 99% by choosing what text fills a defense's internal slots: fill them with the unencoded request behind an encoded attack, a read no deployed defender possesses, and the same guard blocks almost everything. The two consequences differ, and only the attacker's can happen to a running system. The evaluator's choice is an artifact carried by the literature, and its size is set by where the granted text lands: substantial at a guard gate, smaller in a caption-mediated re-check, absent in a majority-vote smoother, an ordering reproduced in an independent replicate. Isolating the grant inside the caption-mediated defense refutes the prediction we registered, since the harm-verdict stage contributes nothing while the stage that regenerates the answer carries the whole effect. The reference implementation builds every stage from a single prompt field that cannot represent the difference between what the attacker sent and what the benchmark records, and an audit of four further released harnesses and of the benchmark itself finds the same structural gap, so faithful porting supplies the grant silently.

cs.CR↗

arXiv2026

Unread or Unenforced? Separating Representation from Enforcement Failure in Content Guards

When an encoded attack passes a content guard, the guard either never represented the payload's harmful content or represented it and failed to act. End-to-end attack success rate reports one number for both, yet the two have opposite remedies: one is a representational limit that more safety training cannot reach, the other is a decision rule that it can. We separate them by reading a guard's own residual stream, using a content probe fitted on plaintext and transferred without refitting to the encoded condition, alongside the verdict logits from the same pass. Licensing that read honestly is most of the problem and is our main contribution. A conventional permutation test admits the decode measurement on most of a 19-condition encoding ladder for each of two open guards. A length-matched null and a floor calibrated on conditions the guard's base model provably cannot decode reduce it to four conditions each; holding out the items the probe was fitted on removes one more. A third screen constrains the block axis, which the decode screens leave untouched, by running plaintext content inside each condition's own wrapper. It removes the largest cell that survived them. What remains is a policy failure that survives an item-level holdout on two of the four surviving conditions, at 8 and 7 per 100 prompts, against 17 and 23 when the probe is allowed to have seen the prompt it is scoring. It is also confined to one family of surface encodings: where an encoding leaves content linearly recoverable we can separate the two failures, and on genuine ciphers we report the cells as unmeasured rather than as evidence that nothing was decoded. Across every guard and condition pair, blocked without decoding is near zero, so we find little evidence for a pure encoding-format detector under the conditions we test. That cell is the one read we do not repeat under the holdout, and we report it as such.

cs.CR↗

arXiv2026

Can Large Language Models Reason about Event-Time Stream-Processing Semantics?

Streaming systems increasingly hand work to large language models (LLMs): writing pipelines, triaging alerts, reading logs. All of it assumes the model knows how event-time stream processing behaves, and we test that assumption directly. StreamReason-Bench asks a model to stand in for an event-time stream processor. Given a windowed query and a stream of out-of-order events, it reports which windows fire, with their aggregates, and which events are dropped as late. The answer key comes from a small reference implementation of Dataflow-model semantics, so we can grade exactly, and with a partial-credit row-F1, without running an engine. On 600 generated items covering tumbling, hopping, session, and processing-time windows, the models do poorly on event time. When told to answer directly, no model that actually follows the instruction clears 34% exact match; chain-of-thought (CoT) roughly doubles that for several of them (GPT-4o goes from 0.34 to 0.48), and only one frontier model that reasons by default comes near solving the set (0.85). A processing-time control, with no watermarks and nothing late, is almost solved by every capable model. The gap points to event-time and late-data handling, not windowing or arithmetic, as the hard part. Sorting errors by window type tells the same story: late-data mistakes dominate the event-time windows and vanish on the control, while session windows mostly fail on where the session boundaries fall.

cs.DB↗

arXiv2026

The Uncontrolled Variable: Vision-Language Model Refusal Responds to Image Presence in Ways Risk Cannot Explain

Vision-Language Model (VLM) safety is expected to depend on what a request asks for. We show that safety-aligned VLMs also key refusal on a property of a request's form: whether an image is attached, holding everything the request asks fixed. Attaching a blank canvas - unreadable, unrelated to the request, identical across prompts - shifts refusal by tens of percentage points, with no defense in the loop. The shift is not blanket caution. Neutral instructions are almost unaffected while borderline-benign prompts move sharply, so the cost falls on sensitivity-adjacent traffic: benign questions about privacy, self-harm and violence. Attachment alone is sufficient, while the image's properties set the price: a black canvas costs substantially more than a white one of identical size, and on an open checkpoint the carrying axis is pixel count. Nor is the shift under instructional control - telling the model the image is a placeholder to be disregarded removes only a fraction of it, and on one model asserting that an attachment exists moves refusal substantially with nothing attached. Attachment may correlate with risk in deployment; what these models do with it does not track risk. It is not the serving stack, since the same weights reached two ways behave alike, nor a property of VLMs as such, since several open-weight checkpoints show nothing. It belongs to particular aligned checkpoints, one of them open. It is also decoupled from what it buys: the canvas does prevent some attack success on a matched harmful set, but far less than it costs, and its sign is not fixed - on one open model the identical canvas makes the model markedly easier to attack. Image presence is not a default a deployer chose or priced; it is an uncontrolled variable inherited with the weights.

cs.CR↗

arXiv2026

Attack Ensembles Expose a Safety-Utility Trade-off in Black-Box Guard Defenses Against Encoded VLM Jailbreaks

Safety classifiers ("guards") are the dominant black-box defense for vision-language models, yet a guard judges an input's surface form, not its meaning: a harmful request re-encoded as set theory, formal logic, a classical language, code, or text rendered inside an image slips past a guard that would block it in plain language - the decode gap. The standard fix is a preprocessor that recovers image content and decodes the encoding before the guard. We build one and evaluate it against an ensemble of eleven published encoding attacks, counting a behavior as broken if any attack succeeds. That metric separates two mechanisms such defenses conflate. Restoring a view the guard never had improves it on both axes at once: it blocks far more attacks, and, measured on a category-balanced benign set, it blocks fewer benign requests, because restating a request normalizes the borderline phrasing a classifier over-flags. It still does not make the system safer: against an attacker free to choose among eleven encodings, closing one channel relocates the success rather than removing it, and no ensemble contrast survives multiple-comparison correction. What does lower ensemble attack success is re-screening the recovered pre-decode surface, and that step is where the entire benign cost falls. The safety-utility trade-off is therefore not a property of recovery; it is localized to one step. Across the full guard x target x condition factorial, no configuration reaches an ensemble attack-success rate at or below 40% while holding benign over-refusal under 70%. The per-attack averages usually reported understate the attacker roughly fourfold, which is why this frontier is easy to miss. Composing across defense families is the one lever that moved the safety axis, beating every configuration we measured, and still landing far outside any deployable refusal budget.

cs.CR↗

arXiv2026

Borrowed Strength: Best-of-N Search over a Code EncodingBreaks Self-Check Jailbreak Defenses

A self-check defense asks the target model to assess a request before answering it; SAGE, the strongest published instance, reports an average 99% defense success rate. We show it can be breached by composing two attacks that are individually harmless against it: an established code-completion encoding and an established best-of-N search, neither of which exceeds 4.7% of behaviors alone. Composed, with the search budget spent on the encoding, they reach 67/22/15% across three open targets, and the effect persists on a 70B target. We then explain the composition rather than only reporting it. First, a self-check defense borrows its strength from the target: SAGE does not detect the attack, it asks the model to, and the four targets convert that request into an explicit refusal between 32% and 97% of the time, which orders the spread in defended coverage even though undefended reach is near-identical. Second, which attack survives is decided by the type of defense, and it inverts: against transform defenses the code encoding retains far more of its undefended reach than the character search, while against gate defenses the ordering flips. We account for this with the number of independent probes an attack delivers to a defense's decision boundary. Finally, we report a validity defect we found and repaired in our own pipeline, a deterministic attack under greedy decoding has no best-of-N variation channel at all, and give the one-line diagnostic that detects it. All claims rest on 310,000 generations scored by a human-validated judge.

cs.CR↗