HiMark: Hierarchical and Controllable Multi-bit Watermarking for Large Language Models

Ke Zhu; Yanli Chen; Ling Xiong; Gejian Zhao; Hanzhou Wu

Home/Archive/arXiv

arXiv · 2610.30601

HiMark: Hierarchical and Controllable Multi-bit Watermarking for Large Language Models

Ke Zhu·Yanli Chen·Ling Xiong·Gejian Zhao·Hanzhou Wu

View original source ↗Read source document ↗Export metadata ↓

Abstract

With the rapid adoption of large language models (LLMs) in text generation applications, it has become increasingly difficult to trace generated content and assign responsibility for its use, raising concerns about provenance, copyright protection, and accountability. Multi-bit watermarking addresses these issues by embedding multi-bit provenance information into generated text, but reliable extraction remains difficult due to sparse per-bit evidence and the trade-off between watermark-extraction accuracy and text quality. To deal with these challenges, we propose HiMark, a hierarchical and controllable multi-bit watermarking framework for LLM-generated text. During watermark embedding, HiMark uses a hierarchical candidate-control mechanism, including initial candidate sampling, layer-wise tournament elimination, support-based refinement, and support-threshold filtering. During watermark detection, a model-agnostic evidence aggregation mechanism computes multi-layer support evidence for each observed token, and then accumulates these token-level votes into the corresponding information for robust watermark extraction. Experiments show that HiMark improves watermark extraction accuracy while maintaining text quality, e.g., the watermark extraction accuracy is 90.01% under the paraphrasing attack when 32 bits were embedded within a text containing 300 tokens, demonstrating its effectiveness for provenance tracking.

Explore related subjects

cs.CR

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Ke Zhu, Yanli Chen, Ling Xiong, Gejian Zhao, Hanzhou Wu. 2026-10-22. HiMark: Hierarchical and Controllable Multi-bit Watermarking for Large Language Models. https://arxiv.org/abs/2610.30601

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

How to Backdoor Image Knowledge Distillation

Knowledge distillation is widely used to transfer behavior from a large teacher model to a smaller student. It is often assumed to be safe when the teacher is clean, because classic backdoor attacks rely on poisoned labels and triggers in supervised training, whereas distillation trains the student to match a teacher's outputs. We show that this assumption can fail when the distillation dataset itself is poisoned. Our attack injects triggered and manipulated images that a clean teacher already predicts as an attacker chosen target label, which causes the student to learn a backdoor even though the teacher remains unaffected. We evaluate this threat across multiple manipulation strategies, including targeted adversarial perturbations and targeted GAN based class transitions, and study how distillation settings influence both accuracy and attack success. We show that the attack remains effective even at a 10% poisoning rates. The results demonstrate that a clean teacher alone is not a sufficient safeguard: poisoned distillation data can produce a strongly backdoored student while maintaining competitive performance on clean images. These findings show that the integrity and provenance of distillation data are part of the security boundary of data intensive KD pipelines, even when the teacher itself is trusted.

cs.CR↗

arXiv2026

Computer Science Conferences Should Require Nonrepudiable Experimental Results

This position paper argues that computer science conferences should require tamper-evident, nonrepudiable attestations of experimental results. We name the underlying problem experiment nonrepudiation: a compliant protocol must bind the numbers in a paper to an actual executed computation in a way the author cannot later alter or deny. The current system relies on self-reported checklists, optional code sharing, and author-controlled logging. None of these mechanisms answer the question a reviewer cannot check: did the code the paper describes produce the numbers the paper reports? We define the problem formally, state the security properties any compliant protocol must satisfy, and describe a threat model that includes attacks current approaches do not prevent. We frame the mechanism that provides this property as a proof-of-compute layer, a general requirement that computed results be admitted as evidence only when accompanied by a verifiable record of the computation that produced them. To show that the problem is solvable, we built K-Veritas, the official reference implementation and public infrastructure for experiment nonrepudiation, written in Go, which produces signed reports without accessing training data. K-Veritas is in use today, with a public registry of sealed records; it is not final, and we intend to improve it through community collaboration. We call on conferences and the community to treat nonrepudiation as a major requirement and to help build an open, independent standard for it.

cs.CR↗

arXiv2026

Giskard : Byzantine Robust and Private Aggregation for Large-Scale Decentralized Learning

In decentralized learning, clients train a machine learning model while keeping their data locally and share their model parameters or gradients with a set of neighbors, making it vulnerable to both privacy attacks and model poisoning attacks. Jointly addressing privacy and Byzantine behavior in decentralized learning is challenging. Indeed, Differential Privacy (DP) applied on the client side is known to increase the variance of honest model updates, which weakens Byzantine-robust aggregation against poisoning attacks. Distributed DP can mitigate this issue by adding noise after aggregation, but implementing it without a trusted curator requires both confidential and Byzantine-resilient computation. Recent work addresses this challenge using secure multi-party computation (MPC) to implement robust aggregation while preserving input confidentiality. However, these solutions scale badly: they either require all-to-all communication between participants or delegate the entire computation to a small subset of parties whose computational and communication cost grows with network size. In this paper, we present Giskard, a scalable protocol for private and Byzantine-robust decentralized aggregation. Giskard organizes $n$ parties into a tree of committees of size $O(\log n)$ and evaluates a differentially private coordinate-wise approximate median via a committee-adapted secure quantile search over the value domain, using MPC within each committee. We assess Giskard both theoretically by proving its security and privacy properties and empirically through extensive experiments involving up to one million participants. Compared to its closest competitors, Giskard reduces per-party communication complexity by close to three orders of magnitude over the closest competitor while exhibiting comparable robustness under up to $n/4$ Byzantine parties.

cs.CR↗