Not All 4-bit Quantizers Are Equal: Deployment-Time Mitigation of PII Leakage in Fine-Tuned Small Language Models

Cristhian Kapelinski; Diego Kreutz

Home/Archive/arXiv

arXiv · 2609.25014

Not All 4-bit Quantizers Are Equal: Deployment-Time Mitigation of PII Leakage in Fine-Tuned Small Language Models

Cristhian Kapelinski·Diego Kreutz

View original source ↗Read source document ↗Export metadata ↓

Abstract

Organizations fine-tune small language models on private data and then compress them to 4 bits for resource-efficient deployment. We show that the compression method also affects privacy. What separates the methods is not the bit width but whether they tune their rounding on a small sample of text, the calibration corpus. On our primary model, when each planted record's own opening text is used as the prompt, the two calibration-based methods we test, Activation-aware Weight Quantization (AWQ) and Gradient-based Post-Training Quantization (GPTQ), each reproduce none of the planted records, while the calibration-corpus-free GGUF Q4_K_M format reproduces 5.3% of them. Tracked across five open models with 0.5-7 billion parameters, AWQ leaks least at every size and in both families, with little accuracy loss at 3-7 billion. Controlled experiments associate the difference with calibration-induced rounding error in channels involved in rare-token prediction. Choosing the 4-bit method is therefore a deployment-time privacy decision, not only a question of speed and quality.

Explore related subjects

cs.CR cs.AI cs.LG

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Cristhian Kapelinski, Diego Kreutz. 2026-08-02. Not All 4-bit Quantizers Are Equal: Deployment-Time Mitigation of PII Leakage in Fine-Tuned Small Language Models. https://arxiv.org/abs/2609.25014

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Erased but Not Forgotten: How Backdoors Compromise Concept Erasure

The expansion of text-to-image diffusion models has raised concerns about harmful outputs, from fabricated depictions of public figures to sexually explicit imagery. To mitigate such risks, prior work has proposed concept erasure methods that aim to sever unwanted concepts from the model via fine-tuning, yet it remains unclear whether these approaches truly remove all links to the harmful concept or merely conceal superficial connections. In this work, we reveal a critical vulnerability, the Erasure Evasion Backdoor (EEB): an adversary binds a backdoor trigger to a concept slated for removal, and this malicious link survives subsequent erasure. We show that both black-box and white-box adversaries can instantiate this threat. Across six state-of-the-art erasure methods, including robust ones that explicitly search for alternative representations of the target concept, EEB consistently exposes harmful content: up to 82% success against celebrity-identity unlearning, up to 94% for object erasure, and up to 16 times amplification of explicit-content exposure. While EEB uncovers a blind spot in current erasure methods, it also provides a diagnostic tool for stress-testing future concept erasure techniques. Our code is available at https://github.com/multimodal-ai-lab/EEB.

cs.CR↗

arXiv2026

Condense to Conduct and Conduct to Condense

In this paper, we present the first explicit examples of low-conductance permutations. The notion of conductance of permutations was introduced by Dodis et al. in "Indifferentiability of Confusion-Diffusion Networks", where the search for low-conductance permutations was first initiated and motivated. As part of our contribution, we not only provide these examples, but also offer a general characterization of the problem: we show that low-conductance permutations are equivalent to permutations possessing the information-theoretic properties of Multi-Source-Somewhere-Condensers, a specific variant of somewhere condensers.

cs.CR↗

arXiv2026

Potential and Challenges of Large Language Models for Reverse Engineering

Reverse engineering (RE) is central to cybersecurity, supporting tasks such as decompilation, deobfuscation, and security analysis. However, RE remains labor-intensive and expertise-demanding, as analysts often manually recover high-level semantics from low-level program representations. Recent advances in large language models (LLMs) provide a promising way to address these challenges through program artifact understanding, semantic reasoning, and tool-augmented problem solving. This potential has stimulated interest in LLM-assisted RE, but existing studies remain scattered across different tasks, targets, methodologies, and evaluation practices. Despite these advances, the literature still lacks a comprehensive survey that consolidates progress, systematizes technical choices, and clarifies open challenges and future opportunities. To fill this gap, we present a systematic survey of LLM-assisted RE, covering 48 peer-reviewed and published research articles identified through our search and selection protocol as of July 1, 2026. We develop a faceted taxonomy that organizes prior studies along six dimensions: task objective, analysis target, methodological approach, evaluation protocol, training scale, and data quality. We extract task formulations and experimental settings from existing studies to support comparison, reproducibility, and future research. From this review, we synthesize research gaps, characterize key challenges, and outline future directions toward more reliable, reproducible, and security-relevant applications of LLMs in RE.

cs.CR↗