Communication-Learning Co-Design for Differentially Private Over-the-Air Federated Distillation

Zihao Hu; Jia Yan; Ying-Jun Angela Zhang

Home/Archive/arXiv

arXiv · 2508.06557

Communication-Learning Co-Design for Differentially Private Over-the-Air Federated Distillation

Zihao Hu·Jia Yan·Ying-Jun Angela Zhang

View original source ↗Read source document ↗Export metadata ↓

Abstract

The ever-growing learning model size nowadays challenges the communication efficiency and privacy preservation of the traditional federated learning (FL). In this paper, we propose a novel differentially private (DP) over-the-air federated distillation (FD) framework, where wireless devices (WDs) periodically share noise-perturbed model outputs with the parameter server by harnessing the superposition property of multi-access channels. Accordingly, over-the-air FD enables the shared responsibility of the DP preservation on the low-dimensional disclosed signals among WDs. We study the communication-learning co-design problem in differentially private over-the-air FD, aiming to maximize the learning convergence rate while meeting the transmit power and DP requirements of WDs. The main challenge is rooted in the intractable learning and privacy analysis in over-the-air FD, together with the strong coupling among the decision variables spanning two timescales. To tackle this problem, we first derive the analytical learning convergence rate and privacy losses of WDs, based on which the optimal transceiver design per FD round and long-term training rounds decision are obtained in the closed forms. Numerical results demonstrate that the proposed differentially private over-the-air FD approach achieves a better learning-privacy trade-off with largely-reduced communication overhead than the conventional FL benchmarks.

Explore related subjects

cs.IT cs.LG eess.SP math.IT

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

Zihao Hu, Jia Yan, Ying-Jun Angela Zhang. 2025-08-06. Communication-Learning Co-Design for Differentially Private Over-the-Air Federated Distillation. https://arxiv.org/abs/2508.06557

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

arXiv2026

Universal Feature Selection with Noisy Observations and Weak Symmetry Conditions

This paper relaxes the restrictive symmetry conditions adopted in [4], [5] and extends their universal feature selection framework to accommodate noisy observations as well as attribute structures that may exhibit directional preferences. We introduce the notion of weak spherical symmetry, quantified by second-moment distances, which allows controlled deviations from rotational invariance. Under this relaxed condition, we develop a universal feature selection framework based on the singular value decomposition of the canonical dependence matrix computed from noisy data. Our main result shows that the selected features achieve asymptotically optimal error exponents up to a residual term that depends on the symmetry deviation $δ$ and the noise levels $η_1, η_2$. When $δ, η_1, η_2$ are relatively small, our result recovers that of [5], thereby demonstrating that exact spherical symmetry is unnecessary. Overall, our findings highlight the robustness of the selection framework against second-moment deviations and observation noise, thereby broadening its applicability across diverse inference tasks and providing a theoretically grounded tool for universal feature selection in practical scenarios.

cs.IT↗

arXiv2026

CPM-LDPC Codes Attaining the Minimum-Distance Bound

We study binary quasi-cyclic LDPC codes whose parity-check matrices are full arrays of single circulant permutation matrices (CPMs), referred to here as CPM-LDPC codes. Their minimum distance is at most $(J+1)!$, where $J$ is the column weight. For every fixed pair of column and row weights $2\le J<L$, we show that this bound is attained for all sufficiently large integer lift sizes. First, we give one integer exponent matrix independent of the lift size $P$. Second, we show that independent uniform exponent choices attain the bound with probability $1-O_{J,L}(P^{-1})$. Both proofs use cycle conditions required by low-weight codewords and a lower bound on the number of terms in vectors satisfying polynomial check equations. Neither construction requires $P$ to be prime. We also give small-lift arrays attaining the bound 24 for $J=3$, $L=4,\ldots,8$, and arrays with distance at least 28 for $J=4$, $L=5,\ldots,8$, together with computational distance verification.

cs.IT↗

arXiv2026

On Sequence Reconstruction Problem for q-ary Deletion Channels

The sequence reconstruction problem for $q$-ary deletion channels, introduced by Levenshtein in 2001, concerns the minimum number of channels required to uniquely recover a transmitted sequence when each channel introduces exactly $t$ deletions. Combinatorially, it is equivalent to determining $N_q(n,d,t)$, the maximum intersection size of two $t$-deletion balls with centers at Levenshtein distance at least $d$, for $q$-ary sequences of length $n$ over the alphabet $Σ_q=\{0,1,\dots,q-1\}$. Levenshtein solved the uncoded case $N_q(n,1,t)$ for all $n\ge t$; subsequently, Gabrys and Yaakobi determined $N_2(n,2,t)$, and Wang et al. extended the result to $N_3(n,2,t)$. In this paper, we study the problem for $q$-ary sequences under minimum Levenshtein distance $d=2$ with channels that introduce exactly $t$ deletions. We determine the exact value of $N_q(n,2,t)$ for all $t\ge 2, q\geq 4$, and for sufficiently large $n$, and construct explicit pairs of sequences attaining the maximum intersection. Furthermore, for each $q\ge3$, we characterize all extremal sequence pairs. In particular, if the intersection size matches the first two terms of $N_q(n,2,t)$, then the two center sequences must contain, at the same positions, length-5 blocks of the forms $(a,b,c,a,b)$ and $(b,a,c,b,a)$ for some distinct $a,b,c\inΣ_q$; for $t\ge q+2$, the exact maximum $N_q(n,2,t)$ is attained precisely by $2q!$ unordered pairs of sequences with a specific block structure. Asymptotically, we prove that for $q\ge 4$ and $t\ge 2$, \[ N_q(n,2,t)=\frac{6}{(t-2)!}n^{t-2}-\frac{3t+13}{(t-3)!}n^{t-3}+\frac{3t^2+25t+64}{4(t-4)!}n^{t-4}+O(n^{t-5}). \] Moreover, $N_q(n,2,t)$ and $N_{q-1}(n,2,t)$ share their first $q-1$ terms, and for $t\ge q$ the coefficient of $n^{t-q}$ in their difference is $\frac{6t-6q+5}{(t-q)!}$.

cs.IT↗