arXiv Science⌕ Search

arXiv · 2611.21477

Knowledge Fermentation: Learning from Shared Variation in Few-Shot Classification

Abstract

A classifier built from a handful of labeled examples knows where each class sits in feature space but not how it varies, which is why prototype rules and linear probes overfit in the low-data regime. Knowledge Fermentation (KF) recovers this second-order knowledge by letting classes exchange what they know about variation. Residuals around class prototypes act as carriers; the variance of each direction is measured on classes that took no part in estimating it, and directions above the level of the surrounding medium define a structured noise that acts on a linear classifier as a closed-form change of geometry. A convex objective trains the classifier at a noise strength calibrated on the distances between prototypes, and unlabeled examples, when available, join as carriers in repeated rounds. We prove that the held-out variances are unbiased for the directions on which they are measured, that the fermented map never expands a direction, and that damping caused by wrong assignments limits itself. On 14 datasets from vision and language and 8 frozen encoders, with all constants fixed, KF is on par with the best of six tuned inductive baselines. With unlabeled carriers it raises one-shot accuracy from 52.4% to 65.8% on average and attains the highest average accuracy among nine transductive methods, significantly so on vision encoders, while information maximization stays ahead on sentence encoders. Controlled studies attribute the gain to the task-level geometry and show that it grows with the within-class variation that the classes of a task share.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Ali Mohammadi Ruzbahani, Abbas Yazdinejad, Hadis Karimipour. 2026-11-16. Knowledge Fermentation: Learning from Shared Variation in Few-Shot Classification. https://arxiv.science/abs/2611.21477

Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite arXiv Science by Indiciae: BibTeX · RIS.

KEEP EXPLORING

Related papers

Efficiency of ANS Entropy Encoders

Asymmetric Numeral Systems (ANS) is a class of entropy encoders that had an immense impact on the data compression, substituting arithmetic and Huffman coding. It was studied by different authors but the precise asymptotics of its redundancy (in relation to the entropy) was not completely understood. We obtain optimal bounds for the redundancy of the tabled ANS (tANS), the most popular ANS variant. Given a sequence $a_1,a_2,\ldots,a_n$ of symbols from an alphabet $\{0,1,\ldots,σ-1\}$ such that each symbol $a$ occurs in it $f_a$ times and $n=2^r$, the tANS encoder using Duda's ``precise initialization'' to fill tANS tables transforms this sequence into a bit string of the following length (the frequencies are not included in the encoding): $\sum\limits_{a\in[0..σ)}f_a\cdot\log\frac{n}{f_a}+O(σ+r)$, where $O(σ+r)$ can be bounded by $σ\log e+r$. The $r$-bit term is an artifact indispensable to ANS; the rest incurs a redundancy of $O(\fracσ{n})$ bits per symbol. We complement this by examples showing that an $Ω(σ+r)$ redundancy is necessary. We argue that similar examples exist for most adequate initialization methods for tANS. Thus, we refute Duda's conjecture that the redundancy is $O(\fracσ{n^2})$ bits per symbol. We also propose a variant of the range ANS (rANS), called rANS with fixed accuracy, parameterized by $k\ge 1$ that in certain conditions might be faster than the standard rANS because it avoids slow explicit division operations. We bound the redundancy for our rANS variant by $\frac{n}{2^k-1}\log e+r+k$.

cs.IT↗

A Primer on Zadoff Chu Sequences

Zadoff-Chu (ZC) sequences are an important manifestation of spread spectrum in modern cellular systems, including LTE and 5G NR. They have to some extent displaced PN and Walsh sequences which were the mainstays of 3G cellular (WCDMA and cdma2000) and the 2G-era IS-95. ZC sequences are complex sequences with unit amplitude and particular phase shifts, as opposed to Walsh and PN codes which are real and binary valued, most commonly $\pm1$. ZC sequences have a number of remarkable and desirable properties that we define in the next section. Because of these properties, they are used for many key functions in current cellular systems, and are likely to be prevalent in future cellular systems as well. In LTE and 5G NR, they are widely used for a number of important initial access and overhead channel functions that are often overlooked by engineers who focus on data transmission. For example, ZC sequences are used for initial access in both the downlink (synchronization sequences) and uplink (random access premables). They are also used for transmitting uplink control information, and as pilot symbols for both uplink channel sounding and fine-grained channel estimation. It is not an exaggeration to say that most types of signals other than the data transmissions in modern cellular standards utilize ZC sequences. In this primer, we define ZC sequences and introduce their key properties, and provide some examples. We also discuss modified ZC sequences that are commonly used in practice, but are not, strictly speaking, ZC sequences. We also overview their uses in LTE and 5G.

cs.IT↗

Frames over finite fields with respect to the Galois inner product and their connections to self-orthogonal and LCD codes

In this paper, we generalize the theory of frames to vector spaces over finite fields equipped with the $\ell$-Galois inner product. We introduce $\ell$-Galois analogues of fundamental concepts in frame theory and investigate extensions of existing results to the $\ell$-Galois setting. In particular, we establish necessary and sufficient conditions for a given matrix to be the $\ell$-Galois Gram matrix of a frame and show that such frames are unique up to isometry. We also extend Gerzon's bound to $\ell$-Galois equiangular systems and show that equality in this bound yields an $\ell$-Galois equiangular tight frame. Furthermore, we investigate the connections between $\ell$-Galois frames and $\ell$-Galois self-orthogonal and LCD codes over finite fields. We establish necessary and sufficient conditions for constructing $\ell$-Galois equiangular tight frames from self-dual codes, generalizing existing results by Shi et al. on the relationship between frames and self-dual codes over finite fields. Beyond self-dual codes, we provide sufficient conditions for constructing $\ell$-Galois equiangular tight frames from linear codes using their automorphism groups, together with explicit examples.

cs.IT↗