arXiv Science⌕ Search

arXiv · 2611.14887

Characterizing Local Breadth in Language Generation

Abstract

Large language models sharpen the validity--breadth trade-off: a generator should avoid hallucinations without collapsing onto a narrow set of valid outputs. We study generation from positive examples of an unknown target in the Kleinberg--Mullainathan model, with arbitrary, possibly adversarial order and no distributional or computational assumptions. Breadth is measured by lower Banach density, which rules out arbitrarily large sparse holes, unlike prefix-based lower asymptotic density. 1. Exact optima in one dimension: For each target $K$ in a countable family, we characterize its optimal lower Banach density $b(K)$ via equivalent combinatorial and topological formulations. One eventually valid generator attains all $b(K)$ simultaneously. Every $α\in[0,1/2]$ occurs as $b(K)$ in some countable family; finite Cantor--Bendixson point rank guarantees $b(K)=1/2$, even without a uniform rank bound. 2. Unavoidable trade-offs in higher dimensions: For $d\ge2$, there need not be a single best generator: optimizing density for one target may worsen the guarantee for another. An explicit family quantifies this intrinsically multi-objective trade-off. 3. Complete characterization of density profiles: We characterize the full region of simultaneously achievable guarantees, with equivalent combinatorial and topological formulations, after excluding geometrically degenerate rectangles. Every prescribed product $\prod_i[0,α_i]$, $α_i\in[0,1/2]$ can occur as an exact achievable region for some language collection. 4. Near-half guarantees and the sharp endpoint: Under a near-logarithmic nondegeneracy condition, one generator achieves filtered density at least $1/2-\varepsilon$ at all finite-point-rank targets, for every $\varepsilon>0$, improving the previous polylogarithmic threshold. The exact $1/2$ endpoint can fail even for a rank-zero singleton family.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Jon Kleinberg, Fan Wei. 2026-11-11. Characterizing Local Breadth in Language Generation. https://arxiv.science/abs/2611.14887

Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite arXiv Science by Indiciae: BibTeX · RIS.

KEEP EXPLORING

Related papers

A Black-Box Approach for Exogenous Replenishment in Online Resource Allocation

In a typical online resource allocation problem, we start with a fixed inventory of resources and make online allocation decisions in response to resource requests that arrive sequentially over a finite horizon. We consider settings where the inventory is replenished over time according to an unknown exogenous process. We introduce black-box methods that extend any existing algorithm, originally designed without considering replenishment, into one that works with an arbitrary (adversarial or stochastic) replenishment process. Our approach preserves the original algorithm's competitive ratio in regimes with large initial inventory, thereby enabling the seamless integration of exogenous replenishment into a large body of existing algorithmic results for both adversarial and stochastic arrival models.

cs.DS↗

Min-Max Connected Multiway Cut

We introduce a variant of the multiway cut that we call the min-max connected multiway cut. Given a graph $G=(V,E)$ and a set $Γ\subseteq V$ of $t$ terminals, partition $V$ into $t$ parts such that each part is connected and contains exactly one terminal; the objective is to minimize the maximum weight of the edges leaving any part of the partition. This problem is a natural modification of the standard multiway cut problem and it differs from it in two ways: first, the cost of a partition is defined to be the maximum size of the boundary of any part, as opposed to the sum of all boundaries, and second, the subgraph induced by each part is required to be connected. Although the modified objective function has been considered before in the literature under the name min-max multiway cut, the requirement on each component to be connected has not been studied as far as we know. Our main motivation for studying this problems is its close connection with the spanning tree congestion problem that has been extensively studied but on which little progress has been made. We show various hardness results for this problem, including a proof of weak NP-hardness of the weighted version of the problem on graphs with treewidth two, and $W[1]$-hardness for the problem when parameterized by the treewidth of the graph. Complementing our lower bounds tightly, we also provide a pseudopolynomial time algorithm as well as an FPTAS for the weighted problem on graphs of bounded treewidth. As a consequence of our investigation we also show that the (unconstrained) min-max multiway cut problem is NP-hard even for three terminals, strengthening the known results.

cs.DS↗

PHastR: Faster Minimal Perfect Hashing with Rotated Layouts

We present PHastR, a minimal perfect hash function of the PHast family. Like PHast+, the fast-to-build variant of PHast, PHastR finds feasible seeds with bit-parallel operations: its seeds move keys along rings of slots, and to evaluate the seeds of a bucket it cyclically shifts, for each key, a word recording which slots of its ring are used. Unlike PHast+, however, it is almost immune to self-collisions, which make two keys of a bucket collide for (almost) all its seeds: more than a third of the buckets bumped by PHast+ with wrapping and multiplier 3 contain one. With 8-bit seeds, PHastR uses $1.94$ bits per key, essentially the same space as the best variant of PHast+ with wrapping, but it is built in $37\%$ of the time, and its queries are $34\%$ faster on $10^9$ keys, as it bumps $0.5\%$ of the keys instead of $3.9\%$; they are faster also than those of PHast, which takes almost thirteen times as long to build, and than those of PtrHash, which uses more than one bit per key more. With a larger expected bucket size PHastR comes within $0.02$ bits per key of the space of PHast, using $1.90$ bits per key, and its queries are still about as fast. Its construction also scales better with the number of threads: with all the threads of a 96-core machine it is more than two and a half times faster than that of PHast+. An offline construction built PHastR on a trillion keys in $9.3$ hours on a single machine. Finally, we argue that when each key can be placed only within a range of $W$ slots, every placement of $n$ random keys on $n$ slots, however found, must bump on large key sets a constant fraction of the keys: we derive an asymptotic lower bound of about $0.1/W$ on the expected value of this fraction, and we prove that, in a Poisson model, the best placement bumps about $1/2W$ of the keys, which is about one tenth of the keys bumped by PHastR.

cs.DS↗