arXiv Science⌕ Search

arXiv · 2611.21710

Learning In-Context Adaptivity for LLMs through Continual Gameplay

Abstract

LLM agents are increasingly deployed in long-horizon tasks where they must explore, gather experience, and adapt their decisions accordingly. We name this capability in-context adaptivity: improving decisions across episodes from experience accumulated in context. We propose to evaluate it in repeated imperfect-information games against unknown opponents. In these games, fixed game-theoretically optimal policies cannot achieve maximum reward, whereas adapting to the opponent increases returns across games. We design a paired protocol that evaluates each agent with and without memory under identical conditions to isolate experience gain from single-game skill. Evaluation shows that memory brings little performance gain for open-weight models and inconsistent gains for frontier models. Our analysis of the agents' summaries and actions identifies a memory-doing gap in which models accurately record opponent tendencies yet rarely depart from default play. To close this gap, we construct a pool of exploitable opponents and post-train Qwen3.5-9B with supervised fine-tuning followed by trajectory-level reinforcement learning. We optimize full-match return across context segments so summary generation and action selection are trained together. On unseen opponents, our 9B RL policy doubles experience gain over supervised fine-tuning in Leduc Hold'em and increases it fivefold in No-Limit Hold'em. Across 35 opponents, our policy matches or surpasses frontier models, generalizes beyond its training horizon, and retains most of its gain on summaries from frontier models.

Explore related subjects

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Jiale Liu, Zun Li, John Schultz. 2026-11-16. Learning In-Context Adaptivity for LLMs through Continual Gameplay. https://arxiv.science/abs/2611.21710

Citations retain the original authors and DOI and include this site’s record URL. If this discovery service contributed to your research, you can also cite arXiv Science by Indiciae: BibTeX · RIS.

KEEP EXPLORING

Related papers

On the closest balanced game

Cooperative games with nonempty core are called balanced, and the set of balanced games is a polyhedron. Given a game with empty core, we look for the closest balanced game, in the sense of the (weighted) Euclidean distance, i.e., the orthogonal projection of the game on the set of balanced games. Besides an analytical approach which becomes rapidly intractable, we propose a fast algorithm to find the closest balanced game, avoiding exponential complexity for the optimization problem, and being able to run up to 20 players. We show experimentally that the probability that the closest game has a core reduced to a singleton tends to 1 when the number of players grow. We provide a mathematical proof that the proportion of facets whose games have a non-singleton core tends to 0 when the number of players grow, by finding an expression of the aymptotic growth of the number of minimal balanced collections. This permits to prove mathematically the experimental result. Consequently, taking the core of the projected game defines a new solution concept, which we call least square core due to its analogy with the least core, and our result shows that the probability that this is a point solution tends to 1 when the number of players grow.

cs.GT↗

Concave is the New Linear: Sybil Attacks Collapse Anti-Plutocratic DAO Voting

Decentralized Autonomous Organizations (DAOs) run protocol governance by letting token holders vote on proposals. The dominant rule, voting power proportional to wallet balance, concentrates control among a small number of large holders, fueling the token-control governance attacks that have already compromised real protocols. To counter this concentration, the community has turned to anti-plutocratic voting mechanisms such as Quadratic Voting (QV), which assign sublinear voting power per token with the goal of dampening the influence of large holders. We prove that no voting rule that derives power solely from wallet balance can succeed on a permissionless blockchain. Through a costed model of on-chain voting that captures realistic blockchain frictions -- including per-wallet splitting and voting costs, fixed setup costs, and minimum-balance requirements -- we show that whenever a wallet of any size yields nonzero voting power, a Sybil attacker who splits tokens across many wallets achieves total voting power that grows at least linearly in their token holdings. For concave rules actually proposed to dampen governance power -- those that are positive, increasing, and finite -- we show that the optimal strategy yields power that is asymptotically linear in token holdings, regardless of the cost scheme. Instantiating the model on real DAOs reveals attack costs orders of magnitude below the value at stake. Replaying the ten most recent finalized proposals of five major DAOs (ENS, Compound, Uniswap, Arbitrum, and ZKsync) under linear, quadratic, logarithmic, and power-($β= 0.25$) voting, we measure Sybil amplification factors between $1,172\times$ and $4,039\times$ under Quadratic Voting, and exceeding $229,000\times$ under steeper power rules.

cs.GT↗

Equilibrium in Multi-Agent Reinforcement Learning

Standard solution concepts for stochastic games, such as MPE and MCCE, are computationally difficult, and thus, standard decentralized reinforcement-learning algorithms should not generally be expected to converge to them. In this paper, we study the equilibrium generated by such algorithms. In particular, we introduce a new solution concept for stochastic games, Markov Bayes coarse correlated equilibrium (MBCCE), defined as a distribution over states and stationary policy profiles such that, after observing the state but before observing her recommended action, no player can gain by choosing a different current action, with the sampled policy profile governing play thereafter. We discuss the parallels between MBCCE and coarse correlated equilibrium (CCE) in finite normal-form games and show that MBCCE retains several of its key properties. We then introduce a corresponding regret notion, adaptive Markov coarse regret (AMCR), and show that vanishing AMCR implies that every accumulation point of the empirical distribution of realized states and policy profiles is an MBCCE. Crucially, we show that achieving AMCR reduces to two standard learning tasks: minimizing external regret at each state and accurately evaluating the current joint policy. We then prove that under mild conditions two natural RL algorithmic designs achieve vanishing AMCR: a decentralized asynchronous actor--critic algorithm through a new two-timescale stochastic-approximation analysis, and a standard episodic multi-agent projected policy-gradient method. Hence, both algorithms approach the MBCCE set, and we establish explicit finite-time convergence rates for both. Finally, we relate MBCCE to stationary MCCE and use this connection together with our algorithmic guarantees to identify a natural class of stochastic games in which these algorithms achieve approximate MCCE.

cs.GT↗