Prior Action Control Causally Alters Language-Model Plasticity

Abstract

Two copies of one language model start from identical weights and receive the same rewards during training. For one copy the rewards depend on its own answers. The other copy is yoked: it receives the first copy's rewards no matter what it answers, a design taken from the learned-helplessness experiments of animal learning. We ask whether the copy whose rewards depended on its answers, the copy with control, later learns a new task faster, that is, whether control changes the model's plasticity. In Experiment 1, copies of Qwen3-0.6B generated their own answers in a two-turn game and afterwards learned a new task. The copies with control learned the new task faster in every pair, and the advantage held on different starting models as well as on the larger Qwen3-1.7B. Copies with no reward signal, trained only on the updates that every copy shared, learned about as fast as the yoked copies. The advantage is therefore mostly a benefit of control, not damage from lacking it. Control had also changed how the copies answered, so the twins practiced different answers. That practice could carry part of the advantage, and Experiment 2 removed the difference. Both copies of a pair trained on the same answers, fixed in advance, in the same order, and received the same number of rewards. Only one copy's rewards followed the answers, and that copy again learned a new task faster. The advantage of the copies with control over their yoked twins could be forecast before the new task began from each copy's weights. In sum, control over rewards during earlier training left models faster at learning a new task later on, and a conventional model evaluation need not show it.