Ant swarm functional control via stigmergic Reinforcement Learning agents

Abstract

In this work, we propose a novel framework for the functional controllability of the ant swarm model, a well-known and relevant model of collective behaviour. Our approach introduces a population of controlling stigmergic agents, trained via Reinforcement Learning (RL), that act on the environment to influence the system dynamics and promote the emergence of ordered behaviour. Stigmergic agents are optimized in a centralized-training decentralized-execution setting, interacting with ants only through the shared pheromone field. The reward design promotes trail pheromone structures and alignment of ant positions with high-pheromone paths, without requiring control of specific microscopic configurations. Our results demonstrate that the learned policies effectively shift the phase transition line that characterizes the global behaviour of the system, enabling the emergence of trails scenarios in regimes that are typically dominated by randomness. This study provides insights into the potential of RL based control strategies for complex systems, contributing to the general understanding of functional controllability in this field.