arXiv Science⌕ Search

arXiv subjects

Yoav Feigenbaum

Publications and source records attributed to Yoav Feigenbaum.

2 recordsLinked to original sources

Sample-Efficient Ensemble Reinforcement Learning at Scale

Ensemble methods such as MeanQ have been shown to improve the sample efficiency of relatively small model-free deep reinforcement learning (RL) architectures, including DQN and similarly sized variants such as Rainbow. However, to our knowledge, the effects of ensembling on large-scale architectures such as Bigger Better Faster (BBF) have not been studied. In this work, we present MeanQ-BBF, a hybrid model-free algorithm that combines the principles of MeanQ and BBF. Through statistically rigorous evaluation on the Atari 100K benchmark, we show that in contrast to naive scaling of the network size of BBF, which degrades performance, MeanQ-BBF is able to leverage the increase in model capacity to improve performance over BBF. Moreover, MeanQ-BBF does this in a computationally efficient way by organizing the added parameters into largely independent learners. Finally, we discuss the performance-cost tradeoff in the context of practical sample-efficient RL.

cs.LG↗

Computing Classical Partition Functions via Cycle-Space Sampling

We present a method for computing thermodynamic properties of classical spin systems by reformulating the partition function as a sum over even subgraphs of the interaction graph. Starting from the high-temperature expansion, we show that the expansion coefficients are determined by the weight distribution of the null space of the binary incidence matrix over $\mathbb{F}_2$, and that this statement holds verbatim for multi-spin interactions once the graph is replaced by a hypergraph. We introduce a Monte Carlo scheme based on the $1/t$ Wang-Landau algorithm that performs a random walk directly in this null space. Because the weight distribution is temperature independent, a single estimate yields the zero-field partition function, and the thermodynamic quantities obtained from its temperature derivatives, at arbitrary temperature. We validate the approach against exact finite-size results on the two-dimensional triangular-lattice Ising model, apply it to the three- and four-dimensional hypercubic Ising models where no exact solution is available, and to the two- and three-dimensional Baxter-Wu models, whose three-spin interactions place them outside the reach of Pfaffian methods. For the two-body models we compare against conventional Wang-Landau sampling in energy space; for the Baxter-Wu models we validate against exact enumeration at the smallest size in each dimension.

cond-mat.stat-mech↗