A Class of Rényi-based Concentration Metrics for Kolmogorov-Arnold Geometry in Multilayer Perceptrons

Abstract

The interpretability of artificial intelligence models has become a central concern, especially for black-box models such as multilayer perceptrons (MLPs). Kolmogorov-Arnold Geometry has recently emerged as a framework for analysing the internal representations of these networks, by examining how the hidden layer locally responds to changes in the input through the minors of its Jacobian. The degree to which a few of these minors dominate the rest, known as their concentration, has so far been measured with the Participation Ratio. Although this metric has useful properties, its maximum value depends on the size of the network, so a given value cannot be read as high or low concentration without reference to the architecture. To overcome this limitation, in this paper we propose the $α$-RC metrics, a family of Rényi-based concentration measures that compare the normalized magnitudes of the minors with the uniform distribution through the Rényi divergence of order $α$. All the metrics in the family are bounded between 0 and 1 regardless of the architecture, with 0 indicating that the magnitude is evenly spread over the minors and 1 that it is carried by a single one. Moreover, the metrics are non-decreasing in $α$, and for $α$ = 2 they reduce to an explicit transformation of the Participation Ratio. Computational experiments with single-hidden-layer MLPs trained on a synthetic dataset show the benefits of our proposal.