arXiv subjects

Athanasios Sideris

Publications and source records attributed to Athanasios Sideris.

6 recordsLinked to original sources

arXiv2025

Online Training and Pruning of Deep Reinforcement Learning Networks

Scaling deep neural networks (NN) of reinforcement learning (RL) algorithms has been shown to enhance performance when feature extraction networks are used but the gained performance comes at the significant expense of increased computational and memory complexity. Neural network pruning methods have successfully addressed this challenge in supervised learning. However, their application to RL is underexplored. We propose an approach to integrate simultaneous training and pruning within advanced RL methods, in particular to RL algorithms enhanced by the Online Feature Extractor Network (OFENet). Our networks (XiNet) are trained to solve stochastic optimization problems over the RL networks' weights and the parameters of variational Bernoulli distributions for 0/1 Random Variables $\xi$ scaling each unit in the networks. The stochastic problem formulation induces regularization terms that promote convergence of the variational parameters to 0 when a unit contributes little to the performance. In this case, the corresponding structure is rendered permanently inactive and pruned from its network. We propose a cost-aware, sparsity-promoting regularization scheme, tailored to the DenseNet architecture of OFENets expressing the parameter complexity of involved networks in terms of the parameters of the RVs in these networks. Then, when matching this cost with the regularization terms, the many hyperparameters associated with them are automatically selected, effectively combining the RL objectives and network compression. We evaluate our method on continuous control benchmarks (MuJoCo) and the Soft Actor-Critic RL agent, demonstrating that OFENets can be pruned considerably with minimal loss in performance. Furthermore, our results confirm that pruning large networks during training produces more efficient and higher performing RL agents rather than training smaller networks from scratch.

cs.LG↗

arXiv2024

Complexity-Aware Training of Deep Neural Networks for Optimal Structure Discovery

We propose a novel algorithm for combined unit and layer pruning of deep neural networks that functions during training and without requiring a pre-trained network to apply. Our algorithm optimally trades-off learning accuracy and pruning levels while balancing layer vs. unit pruning and computational vs. parameter complexity using only three user-defined parameters, which are easy to interpret and tune. We formulate a stochastic optimization problem over the network weights and the parameters of variational Bernoulli distributions for binary Random Variables taking values either 0 or 1 and scaling the units and layers of the network. Optimal network structures are found as the solution to this optimization problem. Pruning occurs when a variational parameter converges to 0 rendering the corresponding structure permanently inactive, thus saving computations both during training and prediction. A key contribution of our approach is to define a cost function that combines the objectives of prediction accuracy and network pruning in a computational/parameter complexity-aware manner and the automatic selection of the many regularization parameters. We show that the proposed algorithm converges to solutions of the optimization problem corresponding to deterministic networks. We analyze the ODE system that underlies our stochastic optimization algorithm and establish domains of attraction for the dynamics of the network parameters. These theoretical results lead to practical pruning conditions avoiding the premature pruning of units and layers during training. We evaluate our method on the CIFAR-10/100 and ImageNet datasets using ResNet architectures and demonstrate that it gives improved results with respect to pruning ratios and test accuracy over layer-only or unit-only pruning and favorably competes with combined unit and layer pruning algorithms requiring pre-trained networks.

cs.LG↗

arXiv2024

Concurrent Training and Layer Pruning of Deep Neural Networks

We propose an algorithm capable of identifying and eliminating irrelevant layers of a neural network during the early stages of training. In contrast to weight or filter-level pruning, layer pruning reduces the harder to parallelize sequential computation of a neural network. We employ a structure using residual connections around nonlinear network sections that allow the flow of information through the network once a nonlinear section is pruned. Our approach is based on variational inference principles using Gaussian scale mixture priors on the neural network weights and allows for substantial cost savings during both training and inference. More specifically, the variational posterior distribution of scalar Bernoulli random variables multiplying a layer weight matrix of its nonlinear sections is learned, similarly to adaptive layer-wise dropout. To overcome challenges of concurrent learning and pruning such as premature pruning and lack of robustness with respect to weight initialization or the size of the starting network, we adopt the "flattening" hyper-prior on the prior parameters. We prove that, as a result of its usage, the solutions of the resulting optimization problem describe deterministic networks with parameters of the posterior distribution at either 0 or 1. We formulate a projected SGD algorithm and prove its convergence to such a solution using stochastic approximation results. In particular, we prove conditions that lead to a layer's weights converging to zero and derive practical pruning conditions from the theoretical results. The proposed algorithm is evaluated on the MNIST, CIFAR-10 and ImageNet datasets and common LeNet, VGG16 and ResNet architectures. The simulations demonstrate that our method achieves state-of the-art performance for layer pruning at reduced computational cost in distinction to competing methods due to the concurrent training and pruning.

cs.LG↗

arXiv2022

Robust Learning of Parsimonious Deep Neural Networks

We propose a simultaneous learning and pruning algorithm capable of identifying and eliminating irrelevant structures in a neural network during the early stages of training. Thus, the computational cost of subsequent training iterations, besides that of inference, is considerably reduced. Our method, based on variational inference principles using Gaussian scale mixture priors on neural network weights, learns the variational posterior distribution of Bernoulli random variables multiplying the units/filters similarly to adaptive dropout. Our algorithm, ensures that the Bernoulli parameters practically converge to either 0 or 1, establishing a deterministic final network. We analytically derive a novel hyper-prior distribution over the prior parameters that is crucial for their optimal selection and leads to consistent pruning levels and prediction accuracy regardless of weight initialization or the size of the starting network. We prove the convergence properties of our algorithm establishing theoretical and practical pruning conditions. We evaluate the proposed algorithm on the MNIST and CIFAR-10 data sets and the commonly used fully connected and convolutional LeNet and VGG16 architectures. The simulations show that our method achieves pruning levels on par with state-of the-art methods for structured pruning, while maintaining better test-accuracy and more importantly in a manner robust with respect to network initialization and initial size.

cs.LG↗

arXiv2021

Physics-Aware Neural Network Flame Closure for Combustion Instability Modeling in a Single-Injector Engine

Neural networks (NN) are implemented as sub-grid flame models in a large-eddy simulation of a single-injector liquid-propellant rocket engine with the aim to replace a look-up table approach. The NN training process presents an extraordinary challenge. The multi-dimensional combustion instability problem involves multi-scale lengths and characteristic times in an unsteady flow problem with nonlinear acoustics, addressing both transient and dynamic-equilibrium behaviors, superimposed on a turbulent reacting flow with very narrow, moving flame regions. Accurate interpolation between the points of the training data becomes vital. A major novel aspect of the proposed NNs is that they are trained to reproduce relevant portions of the information stored in a flamelet table by using only limited data from a few CFD simulations of a single-injector liquid-propellant rocket engine under different dynamical configurations. This is made possible by enriching the training set with contrived data resulting from the physical characteristics of the combustion model and also by including the flame temperature as an extra input to the NNs that are trained to model other flame variables of interest. These physics-aware NN-based closure models are first tested offline by comparing them directly with the flamelet table and then are successfully implemented into CFD simulations in place of the flamelet table and verified on various dynamical configurations. The results from those tests compare favorably with counterpart table-based CFD simulations. Computational advantages of the approach are discussed.

physics.flu-dyn↗

arXiv2020

Neural Network Flame Closure for a Turbulent Combustor with Unsteady Pressure

In this paper, neural network (NN)-based models are generated to replace flamelet tables for sub-grid modeling in large-eddy simulations of a single-injector liquid-propellant rocket engine. In the most accurate case, separate NNs for each of the flame variables are designed and tested by comparing the NN output values with the corresponding values in the table. The gas constant, internal flame energy, and flame heat capacity ratio are estimated with 0.0506%, 0.0852%, and 0.0778% error, respectively. Flame temperature, thermal conductivity, and the coefficient of heat capacity ratio are estimated with 0.63%, 0.68%, and 0.86% error, respectively. The progress variable reaction rate is also estimated with 3.59% error. The errors are calculated based on mean square error over all points in the table. The developed NNs are successfully implemented within the CFD simulation, replacing the flamelet table entirely. The NN-based CFD is validated through comparison of its results with the table-based CFD.

physics.flu-dyn↗