arXiv ScienceSearch

arXiv · 1605.02688

Theano: A Python framework for fast computation of mathematical expressions

The Theano Development Team·Rami Al-Rfou·Guillaume Alain·Amjad Almahairi·Christof Angermueller·Dzmitry Bahdanau·Nicolas Ballas·Frédéric Bastien·Justin Bayer·Anatoly Belikov·Alexander Belopolsky·Yoshua Bengio·Arnaud Bergeron·James Bergstra·Valentin Bisson·Josh Bleecher Snyder·Nicolas Bouchard·Nicolas Boulanger-Lewandowski·Xavier Bouthillier·Alexandre de Brébisson·Olivier Breuleux·Pierre-Luc Carrier·Kyunghyun Cho·Jan Chorowski·Paul Christiano·Tim Cooijmans·Marc-Alexandre Côté·Myriam Côté·Aaron Courville·Yann N. Dauphin·Olivier Delalleau·Julien Demouth·Guillaume Desjardins·Sander Dieleman·Laurent Dinh·Mélanie Ducoffe·Vincent Dumoulin·Samira Ebrahimi Kahou·Dumitru Erhan·Ziye Fan·Orhan Firat·Mathieu Germain·Xavier Glorot·Ian Goodfellow·Matt Graham·Caglar Gulcehre·Philippe Hamel·Iban Harlouchet·Jean-Philippe Heng·Balázs Hidasi·Sina Honari·Arjun Jain·Sébastien Jean·Kai Jia·Mikhail Korobov·Vivek Kulkarni·Alex Lamb·Pascal Lamblin·Eric Larsen·César Laurent·Sean Lee·Simon Lefrancois·Simon Lemieux·Nicholas Léonard·Zhouhan Lin·Jesse A. Livezey·Cory Lorenz·Jeremiah Lowin·Qianli Ma·Pierre-Antoine Manzagol·Olivier Mastropietro·Robert T. McGibbon·Roland Memisevic·Bart van Merriënboer·Vincent Michalski·Mehdi Mirza·Alberto Orlandi·Christopher Pal·Razvan Pascanu·Mohammad Pezeshki·Colin Raffel·Daniel Renshaw·Matthew Rocklin·Adriana Romero·Markus Roth·Peter Sadowski·John Salvatier·François Savard·Jan Schlüter·John Schulman·Gabriel Schwartz·Iulian Vlad Serban·Dmitriy Serdyuk·Samira Shabanian·Étienne Simon·Sigurd Spieckermann·S. Ramana Subramanyam·Jakub Sygnowski·Jérémie Tanguay·Gijs van Tulder

Abstract

Theano is a Python library that allows to define, optimize, and evaluate mathematical expressions involving multi-dimensional arrays efficiently. Since its introduction, it has been one of the most used CPU and GPU mathematical compilers - especially in the machine learning community - and has shown steady performance improvements. Theano is being actively and continuously developed since 2008, multiple frameworks have been built on top of it and it has been used to produce many state-of-the-art machine learning models. The present article is structured as follows. Section I provides an overview of the Theano software and its community. Section II presents the principal features of Theano and how to use them, and compares them with other similar projects. Section III focuses on recently-introduced functionalities and improvements. Section IV compares the performance of Theano against Torch7 and TensorFlow on several machine learning models. Section V discusses current limitations of Theano and potential ways of improving it.

Explore related subjects

Keep this discovery

BibTeXRIS

The Theano Development Team, Rami Al-Rfou, Guillaume Alain, Amjad Almahairi, Christof Angermueller, Dzmitry Bahdanau, Nicolas Ballas, Frédéric Bastien, Justin Bayer, Anatoly Belikov, Alexander Belopolsky, Yoshua Bengio, Arnaud Bergeron, James Bergstra, Valentin Bisson, Josh Bleecher Snyder, Nicolas Bouchard, Nicolas Boulanger-Lewandowski, Xavier Bouthillier, Alexandre de Brébisson, Olivier Breuleux, Pierre-Luc Carrier, Kyunghyun Cho, Jan Chorowski, Paul Christiano, Tim Cooijmans, Marc-Alexandre Côté, Myriam Côté, Aaron Courville, Yann N. Dauphin, Olivier Delalleau, Julien Demouth, Guillaume Desjardins, Sander Dieleman, Laurent Dinh, Mélanie Ducoffe, Vincent Dumoulin, Samira Ebrahimi Kahou, Dumitru Erhan, Ziye Fan, Orhan Firat, Mathieu Germain, Xavier Glorot, Ian Goodfellow, Matt Graham, Caglar Gulcehre, Philippe Hamel, Iban Harlouchet, Jean-Philippe Heng, Balázs Hidasi, Sina Honari, Arjun Jain, Sébastien Jean, Kai Jia, Mikhail Korobov, Vivek Kulkarni, Alex Lamb, Pascal Lamblin, Eric Larsen, César Laurent, Sean Lee, Simon Lefrancois, Simon Lemieux, Nicholas Léonard, Zhouhan Lin, Jesse A. Livezey, Cory Lorenz, Jeremiah Lowin, Qianli Ma, Pierre-Antoine Manzagol, Olivier Mastropietro, Robert T. McGibbon, Roland Memisevic, Bart van Merriënboer, Vincent Michalski, Mehdi Mirza, Alberto Orlandi, Christopher Pal, Razvan Pascanu, Mohammad Pezeshki, Colin Raffel, Daniel Renshaw, Matthew Rocklin, Adriana Romero, Markus Roth, Peter Sadowski, John Salvatier, François Savard, Jan Schlüter, John Schulman, Gabriel Schwartz, Iulian Vlad Serban, Dmitriy Serdyuk, Samira Shabanian, Étienne Simon, Sigurd Spieckermann, S. Ramana Subramanyam, Jakub Sygnowski, Jérémie Tanguay, Gijs van Tulder. 2016-05-09. Theano: A Python framework for fast computation of mathematical expressions. https://arxiv.org/abs/1605.02688

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related papers

Diversity of EML-type operators

The discovery of the EML operator, sufficient to evaluate the standard explicit purely transcendental elementary functions, has led to considerable interest and discussion across multiple scientific disciplines. However, most authors have focused on the binary EML itself, while numerous similar variants with slightly different properties are now known. This article attempts to close this gap by enumerating and classifying them. We also take this opportunity to clarify common misconceptions related to the EML operator. The principal goal, symbolic regression within an architecture as close as possible to proven neural networks which combine matrix multiplication with a single univariate non-linear activation function, remains beyond reach. Instead, we propose a M\"obius layer, with rational functions replacing matrix operations, and showcase the recently discovered activation function eml(x,1/x), which allows exp(x) and ln(x) to be recovered separately, and hence all elementary functions to be evaluated within a rational generalization of the neural network.

cs.SC

Physical Law Ecology: mapping multi-mechanism ecologies as the zeroth step of data-driven scientific discovery

Every data-driven equation discovery method assumes (implicitly and without verification) that the target system obeys a single governing law ($K{=}1$). Here we show that this assumption is the primary bottleneck limiting scientific discovery in multi-mechanism systems, and introduce Physical Law Ecology, a framework that makes $K^*$ (the number of coexisting independent mechanisms) itself the first quantity to be determined from data. The framework automatically mines a pool of topologically distinct candidate equations, constructs a continuous dominance weight field across parameter space, and discovers analytic evolution laws governing mechanism succession---with optional monotonicity constraints encoding irreversible physics. Across four unrelated systems (elastomer mechanics, pool boiling, galactic dynamics, and droplet evaporation), BIC consistently identifies $K^*{=}3$ independent governing topologies. Applied to 163 SPARC galaxies (3,269 spatially resolved measurements), the framework autonomously recovers three gravitational laws whose coexistence provides evidence against the single-universal-acceleration hypothesis of MOND ($p<10^{-34}$). In engineering applications, multi-law weighted prediction reduces error by 67-72\% over single-equation baselines while retaining full interpretability. By establishing the determination of $K^*$ as the zeroth step of scientific discovery-prior to and independent of equation search---this work opens a direction orthogonal to existing symbolic regression: not finding better equations, but mapping the ecology of mechanisms that govern complex systems.

cs.SC