Deep learning frameworks have often focused on either usability or speed, but not both. PyTorch is a machine learning library that shows that these two goals are in fact compatible: it was designed from first principles to support an imperative and Pythonic programming style that supports code as a model, makes debugging easy and is consistent with other popular scientific computing libraries, while remaining efficient and supporting hardware accelerators such as GPUs. In this paper, we detail the principles that drove the implementation of PyTorch and how they are reflected in its architecture. We emphasize that every aspect of PyTorch is a regular Python program under the full control of its user. We also explain how the careful and pragmatic implementation of the key components of its runtime enables them to work together to achieve compelling performance. We demonstrate the efficiency of individual subsystems, as well as the overall speed of PyTorch on several commonly used benchmarks.

I will increase my score to 9. *Summary* PyTorch is an open-source deeplearning library that strives to marry good performance, flexibility and usability. It is specifically designed for researchers with the goal to enable easy experimenting with new features. Through seamless integration in the Python ecosystem it enables the interoperability with other python libraries which makes prototyping easy for the user. One may argue whether a systems/software paper, presenting the implementation details of a library should be published at NeurIPS, or whether it would be a better fit for USENIX or SysML. However, given the impact of the library in the research community I strongly support the publication of this paper at NeurIPS. PyTorch is specifically designed for the research community, thus of high interest to most attendees.

PyTorch is a useful and impactful package, and has made some interesting and influential design decisions. This paper explains the philosophy behind some of these decisions. Although the framework is not new this year, we still expect broad interest in this exposition, and that having an archival paper will be useful for the community to cite. Given the subject of this paper, the style is different from a typical NeurIPS submission and it needed to be evaluated differently. In particular, many of the claims in the paper are not convincingly tested or demonstrated in the paper as we would normally require.

Deep learning frameworks have often focused on either usability or speed, but not both. PyTorch is a machine learning library that shows that these two goals are in fact compatible: it was designed from first principles to support an imperative and Pythonic programming style that supports code as a model, makes debugging easy and is consistent with other popular scientific computing libraries, while remaining efficient and supporting hardware accelerators such as GPUs. In this paper, we detail the principles that drove the implementation of PyTorch and how they are reflected in its architecture. We emphasize that every aspect of PyTorch is a regular Python program under the full control of its user. We also explain how the careful and pragmatic implementation of the key components of its runtime enables them to work together to achieve compelling performance.

In Haskell, for example, there is a datatype of lists, as values in their own right.

Deep learning frameworks have often focused on either usability or speed, but not both. PyTorch is a machine learning library that shows that these two goals are in fact compatible: it was designed from first principles to support an imperative and Pythonic programming style that supports code as a model, makes debugging easy and is consistent with other popular scientific computing libraries, while remaining efficient and supporting hardware accelerators such as GPUs. In this paper, we detail the principles that drove the implementation of PyTorch and how they are reflected in its architecture. We emphasize that every aspect of PyTorch is a regular Python program under the full control of its user. We also explain how the careful and pragmatic implementation of the key components of its runtime enables them to work together to achieve compelling performance.

One of my favorite things about TensorFlow 2.0 is that it offers multiple levels of abstraction, so you can choose the right one for your project. In this article, I'll explain the tradeoffs between two styles you can use to create your neural networks. The first is a symbolic style, in which you build a model by manipulating a graph of layers. The second is an imperative style, in which you build a model by extending a class. I'll introduce these, share notes on important design and usability considerations, and close with quick recommendations to help you choose the right one.

This repository aims at providing a high performing and flexible deep learning platform, by prototyping a pure NumPy interface above MXNet backend. Why obsessed with NumPy interface? First of all, NumPy is an extension to the Python programming language, with support for large, multi-dimensional arrays, matrices, and a large library of high-level mathematical functions to operate on these abstractions. If you just begin to learn deep learning, you should absolutely start from NumPy to gain a firm grasp of its concepts (see, for example, the Stanford's CS231n course). For quick prototyping of advanced deep learning algorithms, you may often start composing with NumPy as well.