Deep Learning (DL) frameworks such as PyTorch and TensorFlow include runtime infrastructures responsible for executing trained models on target hardware, managing memory, data transfers, and multi-accelerator execution, if applicable. Additionally, it is a common practice to deploy pre-trained models on environments distinct from their native development settings. This led to the introduction of interchange formats such as ONNX, which includes its runtime infrastructure, and ONNX Runtime, which work as standard formats that can be used across diverse DL frameworks and languages. Even though these runtime infrastructures have a great impact on inference performance, no previous paper has investigated their energy efficiency. In this study, we monitor the energy consumption and inference time in the runtime infrastructures of three well-known DL frameworks as well as ONNX, using three various DL models. To have nuance in our investigation, we also examine the impact of using different execution providers. We find out that the performance and energy efficiency of DL are difficult to predict. One framework, MXNet, outperforms both PyTorch and TensorFlow for the computer vision models using batch size 1, due to efficient GPU usage and thus low CPU usage. However, batch size 64 makes PyTorch and MXNet practically indistinguishable, while TensorFlow is outperformed consistently. For BERT, PyTorch exhibits the best performance. Converting the models to ONNX yields significant performance improvements in the majority of cases. Finally, in our preliminary investigation of execution providers, we observe that TensorRT always outperforms CUDA.

Deep learning frameworks (DLFs) have been playing an increasingly important role in this intelligence age since they act as a basic infrastructure for an increasingly wide range of AIbased applications. Meanwhile, as multi-programming-language (MPL) software systems, DLFs are inevitably suffering from bugs caused by the use of multiple programming languages (PLs). Hence, it is of paramount significance to understand the bugs (especially the bugs involving multiple PLs, i.e., MPL bugs) of DLFs, which can provide a foundation for preventing, detecting, and resolving bugs in the development of DLFs. To this end, we manually analyzed 1497 bugs in three MPL DLFs, namely MXNet, PyTorch, and TensorFlow. First, we classified bugs in these DLFs into 12 types (e.g., algorithm design bugs and memory bugs) according to their bug labels and characteristics. Second, we further explored the impacts of different bug types on the development of DLFs, and found that deployment bugs and memory bugs negatively impact the development of DLFs in different aspects the most. Third, we found that 28.6%, 31.4%, and 16.0% of bugs in MXNet, PyTorch, and TensorFlow are MPL bugs, respectively; the PL combination of Python and C/C++ is most used in fixing more than 92% MPL bugs in all DLFs. Finally, the code change complexity of MPL bug fixes is significantly greater than that of single-programming-language (SPL) bug fixes in all the three DLFs, while in PyTorch MPL bug fixes have longer open time and greater communication complexity than SPL bug fixes. These results provide insights for bug management in DLFs.

TensorLy is a Python library that aims at making tensor learning simple and accessible. It allows to easily perform tensor decomposition, tensor learning and tensor algebra. The only pre-requisite is to have Python 3 installed. The easiest way is via the Anaconda distribution. Note: TensorLy depends on NumPy by default.

A quick introduction to JAX, MXNet, MATLAB, and FluxPhoto by Gabriel Sollmann on UnsplashMachine learning libraries accelerate the deep learning revolution. They lowered the barrier of entry for practitioners by abstracting many difficult things such as GPU speedup, matrix algebra, and automatic differentiation. In both industries and academia, two deep learning libraries reign supreme: PyTorch and TensorFlow. In this article, I will introduce you to some other deep learning libraries that have considerable usage, either because they achieve speedup in some ways, or because they are used by very specific groups. Let’s begin!JAXhttps://medium.com/media/6bac18fd43fec2b80e2b35ffe642f650/hrefWhat is it? A open-source and in-development numerical framework originally developed by Google (think NumPy but for GPU).Who uses it? Many teams within Google, such as DeepMind.Why should you know about it? JAX was developed by Google to accelerate numerical computing on GPU and Google’s own hardware TPU. Using ideas such as accelerated linear algebra, just-in-time compilation (JIT), and automatic vectorization, JAX achieved great speedup and scale. Even though their syntax are similar to minimize the learning curve, JAX has a different design philosophy from NumPy. JAX encourages functional programming though functions such as vmap and pmap (vectorize + parallelize).Currently, many high-level APIs have been developed for JAX. Notable ones are Haiku and Flax.Apache MXNethttps://medium.com/media/ed0dbfe51985a97c4f9b1700242e23b7/hrefWhat is it? An open-source veteran machine learning framework with front-end bindings for multiple languages including Python, C++, R, Java, and Perl.Who uses it? Amazon AWS.Why should you know about it? MXNet most powerful features are its support for many programming languages and its scalability. Benchmark tests by NVIDIA shows that MXNet is faster than PyTorch and TensorFlow on some deep learning tasks.MXNet comes with Gluon, a high-level API to build neural networks. It also has an ecosystem for image classification (GluonCV) and NLP (GluonNLP).MATLAB Deep Learning Toolboxhttps://medium.com/media/f35c14825e5dd6f7974091faeb658ca1/hrefWhat is it? An add-on toolbox for MATLAB users that can create and train neural networks for a variety of tasks.Who uses it? Academia and industries such as aerospace and mechanical engineering. For example, Airbus used it to detect defects inside airplanes.Why should you know about it? Whatever you feel about MATLAB, it is still a popular programming ecosystem amongst academics and engineers. It has great user support and, in my opinion, the best documentation out of all the deep learning libraries in this list. The deep learning toolbox is geared toward people who want to build systems using minimal programming. Simulink, a graphical programming interface within MATLAB, offers ways to create easy-to-understand deep learning pipelines.Julia Fluxhttps://medium.com/media/c8849fbf650e0032d47f701a04573899/hrefWhat is it? An open-source machine learning library built for Julia programming language.Who uses it? Computing-intensive fields such as pharmaceuticals and finances. For example, Astrazeneca used it to predict drug toxicity.Why should you know about it? Julia programming language gained momentum over the years amongst data scientists, quants, and bioinformatics researchers. It is comparable to C/C++ in terms of speed, and it was designed to be beginners-friendly like Python. An implementation of Julia deep learning on Google TPU showed >200x speedup compared to CPU. If you are already coding in Julia, Flux is a great library to look into.ConclusionI hope that, with this short article, you are introduced to some other deep learning libraries. They all support efficient speedups, GPU scaling, and deployment into productions. There are excellent learning sources for all of them on the internet. Happy coding!Sources[1] https://www.deepmind.com/blog/using-jax-to-accelerate-our-research[2] https://github.com/aws/sagemaker-python-sdk[3] https://developer.nvidia.com/deep-learning-performance-training-inference[4] https://www.mathworks.com/company/user_stories/case-studies/airbus-uses-artificial-intelligence-and-deep-learning-for-automatic-defect-detection.html[5] https://twitter.com/jeffdean/status/1054951415339192321?lang=en[6] https://julialang.org/blog/2012/02/why-we-created-julia/[7] https://juliacomputing.com/case-studies/astra-zeneca/Not Just PyTorch and TensorFlow: 4 Other Deep Learning Libraries You Should Lnow was originally published in Towards Data Science on Medium, where people are continuing the conversation by highlighting and responding to this story.

As you have heard, Meta has finally announced the PyTorch Foundation. It took us almost a year to get to this place where we finally can release it (if you even can say "release" about things like the foundation). In this article, I want to outline why this is done and what this means. Let's start with the problem. For quite some time, Meta was a gatekeeper for PyTorch.

Yuxi (Hayden) Liu is a Software Engineer, Machine Learning at Google. Previously he worked as a machine learning scientist in a variety of data-driven domains and applied his ML expertise in computational advertising, marketing and cybersecurity. He is now developing and improving the machine learning models and systems for ads optimization on the largest search engine in the world. He is an author of a series of machine learning books and an education enthusiast. His first book, also the first edition of Python Machine Learning by Example, ranked the #1 bestseller in Amazon in 2017 and 2018, and was translated into many different languages.

This book will introduce you to the basic principles of deep learning and teach you to build a neural network model from scratch. As you make your way through the book, you will explore deep learning libraries, such as Keras, MXNet, and TensorFlow, and create interesting deep learning models for a variety of tasks and problems, including structured data, computer vision, text data, anomaly detection, and recommendation systems. You'll cover advanced topics, such as generative adversarial networks (GANs), transfer learning, and large-scale deep learning in the cloud. In the concluding chapters, you will learn about the theoretical concepts of deep learning projects, such as model optimization, overfitting, and data augmentation, together with other advanced topics.

Dr. PKS Prakash is a Data Scientist and an author. He has spent last 12 years in developing many data science solution to solve problems from leading companies in healthcare, manufacturing, pharmaceutical and e-commerce domain. He is working as Data Science Manager at ZS Associates. ZS is one of the world's largest business services firms helping clients with commercial success, by creating data-driven strategies using advanced analytics that they can implement within their sales and marketing operations to make them more competitive, and by helping them deliver impact where it matters.

What is a Deep Learning Framework? A deep learning framework is a software package used by researchers and data scientists to design and train deep learning models. The idea with these frameworks is to allow people to train their models without digging into the algorithms underlying deep learning, neural networks, and machine learning. These frameworks offer building blocks for designing, training and validating models through a high level programming interface. Widely used deep learning frameworks such as PyTorch, TensorFlow, MXNet, and others can also use GPU-accelerated libraries such as cuDNN and NCCL to deliver high-performance multi-GPU accelerated training.

Until recently, customers who wanted to use a deep learning (DL) framework with Amazon SageMaker Processing faced increased complexity compared to those using scikit-learn or Apache Spark. This post shows you how SageMaker Processing has simplified running machine learning (ML) preprocessing and postprocessing tasks with popular frameworks such as PyTorch, TensorFlow, Hugging Face, MXNet, and XGBoost. Training an ML model takes many steps. One of them, data preparation, is paramount to creating an accurate ML model. Likewise, you often need to run postprocessing jobs (for example, filtering or collating) and model evaluation jobs (scoring models against different test sets) as part of your ML model development lifecycle.