While Part 1 of this 2-part blog series covered Deep Neural Networks and the different accelerators for implementing Deep Neural Network Models, Part 2 will talk about different Deep Learning Frameworks and hardware frameworks provided by FPGA Vendors. Deep learning framework can be considered as a tool or library that helps us to build DNN models quickly and easily without any in-depth knowledge of the underlying algorithms. It provides a condensed way for defining the models using pre-built and optimized components. Some of the important deep learning frameworks are Caffe, TensorFlow, Pytorch, Keras, etc. Caffe is a deep neural network framework designed to improve speed and modularity. It is developed by Berkeley AI Research.

Previously, we described the AMD ROCm open-source platform that developers can freely use to create innovative software applications that take advantage of the underlying accelerated hardware. The ROCm platform is designed so that a wide range of developers can develop accelerated applications. An entire eco-system has been created, allowing developers to focus on developing their leading-edge applications. Developers require more than just an API to create new applications quickly and confidently. As part of an eco-system, ROCm contains a debugger, tools for performance analysis, system validation, and system management.

Recently, in an official announcement, Google launched an OpenCL-based mobile GPU inference engine for Android. The tech giant claims that the inference engine offers up to 2x speedup over the OpenGL backend on neural networks which include enough workload for the GPU. This GPU inference engine is currently made available in the latest version of TensorFlow Lite (TFLite) library. Open Graphics Library or OpenGL is an API designed for rendering vector graphics through which a client application can control this system. It is a popular software interface that allows a programmer to communicate with graphics hardware.

TensorFlow Lite for AI inference on mobile devices now has support for making use of OpenCL on Android devices. In doing so, the TFLite performance presents around a 2x speed-up over the existing OpenGL back-end. To little surprise, the TensorFlow developers are finding their new OpenCL back-end for TFLite to be much faster than their OpenGL back-end for mobile inference. Thanks to better performance profiling abilities, native FP16 support, constant memory, and OpenCL being better designed for compute than OpenGL ES with compute shaders, the TFLite performance is much improved -- and especially so compared to doing inference on the mobile SoC CPU cores. "Our new OpenCL backend is roughly twice as fast as the OpenGL backend, but does particularly better on Adreno devices (annotated with SD), as we have tuned the workgroup sizes with Adreno's performance profilers mentioned earlier."

Google today announced the launch of an OpenCL-based mobile GPU inference engine for its TensorFlow framework on Android. It's available now in the latest version of the TensorFlow Lite library, and the company claims it offers a two times speedup over the existing OpenGL backend with "reasonably-sized" AI models. OpenGL, which is nearly three decades old, is a platform-agnostic API for rendering 2D and 3D vector graphics. Compute shaders were added with OpenGL ES 3.1, but the TensorFlow team says backward-compatible design decisions limited them from reaching device GPUs' full potential. On the other hand, OpenCL was designed for computation with various accelerators from the beginning, and was thus more relevant to the domain of mobile GPU inference.

This week is the eighth annual International Workshop on OpenCL, SYCL, Vulkan, and SPIR-V, and the event is available online for the very first time in its history thanks to the coronavirus pandemic. One of the event organizers, and the conference chair, is Simon McIntosh-Smith, who is a professor of high performance computing at Bristol University in Great Britain and also the head of its Microelectronics Group. Among other things, McIntosh-Smith was a microprocessor architect at STMicroeletronics, where he designed SIMD units for the dual-core, superscalar Chameleon and SH5 set-top box ASICs back in the late 1990s. McIntosh-Smith moved to Pixelfusion in 1999, which created the first general purpose GPU – arguably eight or nine years before Nvidia did it with its Tesla GPUs, where he was an architect on the 1,536-core chip and software manager for two years. In 2002, McIntosh-Smith was one of the co-founders of ClearSpeed, which created floating point math accelerators used in HPC systems before GPU accelerators came along, and was first director of architecture and applications and then vice president of applications.

The sequence-to-sequence (seq2seq) model for neural machine translation has significantly improved the accuracy of language translation. There have been new efforts to use this seq2seq model for program language translation or program comparisons. In this work, we present the detailed steps of using a seq2seq model to translate CUDA programs to OpenCL programs, which both have very similar programming styles. Our work shows (i) a training input set generation method, (ii) pre/post processing, and (iii) a case study using Polybench-gpu-1.0, NVIDIA SDK, and Rodinia benchmarks.

For the 0.3.3 release of PlaidML, support for running deep learning networks on macOS has improved with the ability to use Apple's native Metal API. Metal offers "near-direct access to the graphics processing unit (GPU)", allowing machine learning tasks to run faster on any Mac where Metal is supported. As previously announced, Mac users could accelerate their PlaidML workloads by using the OpenCL backend. In our internal testing, in some cases, we see an up to 5x speed up by using Metal over OpenCL. Next, run plaidml-setup to select the desired Metal-based device.

Deep Learning (DL) is part of the field of Machine Learning (ML). DL works by approximating a solution to a problem using neural networks. One of the nice properties of about neural networks is that they find patterns in the data (features) by themselves. This is opposed to having to tell your algorithm what to look for, as in the olde times. However, often this means the model starts with a blank state (unless we are transfer learning).

Deep learning is a subset of machine learning based on neural networks. With deep learning the more data the better which can require more computing power. In this case that computing power comes from graphics processing units (GPU), as their architecture is bested suited for the job. Typically the GPU is needed in the training stage of machine learning. At this stage more cores and faster GPUs mean you can train the system faster.