Google claims TensorFlow's OpenCL can double inference performance

Google today announced the launch of an OpenCL-based mobile GPU inference engine for its TensorFlow framework on Android. It's available now in the latest version of the TensorFlow Lite library, and the company claims it offers a two times speedup over the existing OpenGL backend with "reasonably-sized" AI models. OpenGL, which is nearly three decades old, is a platform-agnostic API for rendering 2D and 3D vector graphics. Compute shaders were added with OpenGL ES 3.1, but the TensorFlow team says backward-compatible design decisions limited them from reaching device GPUs' full potential. On the other hand, OpenCL was designed for computation with various accelerators from the beginning, and was thus more relevant to the domain of mobile GPU inference.