VTA: An Open Hardware-Software Stack for Deep Learning

Specialized deep learning hardware is starting to become commonplace in the datacenter, and on the edge. Much of the progress in ML system has been fueled by hardware specialization, which allows faster training and inference at lower energy costs [7, 2, 3, 6]. Using specialized hardware, though, requires deep learning frameworks to be redesigned around novel hardware architectural features and interfaces. Google's TensorFlow XLA and TPU [7] system stack is an excellent example of a complete framework built to take full advantage of hardware acceleration. However at the time of this writing, not much of the TPU hardware design, low-level programming interface, code-generation, or operator libraries are made transparent and available for researchers to maintain, experiment, and customize. In order to understand how hardware specialization is transforming the deep learning system landscape, it is essential to give systems, compilers, and machine learning researchers access to a complete system stack that transparently exposes all of its layers, including the hardware architecture of the accelerator itself, and its low-level programming interface. We present the VTA stack, a complete deep-learning system stack built with TVM [1] around VTA, a generic deep learning hardware accelerator design. VTA aims to serve as a blueprint to deep learning, systems, and compilers researchers who want to understand how hardware accelerators dictate new constraints across the system stack, and how the latter can be better co-designed with hardware. The VTA stack was designed with the following design objectives: - Provide a common deep learning system stack for hardware, compilers, and systems researchers alike to incorporate state-of-the-art optimizations and co-design techniques.