Bringing AI to the Edge

In a little over a decade since researchers uncovered novel techniques to improve its efficiency and effectiveness, deep learning has become a practical technology that now underpins a number of applications that need artificial intelligence (AI). Many of these applications are hosted in the cloud on powerful servers, as tasks sometimes involve the processing of data-rich sources such as images, videos, and audio. Those servers often call on the additional performance of acceleration hardware, which range from graphics processing units to custom devices. These become particularly important for the numerically-intensive process, during which a neural network is trained on new data. Typically, the inferencing process, which uses a trained network to assess new data, is far less compute-intensive than training.