Have you Optimized your Deep Learning Model Before Deployment?

This article presents how to use NVIDIA TensorRT to optimize a deep learning model that you want to deploy on the edge device (mobile, camera, robot, car ….). As an example, think of AI-based computer vision application, they need to process each frame captured by the camera. Thus, each frame makes a forward pass through the layers of the model to compute a certain output (detection, segmentation, classification…). Whatever power your GPU has, we all want the number of frames per second (FPS) at the output to be equal to one at the input (example 24, 30 FPS…). This means that the GPU is processing each frame in real-time.