The Execution Process of a Tensor in a Deep Learning Framework

This article focuses on what is happening behind the execution of a Tensor in the deep learning framework OneFlow. It takes the operator oneflow.relu Hope this article will enlighten you on the system design of deep learning frameworks. First of all, let's look at the following code in PyTorch: In the above code, the input Tensor x is fed to the operator relu, and the result is printed. Everything looks simple and normal. But if someone asks you if you are clear about what is happening behind this and when the Cuda Kernel corresponding to relu is called by the GPU, you may not be so clear.