A Appendix

In order to build our latency prediction model, We test three types of hardware devices, NVIDIA V100, NVIDIA GTX 2080, and NVIDIA GTX 1080. Their respective properties are presented in Table 6. It shows that the server GPU V100 is the most powerful hardware device with the most processing engines (#PE). We map the operations to hardware. These split tiles are assigned to multiple PEs.