[R] Tensor Comprehensions in PyTorch • r/MachineLearning

Up to a dozen assignment expressions per TC definition sounds reasonable. In practice it depends on the layer types. There are basically two scaling limiters: compilation and especially autotuning time grows very fast as a function of a number of expressions; the amount of exploitable parallelism may decrease with the increasing number of inter-dependent operations. One of the goals of TC is to make it easy to define new layers and look at practically achieved performance. It should be as easy as moving the line between two TC defs and changing input/output tensor lists.