Reviews: Memory-Efficient Backpropagation Through Time
–Neural Information Processing Systems
The authors are solving an important problem. RNN training procedures can be greedy for memory. And, given the sequential nature, it's not trivial to simply to scale the training of each sequence over many machines. As a result, it's important to judiciously use memory and computational resources to train RNNs efficiently. I'm pleased to see the authors not only proposing a new instance of a solution, but to provide a user-selectable tradeoff between the quantity of computation and the memory usage.
Neural Information Processing Systems
Jan-20-2025, 17:04:25 GMT
- Technology: