Reviews: Entropy Rate Estimation for Markov Chains with Large State Space
–Neural Information Processing Systems
The paper proposes an entropy estimate for Markov chains by reduction to optimal entropy estimation for i.i.d samples. Sample complexity analysis is provided for different mixing scenarios with a minimax rate established for a particular rate. The estimator is used to assess the capacity of language models. This is a very clear and well-written paper. I appreciate the efforts done by the authors to summarize the results.
Neural Information Processing Systems
Oct-7-2024, 21:00:17 GMT