AdaSTaR Adaptive Data Sampling for Training Self Taught Reasoners

Self-Taught Reasoners (STaR), synonymously known as Rejection sampling FineTuning (RFT), is an integral part of the training pipeline of self-improving reasoning Language Models (LMs). The self-improving mechanism often employs random observation (data) sampling. However, this results in trained observation imbalance; inefficiently over-training on solved examples while under-training on challenging ones.