Towards Instance-Optimal Offline Reinforcement Learning with Pessimism Ming Yin 1,2 and Y u-Xiang Wang 1 1 Department of Computer Science, UC Santa Barbara