Robust Offline Reinforcement Learning -- Certify the Confidence Interval