Offline Learning of Counterfactual Perception as Prediction for Real-World Robotic Reinforcement Learning