InnerThoughts: Disentangling Representations and Predictions in Large Language Models