Mastering Da Vinci Code: A Comparative Study of Transformer, LLM, and PPO-based Agents