Disentangling Logic: The Role of Context in Large Language Model Reasoning Capabilities