Exploring the Limits of Fine-grained LLM-based Physics Inference via Premise Removal Interventions