Fine-grained Analysis of Brain-LLM Alignment through Input Attribution