Layer-Aware Task Arithmetic: Disentangling Task-Specific and Instruction-Following Knowledge