SPIN: SE(3)-Invariant Physics Informed Network for Binding Affinity Prediction