Fast Tree-Field Integrators: From Low Displacement Rank to Topological Transformers