Compute Better Spent: Replacing Dense Layers with Structured Matrices