Searching for Efficient Linear Layers over a Continuous Space of Structured Matrices