Decoding Geometric Properties in Non-Random Data from First Information-Theoretic Principles