Variable Length Embeddings

We introduce a novel deep learning architecture, called Variable Length Embeddings (VLE). A VLE is an autoencoder that differs from traditional ones in one key aspect: whereas conventional autoencoders have a fixed embedding dimension, VLEs (as their name suggests) use a variablelength embedding dimension. Allowing the embedding dimension to vary is a natural idea: not all images are created equal. Images that contain more complex semantics should naturally require more resources to represent efficiently. Viewed through the lens of information theory, this is a well-known idea: we ought to use less resources to represent'easy' samples.