[D] Feeding latent representation as input to Siamese network • r/MachineLearning

Your "speed things up by compressing them" idea (and, yes, it makes sense and is the basis of a lot of techniques in information retrieval) can also work without the autoencoder. Siamese networks process separate inputs through separate paths where each of the distinct paths share their parameters. What's stopping you from, for all of your repository entries, precomputing the forward pass up to the point where the paths of the network merge and storing the result instead of the raw input? This would mean that at lookup time, you'd only have to do a full forward pass for the query image and then substitute in the precomputed representation for the candidates. All this can be done no matter how the network was trained -- it doesn't rely on training a separate autoencoder as a preprocessor.