Google Research into Concept Vectors for Image Search

Google recently released research about a tool called Similar Medical Images Like Yours (SMILY) that uses concept vectors to enhance searching for medical images. The research uses embeddings for image-based search and allows users to influence the search through the interactive refinement of concepts. Google released two papers in succession. The first paper, "Similar image search for histopathology: SMILY" focused on the deep neural network architecture that was used to create the embeddings necessary to find similar images. The second paper, "Human-Centered Tools for Coping with Imperfect Algorithms During Medical Decision-Making," focused on human interaction aspects necessary to improve the usage of the tool created in the first paper.