Multimodal encoders have pushed the boundaries of visual document retrieval, matching textual query tokens directly to image patches and achieving state-of-the-art performance on public benchmarks. Recent models relying on this paradigm have massively scaled the sizes of their query and document representations, presenting obstacles to deployment and scalability in real-world pipelines. Furthermore, purely vision-centric approaches may be constrained by the inherent modality gap still exhibited by modern vision-language models. In this work, we connect these challenges to the paradigm of hybrid retrieval, investigating whether a lightweight dense text retriever can enhance a stronger vision-centric model. Existing hybrid methods, which rely on coarse-grained fusion of ranks or scores, fail to exploit the rich interactions within each model's representation space. To address this, we introduce Guided Query Refinement (GQR), a novel test-time optimization method that refines a primary retriever's query embedding using guidance from a complementary retriever's scores. Through extensive experiments on visual document retrieval benchmarks, we demonstrate that GQR allows vision-centric models to match the performance of models with significantly larger representations, while being up to 14x faster and requiring 54x less memory. Our findings show that GQR effectively pushes the Pareto frontier for performance and efficiency in multimodal retrieval. We release our code at https://github.com/IBM/test-time-hybrid-retrieval

For data scientists and engineers, speed is important along with accuracy. For accuracy, we built Finetuner, which lets you finetune neural networks to achieve top performance on downstream tasks. Concerning speed, Jina was already fast, but now it's even faster. DocArray has been created to remove all the shortcomings in existing data structures, especially for ML and data science-related tasks. Here is a comparison of DocArray with other data structures.

You can now leverage our robust Document Store powered by Elasticsearch to retrieve embeddings in the blink of an eye! We're delighted to announce CLIP-as-service, a low-latency high-scalability service for embedding images and texts. Read our blog to know how you can easily integrate it as a microservice into your neural search solutions. We've also made some interface improvements and added new features If you have technical issues with Jina, we would love to discuss them all during our Office Hours on March 31st, Thursday at 17:00 CET! (convert to your timezone) Sign up here and add to your calendar to receive a notification. Our good first issues repo features some great issues for those looking to take their first steps into open source with Jina.

I had a pleasure to sit down with Joan Fontanals – Principal Engineer with Jina.AI -- framework with lots of capabilities to support your neural search journey. Listen to or watch the podcast and get a chance to win awesome swag from Jina.AI. As a special line of thank-yous, I'd like to mention Saurabh Rai, who kindly designed the Thumbnail of this episode!

To understand the basics of neural search and how it differs from conventional search please go through my previous blog on "Next-gen powered by Jina". It explains how Jina- a cloud-native, open-source company is pioneering the field of neural search. It builds on the idea of semantic search and explains the basic building blocks of the Jina framework required to build intelligent search applications. Just as a recap the idea behind neural search is to leverage state-of-the-art deep neural networks to intelligently retrieve contextual and semantically relevant information from the heaps of data. A neural search system can go way beyond simple text search by allowing you to search through all the formats of data including images, videos, audios, and even PDFs.

TL;DR: Neural Search is a new approach to retrieving information using neural networks. Traditional techniques to search typically meant writing rules to "understand" the data being searched and return the best results. But with neural search, developers don't need to wrack their brains for these rules; The system learns the rules by itself and gets better as it goes along. Even developers who don't know machine learning can quickly build a search engine using open-source frameworks such as Jina. There is a massive amount of data on the web; how can we effectively search through it for relevant information?

Sign in to report inappropriate content. Understand the key concepts in Jina and Agenda: - What is Neural Search and what is Jina - Highlights & features of Jina - Jina hello-world walkthrough - Learn Jina: the fast & the best way - Jina's eco-system and next step - Open-source as a business - Q and A Welcome to join the live stream and share it with your colleagues and friends!

CHOPCHOP is an app that guides you through cooking a several-course meal with the help of Artificial Intelligence – and without sweat! The young team came a long way during their EuropeanPioneers time: In December 2015 CHOPCHOP has been launched supported by the EuropenPioneers team. We spoke with the founder JinA what happened after that. EuropeanPioneer: Hi JinA, what are you up to these days? We also have a new team member that joined us recently to make improvements to our algorithm, which is exciting.