Electronic health records (EHRs) are stored in various database systems with different database models on heterogeneous storage architectures, such as relational databases, document stores, or graph databases. These different database models have a big impact on query complexity and performance. While this has been a known fact in database research, its implications for the growing number of Text-to-Query systems have surprisingly not been investigated so far.In this paper, we present SM3-Text-to-Query, the first multi-model medical Text-to-Query benchmark based on synthetic patient data from Synthea, following the SNOMED-CT taxonomy---a widely used knowledge graph ontology covering medical terminology. SM3-Text-to-Query provides data representations for relational databases (PostgreSQL), document stores (MongoDB), and graph databases (Neo4j and GraphDB (RDF)), allowing the evaluation across four popular query languages, namely SQL, MQL, Cypher, and SPARQL.We systematically and manually develop 408 template questions, which we augment to construct a benchmark of 10K diverse natural language question/query pairs for these four query languages (40K pairs overall). On our dataset, we evaluate several common in-context-learning (ICL) approaches for a set of representative closed and open-source LLMs.Our evaluation sheds light on the trade-offs between database models and query languages for different ICL strategies and LLMs. Last,SM3-Text-to-Query is easily extendable to additional query languages or real, standard-based patient databases.

Electronic health records (EHRs) are stored in various database systems with different database models on heterogeneous storage architectures, such as relational databases, document stores, or graph databases. These different database models have a big impact on query complexity and performance. While this has been a known fact in database research, its implications for the growing number of Text-to-Query systems have surprisingly not been investigated so far.In this paper, we present SM3-Text-to-Query, the first multi-model medical Text-to-Query benchmark based on synthetic patient data from Synthea, following the SNOMED-CT taxonomy---a widely used knowledge graph ontology covering medical terminology. SM3-Text-to-Query provides data representations for relational databases (PostgreSQL), document stores (MongoDB), and graph databases (Neo4j and GraphDB (RDF)), allowing the evaluation across four popular query languages, namely SQL, MQL, Cypher, and SPARQL.We systematically and manually develop 408 template questions, which we augment to construct a benchmark of 10K diverse natural language question/query pairs for these four query languages (40K pairs overall). On our dataset, we evaluate several common in-context-learning (ICL) approaches for a set of representative closed and open-source LLMs.Our evaluation sheds light on the trade-offs between database models and query languages for different ICL strategies and LLMs.

Closed book Abstractive: These systems do not have access to external data store. They store information internally in the model parameters. ChatGPT and other large language models are part of this category. Unlike open book systems, these system do not have access to the latest information.

In this blogpost, we will walkthrough how make use of AWS App Runner, a managed container service to deploy an NLP model inference service. This service uses deepset's haystack for a simple Q&A application and a REST API. Anyone with interests in football transfers can make use of the simple Q&A application. To stay updated with the latest news on football transfers, I noticed that I spend a lot scrolling through tons of tweets to stay updated. Transfer news are part of the excitement in football -- fans want to know which players are joining or leaving.

Recent advancements in NLP question answering (QA)-based systems have been astonishing. QA systems built on top of the most recent language models (BERT, RoBERTa, etc.) can answer factoid-based questions with relative ease and excellent precision. The task involves finding the relevant document passages containing the answer and extracting the answer by scanning the correct word token span. More challenging QA systems engage with so-called "generative question answering". These systems focus on handling questions where the provided context passages are not simply the source tokens for extracted answers, but provide the larger context to synthesize original answers.

Last month I posted a short note on my LinkedIn account on the unfortunate decline of MapR. A company that had brilliant engineers trying to find a place in the crowded data products market. On the bright side, I was happy to see my post generated a lot of discussions and made many people think more about where the "Big Data" industry is going and how this will impact large-scale enterprise analytics that are behind many of the innovations in AI. So here are my predictions on how the change in data-at-scale is driving the evolution of AI. Before we talk about the future of integration data patterns, let's recap three Big Data architectural patterns and how they are different.

I am not a fan of the term NoSQL. Many others are, however, and it has become a permanent part of the collective data storage nomenclature, meant to describe schema-less, non-relational data storage schemes. NoSQL is an umbrella term, one which encompasses a number of different technologies. These different technologies aren't even necessarily related in any way beyond the single defining characteristic of NoSQL: they are not relational in nature; for right or wrong, Structured Query Language (SQL) has become conflated with relational database management systems over the years. So, while I am not personally a fan of the term NoSQL, I can appreciate why others are, given that it quickly implies what it is we are talking about by explicitly stating what we are not talking about.