As the need for machine learning (ML) increases rapidly across all industry sectors, there is a significant interest among commercial database providers to support "Query 2.0", which integrates model inference into SQL queries. Debugging Query 2.0 is very challenging since an unexpected query result may be caused by the bugs in training data (e.g., wrong labels, corrupted features). In response, we propose Rain, a complaint-driven training data debugging system. Rain allows users to specify complaints over the query's intermediate or final output, and aims to return a minimum set of training examples so that if they were removed, the complaints would be resolved. To the best of our knowledge, we are the first to study this problem. A naive solution requires retraining an exponential number of ML models. We propose two novel heuristic approaches based on influence functions which both require linear retraining steps. We provide an in-depth analytical and empirical analysis of the two approaches and conduct extensive experiments to evaluate their effectiveness using four real-world datasets. Results show that Rain achieves the highest recall@k among all the baselines while still returns results interactively.

The Russian Drug Reaction Corpus (RuDReC) is a new partially annotated corpus of consumer reviews in Russian about pharmaceutical products for the detection of health-related named entities and the effectiveness of pharmaceutical products. The corpus itself consists of two parts, the raw one and the labelled one. The raw part includes 1.4 million health-related user-generated texts collected from various Internet sources, including social media. The labelled part contains 500 consumer reviews about drug therapy with drug- and disease-related information. Labels for sentences include health-related issues or their absence. The sentences with one are additionally labelled at the expression level for identification of fine-grained subtypes such as drug classes and drug forms, drug indications, and drug reactions. Further, we present a baseline model for named entity recognition (NER) and multi-label sentence classification tasks on this corpus. The macro F1 score of 74.85% in the NER task was achieved by our RuDR-BERT model. For the sentence classification task, our model achieves the macro F1 score of 68.82% gaining 7.47% over the score of BERT model trained on Russian data. We make the RuDReC corpus and pretrained weights of domain-specific BERT models freely available at https://github.com/cimm-kzn/RuDReC

Representation learning for knowledge graphs (KGs) has focused on the problem of answering simple link prediction queries. In this work we address the more ambitious challenge of predicting the answers of conjunctive queries with multiple missing entities. We propose Bi-Directional Query Embedding (\textsc{BiQE}), a method that embeds conjunctive queries with models based on bi-directional attention mechanisms. Contrary to prior work, bidirectional self-attention can capture interactions among all the elements of a query graph. We introduce a new dataset for predicting the answer of conjunctive query and conduct experiments that show \textsc{BiQE} significantly outperforming state of the art baselines.

Open Domain Question Answering requires systems to retrieve external knowledge and perform multi-hop reasoning by composing knowledge spread over multiple sentences. In the recently introduced open domain question answering challenge datasets, QASC and OpenBookQA, we need to perform retrieval of facts and compose facts to correctly answer questions. In our work, we learn a semantic knowledge ranking model to re-rank knowledge retrieved through Lucene based information retrieval systems. We further propose a ``knowledge fusion model'' which leverages knowledge in BERT-based language models with externally retrieved knowledge and improves the knowledge understanding of the BERT-based language models. On both OpenBookQA and QASC datasets, the knowledge fusion model with semantically re-ranked knowledge outperforms previous attempts.

The relevance of search results is essential for finding information. Indeed, a user will almost never look further than the first few results of a search engine. It is therefore necessary that the relevant information is ranked as high as possible so that the information sought by the user is found in the first results. The order, or "ranking" of search results is essential for search engines, which will therefore use more or less complex algorithms to display the results that users will find most relevant first. It is usually not possible to find the algorithms used by popular search engines.

Curated Online Resource Puts Journalists a Click Away From Hundreds of Healthcare, Economic, Industry and Social Science Experts for Quick and Reliable Sources on the Current Coronavirus Pandemic. In response to unprecedented demand for expert sources and fact-based insights during the COVID-19 pandemic, ExpertFile has launched the COVID-19 Experts Search Engine, a specialized online resource designed to help newsrooms around the world;access reliable experts to speak on a variety of topics related to the coronavirus. With millions affected worldwide by the COVID-19 pandemic, the dangers of misinformation and factual inaccuracy pose a potentially devastating impact on society. As the largest curated, open-access search engine of international expert sources, ExpertFile worked quickly and in close consultation with its members -- including healthcare professionals, university academics, NGO's, corporations, industry associations and journalists -- to build the COVID-19 Experts Search Engine. "Facts matter more than opinions when real lives are at stake. We understand that journalists need evidence-based information, and they need it quickly," said Peter Evans, Co-Founder & CEO of ExpertFile.

Consumer sentiment has turned sharply negative as the virus has disrupted every aspect of daily American life. According to a consumer survey from Engine, 88% of consumers in the U.S. are now concerned about the pandemic. And according to another survey of roughly 2,600 U.S. adults from L.E.K. Consulting and Civis (.pdf), between 80% and 90% of adults expect a recession next year. In addition to measuring consumer sentiment, the survey explored how the coronavirus has shifted buying patterns across industries. Generally, the survey finds "significant increases in at-home activities, particularly cooking at home, watching television, browsing social media and exercising at home."

We are at a very exciting juncture in the development of artificial intelligence (AI). We are starting to see implementations of the third wave of the technology, which involves machines far surpassing human capabilities in various application domains, creating all kinds of opportunities for businesses. To leverage this to its full potential, companies need to rethink how they operate and put AI at the heart of everything they do. The first AI wave started with statistics-based systems. The best-known use is likely the information retrieval algorithms used by big internet companies like Google in the early years of AI, such as PageRank search engine.

It took me a long time to realise that search is the biggest problem in NLP. Just look at Google, Amazon and Bing. These are multi-billion dollar businesses possible only due to their powerful search engines. My initial thoughts on search were centered around unsupervised ML, but I participated in Microsoft Hackathon 2018 for Bing and came to know the various ways a search engine can be made with deep learning. Do you find this in-depth technical education about NLP applications to be useful?

Current state-of-the-art approximate nearest neighbor search (ANNS) algorithms generate indices that must be stored in main memory for fast high-recall search. This makes them expensive and limits the size of the dataset. We present a new graph-based indexing and search system called DiskANN that can index, store, and search a billion point database on a single workstation with just 64GB RAM and an inexpensive solid-state drive (SSD). Contrary to current wisdom, we demonstrate that the SSD-based indices built by DiskANN can meet all three desiderata for large-scale ANNS: high-recall, low query latency and high density (points indexed per node). On the billion point SIFT1B bigann dataset, DiskANN serves 5000 queries a second with 3ms mean latency and 95% 1-recall@1 on a 16 core machine, where state-of-the-art billion-point ANNS algorithms with similar memory footprint like FAISS and IVFOADC G P plateau at around 50% 1-recall@1.