Information Retrieval
Build A Search Engine With Python: Computer Science & Python
Unit 2 will take you in more depth in using procedures, loops, and the logical constructs in order to add more functionality to the crawler built-in unit 1. Unit 3 is about managing data through mastering the use of the different data types to be able to create the search engine index. Our biggest goal tho is to learn about computer science, So unit 4 focuses on teaching you how computers store data and how to be cost-effective when doing that. By the end of unit 5, you'll have a better understanding of how programs run and how to implement a hash table for our search engine. Unit 6 will extend the grammar we introduced in unit 1 and will show you how to get the best result for a search query. If the course gets much interaction and feedback, we'll work on units to demonstrate how to code real-world Python applications I hope you're as excited as I'm to start this learning journey, so just the enrollment today and thank me later.
AliCG: Fine-grained and Evolvable Conceptual Graph Construction for Semantic Search at Alibaba
Zhang, Ningyu, Jia, Qianghuai, Deng, Shumin, Chen, Xiang, Ye, Hongbin, Chen, Hui, Tou, Huaixiao, Huang, Gang, Wang, Zhao, Hua, Nengwei, Chen, Huajun
Conceptual graphs, which is a particular type of Knowledge Graphs, play an essential role in semantic search. Prior conceptual graph construction approaches typically extract high-frequent, coarse-grained, and time-invariant concepts from formal texts. In real applications, however, it is necessary to extract less-frequent, fine-grained, and time-varying conceptual knowledge and build taxonomy in an evolving manner. In this paper, we introduce an approach to implementing and deploying the conceptual graph at Alibaba. Specifically, We propose a framework called AliCG which is capable of a) extracting fine-grained concepts by a novel bootstrapping with alignment consensus approach, b) mining long-tail concepts with a novel low-resource phrase mining approach, c) updating the graph dynamically via a concept distribution estimation method based on implicit and explicit user behaviors. We have deployed the framework at Alibaba UC Browser. Extensive offline evaluation as well as online A/B testing demonstrate the efficacy of our approach.
Conversational Question Answering: A Survey
Zaib, Munazza, Zhang, Wei Emma, Sheng, Quan Z., Mahmood, Adnan, Zhang, Yang
Question answering (QA) systems provide a way of querying the information available in various formats including, but not limited to, unstructured and structured data in natural languages. It constitutes a considerable part of conversational artificial intelligence (AI) which has led to the introduction of a special research topic on Conversational Question Answering (CQA), wherein a system is required to understand the given context and then engages in multi-turn QA to satisfy the user's information needs. Whilst the focus of most of the existing research work is subjected to single-turn QA, the field of multi-turn QA has recently grasped attention and prominence owing to the availability of large-scale, multi-turn QA datasets and the development of pre-trained language models. With a good amount of models and research papers adding to the literature every year recently, there is a dire need of arranging and presenting the related work in a unified manner to streamline future research. This survey, therefore, is an effort to present a comprehensive review of the state-of-the-art research trends of CQA primarily based on reviewed papers from 2016-2021. Our findings show that there has been a trend shift from single-turn to multi-turn QA which empowers the field of Conversational AI from different perspectives. This survey is intended to provide an epitome for the research community with the hope of laying a strong foundation for the field of CQA.
Database Reasoning Over Text
Thorne, James, Yazdani, Majid, Saeidi, Marzieh, Silvestri, Fabrizio, Riedel, Sebastian, Halevy, Alon
Neural models have shown impressive performance gains in answering queries from natural language text. However, existing works are unable to support database queries, such as "List/Count all female athletes who were born in 20th century", which require reasoning over sets of relevant facts with operations such as join, filtering and aggregation. We show that while state-of-the-art transformer models perform very well for small databases, they exhibit limitations in processing noisy data, numerical operations, and queries that aggregate facts. We propose a modular architecture to answer these database-style queries over multiple spans from text and aggregating these at scale. We evaluate the architecture using WikiNLDB, a novel dataset for exploring such queries. Our architecture scales to databases containing thousands of facts whereas contemporary models are limited by how many facts can be encoded. In direct comparison on small databases, our approach increases overall answer accuracy from 85% to 90%. On larger databases, our approach retains its accuracy whereas transformer baselines could not encode the context.
CEREC: A Corpus for Entity Resolution in Email Conversations
Dakle, Parag Pravin, Moldovan, Dan I.
We present the first large scale corpus for entity resolution in email conversations (CEREC). The corpus consists of 6001 email threads from the Enron Email Corpus containing 36,448 email messages and 60,383 entity coreference chains. The annotation is carried out as a two-step process with minimal manual effort. Experiments are carried out for evaluating different features and performance of four baselines on the created corpus. For the task of mention identification and coreference resolution, a best performance of 59.2 F1 is reported, highlighting the room for improvement. An in-depth qualitative and quantitative error analysis is presented to understand the limitations of the baselines considered.
Database Workload Characterization with Query Plan Encoders
Smart databases are adopting artificial intelligence (AI) technologies to achieve instance optimality, and in the future, databases will come with prepackaged AI models within their core components. The reason is that every database runs on different workloads, demands specific resources, and settings to achieve optimal performance. It prompts the necessity to understand workloads running in the system along with their features comprehensively, which we dub as workload characterization. To address this workload characterization problem, we propose our query plan encoders that learn essential features and their correlations from query plans. Our pretrained encoders capture the structural and the computational performance of queries independently.
Postdoc Position in Computer Science - Applied Machine Learning and Information Retrieval
Postdoctoral fellowship in Applied Machine Learning and Information Retrieval is available at the Department of Computer Science, University of Copenhagen, Denmark. The application, in English, should be submitted electronically by clicking APPLY ONLINE above. The postdoctoral fellow will join the Machine Learning Section at DIKU. The Machine Learning section is among the leading research environments in Artificial Intelligence and Web & Information Retrieval in Europe (in the top 5 for 2020, according to csrankings.org), with a strong presence at top-tier conferences, continuous collaboration in international & national research networks, and solid synergies with big tech, small tech, and industry. The Machine Learning section consists of a vibrant selection of approximately 65 talented researchers (40 of whom are PhD and postdoctoral fellows) from around the world with a diverse set of backgrounds and a common incessant scientific curiosity and openness to innovation.
Database Workload Characterization with Query Plan Encoders
Paul, Debjyoti, Cao, Jie, Li, Feifei, Srikumar, Vivek
Smart databases are adopting artificial intelligence (AI) technologies to achieve {\em instance optimality}, and in the future, databases will come with prepackaged AI models within their core components. The reason is that every database runs on different workloads, demands specific resources, and settings to achieve optimal performance. It prompts the necessity to understand workloads running in the system along with their features comprehensively, which we dub as workload characterization. To address this workload characterization problem, we propose our query plan encoders that learn essential features and their correlations from query plans. Our pretrained encoders capture the {\em structural} and the {\em computational performance} of queries independently. We show that our pretrained encoders are adaptable to workloads that expedite the transfer learning process. We performed independent assessments of structural encoder and performance encoders with multiple downstream tasks. For the overall evaluation of our query plan encoders, we architect two downstream tasks (i) query latency prediction and (ii) query classification. These tasks show the importance of feature-based workload characterization. We also performed extensive experiments on individual encoders to verify the effectiveness of representation learning and domain adaptability.
Predicting Links on Wikipedia with Anchor Text Information
Brochier, Robin, Béchet, Frédéric
Wikipedia, the largest open-collaborative online encyclopedia, is a corpus of documents bound together by internal hyperlinks. These links form the building blocks of a large network whose structure contains important information on the concepts covered in this encyclopedia. The presence of a link between two articles, materialised by an anchor text in the source page pointing to the target page, can increase readers' understanding of a topic. However, the process of linking follows specific editorial rules to avoid both under-linking and over-linking. In this paper, we study the transductive and the inductive tasks of link prediction on several subsets of the English Wikipedia and identify some key challenges behind automatic linking based on anchor text information. We propose an appropriate evaluation sampling methodology and compare several algorithms. Moreover, we propose baseline models that provide a good estimation of the overall difficulty of the tasks.