Embedding-based Retrieval with LLM for Effective Agriculture Information Extracting from Unstructured Data

Information extraction (IE) refers to the process of extracting information from unstructured text and transform it into structured data. Nowadays, in an information era, the rapid increase in the amount of data has made this type of task increasingly important. IE is labour-intensive and time-consuming, so lots of domains have switched to automatic or semi-automatic IE Wang et al. [2018] Saggion et al. [2007]. The Internet provides a vast amount of information for agriculture, but the lack of effective data processing methods leads to that much agricultural information remains unarchived, buried in news, papers, and government and organization websites. This may mainly be due to the shortage of annotated corpora Nismi Mol and Santosh Kumar [2023]. These documents cannot be easily analyzed or queried in their raw form and require some form of information extraction to be easily utilised in applications. Searching and managing this unstructured information efficiently is not only a difficult challenge for farmers, but for agriculture professionals as well.