Structured information extraction from complex scientific text with fine-tuned large language models

This completion can be formatted as either English sentences or a more structured schema such as a list of JSON documents. Large language models (LLMs) such as GPT-3 [12], PaLM To use this method, one only has to define the desired [25], Megatron [26], OPT [27], Gopher [28], and FLAN [29] output structure--for example, a list of JSON objects with a have been shown to have remarkable ability to leverage semantic predefined set of keys--and annotate 100 500 text passages information between tokens in natural language sequences using this format. GPT-3 is then fine-tuned on these of varying length. They are particularly adept at examples, and the resulting model is able to accurately extract sequence-to-sequence (seq2seq) tasks, where a text input is desired information from text and output information in used to seed a text response from the model. In this paper the same structured representation as shown in Figure 1.