BloombergGPT: The First GPT for Finance

Bloomberg has been a leader in AI, machine learning, and NLP in finance for over a decade. They've developed a mixed approach that combines finance data with general-purpose datasets to train a model that achieves best-in-class financial results while maintaining competitive performance on general-purpose LLM benchmarks. To develop BloombergGPT, the ML Product and Research group collaborated with the AI Engineering team to create one of the largest domain-specific datasets yet. They drew on Bloomberg's existing data creation, collection, and curation resources, using their extensive archive of financial data to create a comprehensive 363 billion token dataset consisting of English financial documents. They then augmented this data with a 345 billion token public dataset to create a training corpus with over 700 billion tokens.