Goto

Collaborating Authors

 Large Language Model


FinBERT-QA: Financial Question Answering with pre-trained BERT Language Models

arXiv.org Artificial Intelligence

Motivated by the emerging demand in the financial industry for the automatic analysis of unstructured and structured data at scale, Question Answering (QA) systems can provide lucrative and competitive advantages to companies by facilitating the decision making of financial advisers. Consequently, we propose a novel financial QA system using the transformer-based pre-trained BERT language model to address the limitations of data scarcity and language specificity in the financial domain. Our system focuses on financial non-factoid answer selection, which retrieves a set of passage-level texts and selects the most relevant as the answer. To increase efficiency, we formulate the answer selection task as a re-ranking problem, in which our system consists of an Answer Retriever using BM25, a simple information retrieval approach, to first return a list of candidate answers, and an Answer Re-ranker built with variants of pre-trained BERT language models to re-rank and select the most relevant answers. We investigate various learning, further pre-training, and fine-tuning approaches for BERT. Our experiments suggest that FinBERT-QA, a model built from applying the Transfer and Adapt further fine-tuning and pointwise learning approach, is the most effective, improving the state-of-the-art results of task 2 of the FiQA dataset by 16% on MRR, 17% on NDCG, and 21% on Precision@1.


The Android Show, Rad Power's New Ebikes, and Yale's ADT Smart Lock--Your Gear News of the Week

WIRED

Google's annual I/O developer conference is coming up on May 20--and all signs point to it being a big one. It's where we typically learn everything new coming to Android, Google's Gemini artificial intelligence assistant, and all of the company's other platforms, from Wear OS to Android Auto. But this week, Google announced a virtual event called The Android Show: I/O Edition, which takes place a week earlier, on May 13 at 10 am Pacific (1 pm ET). A teaser video showed Google's Sameer Samat, president of the Android ecosystem, getting ready on camera and announcing the show. The Android Show will likely explore all the new features coming to Android 16, the next version of Google's mobile operating system.


Bringing AI to the Edge

Communications of the ACM

This year, U.S. rail carrier Amtrak will be installing two novel inspection gateways from Duos Technologies along its busy Northeast Corridor. The barn-like Duos structures straddle railway tracks; as passenger trains speed through at up to 125 miles per hour, 97 cameras and dozens of LED lights arrayed around the sides, top, and bottom of the tracks will capture thousands of high-resolution images of the railcars. These images are aggregated and processed on site in real time to present a complete, 360-degree, highly detailed view of the train. Artificial intelligence (AI) algorithms running on Nvidia GPUs will analyze the images locally; if the model flags a potential structural or mechanical flaw, train personnel will be notified in less than a minute. The Duos portal is one of many new examples of what is loosely categorized as edge AI, or the deployment and operation of AI models outside of massive cloud datacenters.


Inside the Battle Over OpenAI's Corporate Restructuring

WIRED

Last October, the news that OpenAI was planning to simplify its unusual nonprofit structure caught the attention of economic-justice activist Orson Aguilar. He feared that the ChatGPT maker's plan to transition into a more conventional company, from which investors could generate unlimited returns, would financially hurt the working-class communities he has spent nearly 30 years fighting to protect. Aguilar's new organization, LatinoProsperity, focuses on intergenerational wealth building, and he believed cutting-edge AI chatbots such as ChatGPT would become an integral part of many good-paying jobs of the future. But after reading about OpenAI's desires, he worried that transitioning into a public-benefit corporation empowered to chase profits would enrich the already wealthy and neglect the startup's stated mission to benefit all of humanity with AI. Aguilar decided to make a phone call that day, kicking off a series of events that eventually led him to become one of the leading voices battling over OpenAI's future and the establishment of what may become the deepest-pocketed charitable foundation in the world. Today, OpenAI's for-profit business is controlled by a nonprofit, and the returns for investors are capped.


Your deepest-held beliefs form a pattern than can be predicted by AI

New Scientist

Are the Harry Potter books any good? Your position on these thorny questions and more can now be predicted by an artificial intelligence model that was trained on the beliefs of more than 40,000 people โ€“ and could potentially be used for mass manipulation. Large language models (LLMs) like ChatGPT already create maps of words and their meanings, to such an extent that calculations can be performed on them: taking the word "king", subtracting "man" and adding "woman"โ€ฆ


'Jeopardy' host Ken Jennings 'deeply skeptical' of AI, years after losing to supercomputer

FOX News

"Jeopardy!" host Ken Jennings tells Fox News Digital he wants to know a human is behind any creative projects, not AI. "I'm deeply skeptical of AI," Jennings told Fox News Digital at the TCM Classic Film Festival. "Obviously, these current iterations of LLMs [Large Language Models] would clean Watson's clock at'Jeopardy!' The technology has moved on. I've played with chatbots and'Jeopardy!' clues, and they're very hard to stump," he said.


Dystopian eye-scanning tech rolls out in five US states to track your money, identity and every move

Daily Mail - Science & tech

The boss of the AI tool ChatGPT has revealed that his eyeball-scanning orbs are coming to the US, as questions still swirl around this dystopian step into the future. Sam Altman announced Wednesday that the identity verification technology will now be available in six cities - Atlanta, Austin, Los Angeles, Miami, Nashville, and San Francisco. The expansion into the US is all part of Altman's plan to create a new global identity and financial network. Currently, Altman's cryptocurrency company World has rolled out the orb devices in more than 35 cities across over 20 countries worldwide. The main purpose of these eyeball scanners is to verify that each user is a'unique human,' not a bot or duplicate account.


Learning to Plan Before Answering: Self-Teaching LLMs to Learn Abstract Plans for Problem Solving

arXiv.org Artificial Intelligence

In the field of large language model (LLM) post-training, the effectiveness of utilizing synthetic data generated by the LLM itself has been well-presented. However, a key question remains unaddressed: what essential information should such self-generated data encapsulate? Existing approaches only produce step-by-step problem solutions, and fail to capture the abstract meta-knowledge necessary for generalization across similar problems. Drawing insights from cognitive science, where humans employ high-level abstraction to simplify complex problems before delving into specifics, we introduce a novel self-training algorithm: LEarning to Plan before Answering (LEPA). LEPA trains the LLM to formulate anticipatory plans, which serve as abstract meta-knowledge for problem-solving, before engaging with the intricacies of problems. This approach not only outlines the solution generation path but also shields the LLM from the distraction of irrelevant details. During data generation, LEPA first crafts an anticipatory plan based on the problem, and then generates a solution that aligns with both the plan and the problem. LEPA refines the plan through self-reflection, aiming to acquire plans that are instrumental in yielding correct solutions. During model optimization, the LLM is trained to predict both the refined plans and the corresponding solutions. By efficiently extracting and utilizing the anticipatory plans, LEPA demonstrates remarkable superiority over conventional algorithms on various challenging natural language reasoning benchmarks.


Can Language Models Represent the Past without Anachronism?

arXiv.org Artificial Intelligence

Before researchers can use language models to simulate the past, they need to understand the risk of anachronism. We find that prompting a contemporary model with examples of period prose does not produce output consistent with period style. Fine-tuning produces results that are stylistically convincing enough to fool an automated judge, but human evaluators can still distinguish fine-tuned model outputs from authentic historical text. We tentatively conclude that pretraining on period prose may be required in order to reliably simulate historical perspectives for social research.


Keep the General, Inject the Specific: Structured Dialogue Fine-Tuning for Knowledge Injection without Catastrophic Forgetting

arXiv.org Artificial Intelligence

Large Vision Language Models have demonstrated impressive versatile capabilities through extensive multimodal pre-training, but face significant limitations when incorporating specialized knowledge domains beyond their training distribution. These models struggle with a fundamental dilemma: direct adaptation approaches that inject domain-specific knowledge often trigger catastrophic forgetting of foundational visual-linguistic abilities. We introduce Structured Dialogue Fine-Tuning (SDFT), an effective approach that effectively injects domain-specific knowledge while minimizing catastrophic forgetting. Drawing inspiration from supervised fine-tuning in LLMs and subject-driven personalization in text-to-image diffusion models, our method employs a three-phase dialogue structure: Foundation Preservation reinforces pre-trained visual-linguistic alignment through caption tasks; Contrastive Disambiguation introduces carefully designed counterfactual examples to maintain semantic boundaries; and Knowledge Specialization embeds specialized information through chain-of-thought reasoning. Experimental results across multiple domains confirm SDFT's effectiveness in balancing specialized knowledge acquisition with general capability retention. Our key contributions include a data-centric dialogue template that balances foundational alignment with targeted knowledge integration, a weighted multi-turn supervision framework, and comprehensive evaluation across diverse knowledge types.