Hammond, Kristian J.
Satyrn: A Platform for Analytics Augmented Generation
Sterbentz, Marko, Barrie, Cameron, Shahi, Shubham, Dutta, Abhratanu, Hooshmand, Donna, Pack, Harper, Hammond, Kristian J.
Large language models (LLMs) are capable of producing documents, and retrieval augmented generation (RAG) has shown itself to be a powerful method for improving accuracy without sacrificing fluency. However, not all information can be retrieved from text. We propose an approach that uses the analysis of structured data to generate fact sets that are used to guide generation in much the same way that retrieved documents are used in RAG. This analytics augmented generation (AAG) approach supports the ability to utilize standard analytic techniques to generate facts that are then converted to text and passed to an LLM. We present a neurosymbolic platform, Satyrn that leverages AAG to produce accurate, fluent, and coherent reports grounded in large scale databases. In our experiments, we find that Satyrn generates reports in which over 86% accurate claims while maintaining high levels of fluency and coherence, even when using smaller language models such as Mistral-7B, as compared to GPT-4 Code Interpreter in which just 57% of claims are accurate.
The Sixth International Conference on Intelligent User Interfaces
Hammond, Kristian J.
The chapters in this book examine the state of today's agent technology and point the way toward the exciting developments of the next millennium. Contributors include Donald A. Norman, Nicholas Negroponte, Brenda Laurel, Thomas Erickson, Ben Shneiderman, Thomas W. Malone, Pattie Maes, David C. Smith, Gene Ball, Guy A. Boy, Doug Riecken, Yoav Shoham, Tim Finin, Michael R. Genesereth, Craig A. Knoblock, Philip R. Cohen, Hector J. Levesque, and James E. White, among others. He then went on to outline that drive a field forward. Francisco's W Hotel, the conference not, to succeed when placed in front Along with the program committee, included work from researchers and of real users. He argued that we are the conference web site and online have faced increasingly challenging now living in a time where we can submissions and reviewing.
Question Answering from Frequently Asked Question Files: Experiences with the FAQ FINDER System
Burke, Robin D., Hammond, Kristian J., Kulyukin, Vladimir, Lytinen, Steven L., Tomuro, Noriko, Schoenberg, Scott
This article describes FAQ FINDER, a natural language question-answering system that uses files of frequently asked questions as its knowledge base. Unlike AI question-answering systems that focus on the generation of new answers, FAQ FINDER retrieves existing ones found in frequently asked question files. Unlike information-retrieval approaches that rely on a purely lexical metric of similarity between query and document, FAQ FINDER uses a semantic knowledge base (WORDNET) to improve its ability to match question and answer. We include results from an evaluation of the system's performance and show that a combination of semantic and statistical techniques works better than any single approach.
Question Answering from Frequently Asked Question Files: Experiences with the FAQ FINDER System
Burke, Robin D., Hammond, Kristian J., Kulyukin, Vladimir, Lytinen, Steven L., Tomuro, Noriko, Schoenberg, Scott
This article describes FAQ FINDER, a natural language question-answering system that uses files of frequently asked questions as its knowledge base. Unlike AI question-answering systems that focus on the generation of new answers, FAQ FINDER retrieves existing ones found in frequently asked question files. Unlike information-retrieval approaches that rely on a purely lexical metric of similarity between query and document, FAQ FINDER uses a semantic knowledge base (WORDNET) to improve its ability to match question and answer. We include results from an evaluation of the system's performance and show that a combination of semantic and statistical techniques works better than any single approach.