Goto

Collaborating Authors

 data interpretation


Can AI agents understand spoken conversations about data visualizations in online meetings?

Sharma, Rizul, Jiang, Tianyu, Lee, Seokki, Aurisano, Jillian

arXiv.org Artificial Intelligence

In this short paper, we present work evaluating an AI agent's understanding of spoken conversations about data visualizations in an online meeting scenario. There is growing interest in the development of AI-assistants that support meetings, such as by providing assistance with tasks or summarizing a discussion. The quality of this support depends on a model that understands the conversational dialogue. To evaluate this understanding, we introduce a dual-axis testing framework for diagnosing the AI agent's comprehension of spoken conversations about data. Using this framework, we designed a series of tests to evaluate understanding of a novel corpus of 72 spoken conversational dialogues about data visualizations. We examine diverse pipelines and model architectures, LLM vs VLM, and diverse input formats for visualizations (the chart image, its underlying source code, or a hybrid of both) to see how this affects model performance on our tests. Using our evaluation methods, we found that text-only input modalities achieved the best performance (96%) in understanding discussions of visualizations in online meetings.


Towards a Personal Health Large Language Model

Cosentino, Justin, Belyaeva, Anastasiya, Liu, Xin, Furlotte, Nicholas A., Yang, Zhun, Lee, Chace, Schenck, Erik, Patel, Yojan, Cui, Jian, Schneider, Logan Douglas, Bryant, Robby, Gomes, Ryan G., Jiang, Allen, Lee, Roy, Liu, Yun, Perez, Javier, Rogers, Jameson K., Speed, Cathy, Tailor, Shyam, Walker, Megan, Yu, Jeffrey, Althoff, Tim, Heneghan, Conor, Hernandez, John, Malhotra, Mark, Stern, Leor, Matias, Yossi, Corrado, Greg S., Patel, Shwetak, Shetty, Shravya, Zhan, Jiening, Prabhakara, Shruthi, McDuff, Daniel, McLean, Cory Y.

arXiv.org Artificial Intelligence

In health, most large language model (LLM) research has focused on clinical tasks. However, mobile and wearable devices, which are rarely integrated into such tasks, provide rich, longitudinal data for personal health monitoring. Here we present Personal Health Large Language Model (PH-LLM), fine-tuned from Gemini for understanding and reasoning over numerical time-series personal health data. We created and curated three datasets that test 1) production of personalized insights and recommendations from sleep patterns, physical activity, and physiological responses, 2) expert domain knowledge, and 3) prediction of self-reported sleep outcomes. For the first task we designed 857 case studies in collaboration with domain experts to assess real-world scenarios in sleep and fitness. Through comprehensive evaluation of domain-specific rubrics, we observed that Gemini Ultra 1.0 and PH-LLM are not statistically different from expert performance in fitness and, while experts remain superior for sleep, fine-tuning PH-LLM provided significant improvements in using relevant domain knowledge and personalizing information for sleep insights. We evaluated PH-LLM domain knowledge using multiple choice sleep medicine and fitness examinations. PH-LLM achieved 79% on sleep and 88% on fitness, exceeding average scores from a sample of human experts. Finally, we trained PH-LLM to predict self-reported sleep quality outcomes from textual and multimodal encoding representations of wearable data, and demonstrate that multimodal encoding is required to match performance of specialized discriminative models. Although further development and evaluation are necessary in the safety-critical personal health domain, these results demonstrate both the broad knowledge and capabilities of Gemini models and the benefit of contextualizing physiological data for personal health applications as done with PH-LLM.


What Career Opportunities are Available in Artificial Intelligence?

#artificialintelligence

Artificial Intelligence is a growing field and it provides a number of career opportunities. To know more about it read this post. Artificial Intelligence refers to the combination of computer science and robust datasets that facilitates problem-solving. In addition, it is a simulation of human intelligence that is capable of building machines capable of performing tasks. Artificial Intelligence makes a computer or a software capable of thinking just like the human mind.


What is Artificial Intelligence? How does AI work, Types, Trends and Future of it?

#artificialintelligence

Let's take a detailed look. This is the most common form of AI that you'd find in the market now. These Artificial Intelligence systems are designed to solve one single problem and would be able to execute a single task really well. By definition, they have narrow capabilities, like recommending a product for an e-commerce user or predicting the weather. This is the only kind of Artificial Intelligence that exists today. They're able to come close to human functioning in very specific contexts, and even surpass them in many instances, but only excelling in very controlled environments with a limited set of parameters. AGI is still a theoretical concept. It's defined as AI which has a human-level of cognitive function, across a wide variety of domains such as language processing, image processing, computational functioning and reasoning and so on.


Top 10 AI Jobs You Will Fit in if You Have a Computer Science Degree

#artificialintelligence

Your computer science degree will give you the opportunity to develop your commercial skills and boost your career in the field of AI. Jobs in artificial intelligence require advanced computer science skills. Since artificial intelligence is a developing field, there are many AI jobs that you will fit in if you have a computer science degree. Have a look at the following jobs. AI engineers are problem-solvers who develop, test and apply different models of Artificial Intelligence.


Data Interpretation Support in Rescue Operations: Application for French Firefighters

Chehade, Samer, Matta, Nada, Pothin, Jean-Baptiste, Cogranne, Rémi

arXiv.org Artificial Intelligence

--This work aims at developing a system that supports French firefighters in data interpretation during rescue operations. An application ontology is proposed based on existing crisis management ones and operational expertise collection. After that, a knowledge-based system will be developed and integrated in firefighters' environment. Our first studies are shown in this paper. Rescue of people consists in saving their life in case of distress situations by applying responsive operations. In France, it is defined as specific tasks to be accomplished by public services in order to ensure the safety of patients and victims by making them able to escape from dangers, securing intervention sites, providing medical help, and finally, ensuring the evacuation to an appropriate place of reception [1].


How To Boost Your ROI By 223% With Conversion Optimization Tools

#artificialintelligence

Conversion optimization tools are estimated to have an average ROI of 223%. And that is totally expected as they're largely responsible for most conversions and revenue. However, CRO tools are more expensive than many other marketing tools, too (as you'll soon see in this article). And there are so many of them out there. But at the end of the day, it's not using these tools that matters but what they do for your business.


Machine learning perspectives on Mexico's digital transformation

#artificialintelligence

Abstract: Wide reaching and continuously evolving value propositions are the gears of network orchestrator's business models (NOBMs). Digital transformation enters when reliable data and meaningful information became digital enabler (DE) fuel of NOBMs. Moreover, Machine Learning (ML) capabilities can work as a catalyzer to increase knowledge rate acquisition for business processes or economical activities. This paper sets up DE and ML example binds for 4 different industries, proposing that high-quality data obtained from a rich context augments the profitability of the model. Finally, we conclude that the highly variable context from México provides an ideal environment in which ML augments harmonization between DE and NOBMs.


Five reasons why data growth will outstrip processing for the foreseeable future

#artificialintelligence

A while back, I documented what I called, with no small amount of hubris, "Jonno's first Law" – namely that data will always be created at a greater rate than it can be processed. This principle, I believe, is fundamental to why we will fail to see the ultimate vision of artificial intelligence (which I first studied at university 30 years ago) become reality, perhaps for some decades. So, what is driving the'law'? Most simply that Moore's Law, which states that the number of transistors on a chip will double periodically, is not the only principle at play. Other principles are economic, contextual and consequences of the way we choose to create data.