Goto

Collaborating Authors

 pattern description


Knowledge-Augmented Vision Language Models for Underwater Bioacoustic Spectrogram Analysis

arXiv.org Artificial Intelligence

Marine mammals depend on acoustic communication for navigation, social interactions, and finding food across vast ocean environments. As climate change and human activities threaten many species with extinction, understanding these vocalizations has become essential for conservation efforts under Sustainable Development Goal 14 - Life Below Water. Automatically classifying marine mammal sounds from recordings presents major challenges. Underwater soundscapes are complex, each species has unique vocal patterns, and interpreting acoustic features requires biological expertise. Current approaches face a three-way trade-off between performance, cost, and interpretability.


An Exploration of Pattern Mining with ChatGPT

arXiv.org Artificial Intelligence

The paper is organized as follows. First, I review existing approaches to pattern mining, human-AI collaboration with Chat-This paper takes an exploratory approach to examine the use of GPT, and patterns for LLMs. I then describe an eight-step process ChatGPT for pattern mining. It proposes an eight-step collaborative for mining patterns from known uses with ChatGPT. Next, I present process that combines human insight with AI capabilities to the application of the process to create a pattern language for integrating extract patterns from known uses. The paper offers a practical LLMs with data sources and tools.Ithen describe the patterns demonstration of this process by creating a pattern language for that were created and subsequently revised using ChatGPT.


Verbalized Graph Representation Learning: A Fully Interpretable Graph Model Based on Large Language Models Throughout the Entire Process

arXiv.org Artificial Intelligence

Representation learning on text-attributed graphs (TAGs) has attracted significant interest due to its wide-ranging real-world applications, particularly through Graph Neural Networks (GNNs). Traditional GNN methods focus on encoding the structural information of graphs, often using shallow text embeddings for node or edge attributes. This limits the model to understand the rich semantic information in the data and its reasoning ability for complex downstream tasks, while also lacking interpretability. With the rise of large language models (LLMs), an increasing number of studies are combining them with GNNs for graph representation learning and downstream tasks. While these approaches effectively leverage the rich semantic information in TAGs datasets, their main drawback is that they are only partially interpretable, which limits their application in critical fields. In this paper, we propose a verbalized graph representation learning (VGRL) method which is fully interpretable. In contrast to traditional graph machine learning models, which are usually optimized within a continuous parameter space, VGRL constrains this parameter space to be text description which ensures complete interpretability throughout the entire process, making it easier for users to understand and trust the decisions of the model. We conduct several studies to empirically evaluate the effectiveness of VGRL and we believe these method can serve as a stepping stone in graph representation learning.


Verbalized Machine Learning: Revisiting Machine Learning with Language Models

arXiv.org Artificial Intelligence

Motivated by the large progress made by large language models (LLMs), we introduce the framework of verbalized machine learning (VML). In contrast to conventional machine learning models that are typically optimized over a continuous parameter space, VML constrains the parameter space to be human-interpretable natural language. Such a constraint leads to a new perspective of function approximation, where an LLM with a text prompt can be viewed as a function parameterized by the text prompt. Guided by this perspective, we revisit classical machine learning problems, such as regression and classification, and find that these problems can be solved by an LLM-parameterized learner and optimizer. The major advantages of VML include (1) easy encoding of inductive bias: prior knowledge about the problem and hypothesis class can be encoded in natural language and fed into the LLM-parameterized learner; (2) automatic model class selection: the optimizer can automatically select a concrete model class based on data and verbalized prior knowledge, and it can update the model class during training; and (3) interpretable learner updates: the LLM-parameterized optimizer can provide explanations for why each learner update is performed. We conduct several studies to empirically evaluate the effectiveness of VML, and hope that VML can serve as a stepping stone to stronger interpretability and trustworthiness in ML.