done
COMMA: A Communicative Multimodal Multi-Agent Benchmark
Ossowski, Timothy, Chen, Jixuan, Maqbool, Danyal, Cai, Zefan, Bradshaw, Tyler, Hu, Junjie
The rapid advances of multi-modal agents built on large foundation models have largely overlooked their potential for language-based communication between agents in collaborative tasks. This oversight presents a critical gap in understanding their effectiveness in real-world deployments, particularly when communicating with humans. Existing agentic benchmarks fail to address key aspects of inter-agent communication and collaboration, particularly in scenarios where agents have unequal access to information and must work together to achieve tasks beyond the scope of individual capabilities. To fill this gap, we introduce a novel benchmark designed to evaluate the collaborative performance of multimodal multi-agent systems through language communication. Our benchmark features a variety of scenarios, providing a comprehensive evaluation across four key categories of agentic capability in a communicative collaboration setting. By testing both agent-agent and agent-human collaborations using open-source and closed-source models, our findings reveal surprising weaknesses in state-of-the-art models, including proprietary models like GPT-4o. These models struggle to outperform even a simple random agent baseline in agent-agent collaboration and only surpass the random baseline when a human is involved.
How To Create An AI-Ready Company Culture
Company cultures will need to change if AI is to thrive in them. An AI-ready culture will enable the right AI approaches to be matched with the right use cases, with minimal resistance, and with surrounding workflows and customer experiences adapted appropriately. Because AI is going to impact work in many little ways before it transforms companies in giant leaps, creating a path for those small successes will enable the big moves later on. And it won't be possible to supervise all these quick wins from the top, so they'll rely on a supportive culture to make them occur. How can you change the culture for AI even with employees fully aware that AI might make many jobs outmoded?
CoRL: Environment Creation and Management Focused on System Integration
Merrick, Justin D., Heiner, Benjamin K., Long, Cameron, Stieber, Brian, Fierro, Steve, Gangal, Vardaan, Blake, Madison, Blackburn, Joshua
Existing reinforcement learning environment libraries use monolithic environment classes, provide shallow methods for altering agent observation and action spaces, and/or are tied to a specific simulation environment. The Core Reinforcement Learning library (CoRL) is a modular, composable, and hyper-configurable environment creation tool. It allows minute control over agent observations, rewards, and done conditions through the use of easy-to-read configuration files, pydantic validators, and a functor design pattern. Using integration pathways allows agents to be quickly implemented in new simulation environments, encourages rapid exploration, and enables transition of knowledge from low-fidelity to high-fidelity simulations. Natively multi-agent design and integration with Ray/RLLib (Liang et al., 2018) at release allow for easy scalability of agent complexity and computing power. The code is publicly released and available at https://github.com/act3-ace/CoRL.
It's DONE: Direct ONE-shot learning with quantile weight imprinting
Hosoda, Kazufumi, Nishida, Keigo, Seno, Shigeto, Mashita, Tomohiro, Kashioka, Hideki, Ohzawa, Izumi
Learning a new concept from one example is a superior function of the human brain and it is drawing attention in the field of machine learning as a one-shot learning task. In this paper, we propose one of the simplest methods for this task with a nonparametric weight imprinting, named Direct ONE-shot learning (DONE). DONE adds new classes to a pretrained deep neural network (DNN) classifier with neither training optimization nor pretrained-DNN modification. DONE is inspired by Hebbian theory and directly uses the neural activity input of the final dense layer obtained from data that belongs to the new additional class as the synaptic weight with a newly-provided-output neuron for the new class, transforming all statistical properties of the neural activity into those of synaptic weight by quantile normalization. DONE requires just one inference for learning a new concept and its procedure is simple, deterministic, not requiring parameter tuning and hyperparameters. DONE overcomes a severe problem of existing weight imprinting methods that DNN-dependently interfere with the classification of original-class images. The performance of DONE depends entirely on the pretrained DNN model used as a backbone model, and we confirmed that DONE with current well-trained backbone models perform at a decent accuracy.
Compositional Semantic Parsing with Large Language Models
Drozdov, Andrew, Schärli, Nathanael, Akyürek, Ekin, Scales, Nathan, Song, Xinying, Chen, Xinyun, Bousquet, Olivier, Zhou, Denny
Humans can reason compositionally when presented with new tasks. Previous research shows that appropriate prompting techniques enable large language models (LLMs) to solve artificial compositional generalization tasks such as SCAN. In this work, we identify additional challenges in more realistic semantic parsing tasks with larger vocabulary and refine these prompting techniques to address them. Our best method is based on least-to-most prompting: it decomposes the problem using prompting-based syntactic parsing, then uses this decomposition to select appropriate exemplars and to sequentially generate the semantic parse. This method allows us to set a new state of the art for CFQ while requiring only 1% of the training data used by traditional approaches. Due to the general nature of our approach, we expect similar efforts will lead to new results in other tasks and domains, especially for knowledge-intensive applications.
To Make AI Fair, Here's What Must Be Done
With clear frameworks, we can weed out AI that perpetuates discrimination against the most vulnerable people and focus on building AI that makes society better. Across the world, many artificial intelligence (AI) regulations are in the works, intent on promoting equity, accountability, and transparency. But these will not be enough to make AI equitable. There needs to be practical know-how on how to build AI so that it does not exacerbate social inequality. That means setting out clear ways for social scientists, affected communities, and developers to work together.
Why Artificial Intelligence Lacks Creativity and What Can De Done to Help it
Artificial intelligence has worked its way into nearly every aspect of our lives. We've gone from envisioning a future with flying cars and robot butlers to living in a world with self-driving cars and voice assistants that we carry in our pockets. Despite this, the fact remains that AI is not as far advanced as it could be. The current AI algorithms are only able to imitate or copy information. For example, it can compose a sonata in the style of Debussy or replicate a poem by Pushkin, but it cannot infuse meaning or emotion into the composition.
Top Machine Learning Projects that Can be Done Using Python
Machine learning is a branch of artificial intelligence (AI) and computer science which focuses on the use of data and algorithms to imitate the way that humans learn, gradually improving its accuracy. Pylearn2 is a library designed to make machine learning research easy. The Numenta Platform for Intelligent Computing (NuPIC) is a machine intelligence platform that implements the HTM learning algorithms. HTM is a detailed computational theory of the neocortex. At the core of HTM are time-based continuous learning algorithms that store and recall spatial and temporal patterns.
There Is Work To Be Done: AI And The Future Of Work
Can robots and workers co-exist? Workers, policymakers, and the media are concerned with the idea that automation, or technological change, will displace millions of American workers--and they are partially right. Andrew Yang, an early 2020 Presidential hopeful is already running on the idea that "the robots are coming" – though the story is not so simple. There have been, and will continue to be, technological breakthroughs that replace workers and reshape our economy. The next big worker-displacing technology is supposedly artificial intelligence (AI), which is thought to have the potential to replace millions of workers performing routine and menial tasks.