An important task in machine learning (ML) research is comparing prior work, which is often performed via ML leaderboards: a tabular overview of experiments with comparable conditions (e.g., same task, dataset, and metric). However, the growing volume of literature creates challenges in creating and maintaining these leaderboards. To ease this burden, researchers have developed methods to extract leaderboard entries from research papers for automated leaderboard curation. Yet, prior work varies in problem framing, complicating comparisons and limiting real-world applicability. In this position paper, we present the first overview of Automatic Leaderboard Generation (ALG) research, identifying fundamental differences in assumptions, scope, and output formats. We propose an ALG unified conceptual framework to standardise how the ALG task is defined. We offer ALG benchmarking guidelines, including recommendations for datasets and metrics that promote fair, reproducible evaluation. Lastly, we outline challenges and new directions for ALG, such as, advocating for broader coverage by including all reported results and richer metadata.

To answer your question about the baseline, we experimented with two new sample audio generated by the same (Karplus-Strong) algorithm and tested against Alexa. The result is shown in Table.1. The musical audio does not fool Alexa. Thank you again for your constructive feedback! Currently, we are also trying to activate the wake-word using our adversary.

Efficient state space models (SSMs), such as linear recurrent neural networks and linear attention variants, offer computational advantages over Transformers but struggle with tasks requiring long-range in-context retrieval-like text copying, associative recall, and question answering over long contexts. Previous efforts to address these challenges have focused on architectural modifications, often reintroducing computational inefficiencies. In this paper, we propose a novel training procedure, Birdie, that significantly enhances the in-context retrieval capabilities of SSMs without altering their architecture. Our approach combines bidirectional input processing with dynamic mixtures of specialized pre-training objectives, optimized via reinforcement learning. We introduce a new bidirectional SSM architecture that seamlessly transitions from bidirectional context processing to causal generation. Experimental evaluations demonstrate that Birdie markedly improves performance on retrieval-intensive tasks such as multi-number phone book lookup, long paragraph question-answering, and infilling. This narrows the performance gap with Transformers, while retaining computational efficiency. Our findings highlight the importance of training procedures in leveraging the fixed-state capacity of SSMs, offering a new direction to advance their capabilities. All code and pre-trained models are available at https://www.github.com/samblouir/birdie, with support for JAX and PyTorch.

The 33rd International Joint Conference on Artificial Intelligence (IJCAI-24), which took place on Jeju Island, South Korea, has now drawn to a close. The first three days of the event saw the running of tutorials, workshops, and the doctorial consortium track. You can see our round-up of these here. The official opening ceremony of the conference marked the start of four days of invited talks, posters, oral presentations, panel discussions, and more. In this post, we give a flavour of this second part of the event.

The development of conversational agents to interact with patients and deliver clinical advice has attracted the interest of many researchers, particularly in light of the COVID-19 pandemic. The training of an end-to-end neural based dialog system, on the other hand, is hampered by a lack of multi-turn medical dialog corpus. We make the very first attempt to release a high-quality multi-turn Medical Dialog dataset relating to Covid-19 disease named CDialog, with over 1K conversations collected from the online medical counselling websites. We annotate each utterance of the conversation with seven different categories of medical entities, including diseases, symptoms, medical tests, medical history, remedies, medications and other aspects as additional labels. Finally, we propose a novel neural medical dialog system based on the CDialog dataset to advance future research on developing automated medical dialog systems. We use pre-trained language models for dialogue generation, incorporating annotated medical entities, to generate a virtual doctor's response that addresses the patient's query. Experimental results show that the proposed dialog models perform comparably better when supplemented with entity information and hence can improve the response quality.

While Mark Zuckerberg has been away in Washington, D.C. last week testifying about data misuse and cybersecurity, we can only assume that his personal robot assistant, Jarvis, has been helping run his household. Named after Iron Man's infamous aide, Zuckerberg has boasted before that his Jarvis can play music, control household appliances, and even entertain the kids. Jarvis learns from his human family -- their voices, their musical preferences, their preferred temperatures, and environment. So, is this the best that Artificial Intelligence (AI) can achieve? AI leaders will tell you: no way.

Thanks to the support of the Don and Betty Walker Scholarship, limited funding is available for supporting student travel, registration and accommodation. Please apply here by June 20th. Decisions will be announced on June 25th. We are looking for student volunteers to help out during the conference. In return for one day of service, you will be given complementary registration for the main conference.

At ICML last year and the year before the amount of capacity that needed to fit everyone on any single day was about 1500. My advice was to expect 2000 and have capacity for 2500 because "New York" and "Machine Learning". I was not involved in the venue negotiations, but my understanding is that they were difficult, with liabilities over 1M for IMLS the nonprofit which oversees ICML year to year. The result was a conference plan with a maximum capacity of 1800 for the main conference, a bit less for workshops, and perhaps 1000 for tutorials. Then the NIPS registration numbers came in: 3900 last winter.

The deadline for the discounted AAAI rate is June AAAI-13 invites short papers on latebreaking for volunteer applications is May 1, 19, 2013. A special student rate has al-developments in the field of 2013.