Benchmark plays a pivotal role in assessing the advancements of large language models (LLMs). While numerous benchmarks have been proposed to evaluate LLMs' capabilities, there is a notable absence of a dedicated benchmark for assessing their musical abilities. To address this gap, we present ZIQI-Eval, a comprehensive and large-scale music benchmark specifically designed to evaluate the music-related capabilities of LLMs. ZIQI-Eval encompasses a wide range of questions, covering 10 major categories and 56 subcategories, resulting in over 14,000 meticulously curated data entries. By leveraging ZIQI-Eval, we conduct a comprehensive evaluation over 16 LLMs to evaluate and analyze LLMs' performance in the domain of music. Results indicate that all LLMs perform poorly on the ZIQI-Eval benchmark, suggesting significant room for improvement in their musical capabilities. With ZIQI-Eval, we aim to provide a standardized and robust evaluation framework that facilitates a comprehensive assessment of LLMs' music-related abilities. The dataset is available at GitHub\footnote{https://github.com/zcli-charlie/ZIQI-Eval} and HuggingFace\footnote{https://huggingface.co/datasets/MYTH-Lab/ZIQI-Eval}.

While existing alignment paradigms have been integral in developing large language models (LLMs), LLMs often learn an averaged human preference and struggle to model diverse preferences across cultures, demographics, and communities. We propose Modular Pluralism, a modular framework based on multi-LLM collaboration for pluralistic alignment: it "plugs into" a base LLM a pool of smaller but specialized community LMs, where models collaborate in distinct modes to flexibility support three modes of pluralism: Overton, steerable, and distributional. Modular Pluralism is uniquely compatible with black-box LLMs and offers the modular control of adding new community LMs for previously underrepresented communities. We evaluate Modular Pluralism with six tasks and four datasets featuring questions/instructions with value-laden and perspective-informed responses. Extensive experiments demonstrate that Modular Pluralism advances the three pluralism objectives across six black-box and open-source LLMs. Further analysis reveals that LLMs are generally faithful to the inputs from smaller community LLMs, allowing seamless patching by adding a new community LM to better cover previously underrepresented communities.

Tool learning empowers large language models (LLMs) as agents to use external tools and extend their utility. Existing methods employ one single LLM-based agent to iteratively select and execute tools, thereafter incorporating execution results into the next action prediction. Despite their progress, these methods suffer from performance degradation when addressing practical tasks due to: (1) the pre-defined pipeline with restricted flexibility to calibrate incorrect actions, and (2) the struggle to adapt a general LLM-based agent to perform a variety of specialized actions. To mitigate these problems, we propose ConAgents, a Cooperative and interactive Agents framework, which coordinates three specialized agents for tool selection, tool execution, and action calibration separately. ConAgents introduces two communication protocols to enable the flexible cooperation of agents. To effectively generalize the ConAgents into open-source models, we also propose specialized action distillation, enhancing their ability to perform specialized actions in our framework. Our extensive experiments on three datasets show that the LLMs, when equipped with the ConAgents, outperform baselines with substantial improvement (i.e., up to 14% higher success rate).

Just a few weeks after Apple laid out its grand new AI project, it seems Amazon is getting in on the act too. The tech giant is about to fit its smart assistant Alexa with powerful generative AI capabilities that make it much smarter, according to a report. Alexa will be fitted with a'conversational generative AI', it says – although it's unclear what AI model this will actually be. It means she will be able to respond faster and in more human-like language in response to complicated prompts or queries. Amazon's big rivals in tech already have their own AI chatbots, including Google (Gemini) and X (Grok) while Microsoft and Apple have integrations with ChatGPT.

In R. C. Sherriff's novel The Hopkins Manuscript, readers are transported to a world 800 years after a cataclysmic event ended Western civilization. In pursuit of clues about a blank spot in their planet's history, scientists belonging to a new world order discover diary entries in a swamp-infested wasteland formerly known as England. For the inhabitants of this new empire, it is only through this record of a retired school teacher's humdrum rural life, his petty vanities and attempts to breed prize-winning chickens, that they begin to learn about 20th-century Britain. If I were to teach futuristic beings about life on earth, I once believed I could produce a time capsule more profound than Sherriff's small-minded protagonist, Edgar Hopkins. But scrolling through my decade-old Facebook posts this week, I was presented with the possibility that my legacy may be even more drab.

It's been a quiet week of news, but we've been feverishly testing Microsoft's new Surface Pro and Surface Laptop Copilot AI PCs. In this episode, Devindra and Sam will answer your questions about Microsoft's new hardware, and we'll deliver some of our first impressions. It turns out Microsoft may have finally gotten Windows on Arm support right! And some of the Copilot AI features are actually useful, surprisingly enough. But we'll have to wait a few months to test out the controversial Recall feature, which was pulled from the Copilot launch. Listen below or subscribe on your podcast app of choice. If you've got suggestions or topics you'd like covered on the show, be sure to email us or drop a note in the comments! And be sure to check out our other podcast, Engadget News! Wired report: AI search engine Perplexity is ignoring robots.txt Listener question: What do you do with 8 gig fiber home internet? Joining me today is Senior Writer from Engadget, Sam Rutherford. I'm doing okay because we finally have some Copilot Plus PCs. Sam has the Surface Laptop, I have the Surface Pro. And we've just started testing these things. They came in late and we're just like trying to get Arubia as quickly as we can for both of us, but we've got some impressions here. We're going to be taking some questions from our live stream. Cause it's a pretty light news week, but yeah, if you join us Thursday mornings, around 10 30 AM Eastern on our YouTube channel. You too can participate and ask us questions. See us show off some gadgets. We'll show off some stuff live from the Surface Pro. So if you're listening to this in audio form, go back and watch the video, cause you can actually see us test out some features and show off the hardware too. As always folks, if you're enjoying this podcast, please subscribe to us in iTunes or your podcatcher of choice, leave us a review in iTunes. That's always super helpful and drop us [00:01:00]an email at podcast at engadget. Question for you, Sam, what was your first impression upon tearing open the Surface Laptop? Sam: Right away I think it's good they didn't mess with the design. The design was never the issue for the Surfaces, they're, very beautifully crafted. And, opening up and this is going to sound like silly, but it's it functioned exactly like a windows 11 laptop is supposed to. And that was like, Hey, this is actually an improvement from, previous attempts at windows on arm right away. It seems like they, Microsoft has nailed all the important aspects.

Kurt "The Cyberguy" Knutsson explains how facial recognition technology can help you find your perfect match. Doing fun, exciting things with your partner is a great way to keep love alive. Especially once you've been dating for a long time, coming up with new date ideas can get a little more challenging, but it's important. "One of the things that I always say for keeping the love alive is doing new things, novel activities," said Jaime Bronstein, a relationship therapist from Illinois as well as a coach. She previously spoke to Fox News Digital in a phone interview.

The article presents some current observations (as of April 10, 2024) on the integration of AI-generated images within processes of media convergence. It draws on two different concepts of intermediality. Primary intermediality concepts are motivated by the object when a new type of technology develops the potential to become socially relevant as a media form and thus a socially, politically, or culturally important communicative factor. Due to their uncertain 'measurements' within the wider media ecology, however, the new, still potential media form appears hybrid. The "inter-" or "between-" of this initial intermediality moment thus refers to the questionable "site" and the questionable description of the potential media form between already existing technologies and cultural forms and their conceptual measurements. For secondary concepts of intermediality, in contrast, it can be assumed that the boundaries of media forms and their application have already been drawn and are reasonably undisputed. This then raises the question of intentional and staged references to AI imagery within other media forms and pictures. The article discusses indicators of both intermediality moments using current examples and controversies surrounding AI images. The thesis is that there can be no talk of a seamless 'integration' of AI images into the wider media landscape at the moment (within films, comic books, or video games, for example) - as one of countless other image production techniques - and that the medial 'site' of AI image circulation - at least where it is not a matter of deception, but rather their conscious use as AI images - especially in social media communication and in fan cultures, but with repercussions for the more general media ecology and image interpretation, insofar as the suspicion that an image could be AI-generated is now increasingly present as a "hermeneutics of suspicion".

Despite the significant progress of large language models (LLMs) in various tasks, they often produce factual errors due to their limited internal knowledge. Retrieval-Augmented Generation (RAG), which enhances LLMs with external knowledge sources, offers a promising solution. However, these methods can be misled by irrelevant paragraphs in retrieved documents. Due to the inherent uncertainty in LLM generation, inputting the entire document may introduce off-topic information, causing the model to deviate from the central topic and affecting the relevance of the generated content. To address these issues, we propose the Retrieve-Plan-Generation (RPG) framework. RPG generates plan tokens to guide subsequent generation in the plan stage. In the answer stage, the model selects relevant fine-grained paragraphs based on the plan and uses them for further answer generation. This plan-answer process is repeated iteratively until completion, enhancing generation relevance by focusing on specific topics. To implement this framework efficiently, we utilize a simple but effective multi-task prompt-tuning method, enabling the existing LLMs to handle both planning and answering. We comprehensively compare RPG with baselines across 5 knowledge-intensive generation tasks, demonstrating the effectiveness of our approach.

In the japanese anime industry, predicting whether an upcoming product will be popular is crucial. This paper presents a dataset and methods on predicting anime popularity using a multimodal textimage dataset constructed exclusively from freely available internet sources. The dataset was built following rigorous standards based on real-life investment experiences. A deep neural network architecture leveraging GPT-2 and ResNet-50 to embed the data was employed to investigate the correlation between the multimodal text-image input and a popularity score, discovering relevant strengths and weaknesses in the dataset. To measure the accuracy of the model, mean squared error (MSE) was used, obtaining a best result of 0.011 when considering all inputs and the full version of the deep neural network, compared to the benchmark MSE 0.412 obtained with traditional TF-IDF and PILtotensor vectorizations. This is the first proposal to address such task with multimodal datasets, revealing the substantial benefit of incorporating image information, even when a relatively small model (ResNet-50) was used to embed them.