The neural semi-Markov Conditional Random Field (semi-CRF) framework has demonstrated promise for event-based piano transcription. In this framework, all events (notes or pedals) are represented as closed intervals tied to specific event types. The neural semi-CRF approach requires an interval scoring matrix that assigns a score for every candidate interval. However, designing an efficient and expressive architecture for scoring intervals is not trivial. In this paper, we introduce a simple method for scoring intervals using scaled inner product operations that resemble how attention scoring is done in transformers. We show theoretically that, due to the special structure from encoding the non-overlapping intervals, under a mild condition, the inner product operations are expressive enough to represent an ideal scoring matrix that can yield the correct transcription result. We then demonstrate that an encoder-only non-hierarchical transformer backbone, operating only on a low-time-resolution feature map, is capable of transcribing piano notes and pedals with high accuracy and time precision. The experiment shows that our approach achieves the new state-of-the-art performance across all subtasks in terms of the F1 measure on the Maestro dataset.

Large language models require updates to remain up-to-date or adapt to new domains by fine-tuning them with new documents. One key is memorizing the latest information in a way that the memorized information is extractable with a query prompt. However, LLMs suffer from a phenomenon called "perplexity curse"; despite minimizing document perplexity during fine-tuning, LLMs struggle to extract information through a prompt sentence. In this new knowledge acquisition and extraction, we find a very intriguing fact that LLMs can accurately answer questions about the first sentence, but they struggle to extract information described in the middle or end of the documents used for fine-tuning. Our study suggests that the auto-regressive training causes this issue; each token is prompted by reliance on all previous tokens, which hinders the model from recalling information from training documents by question prompts. To conduct the in-depth study, we publish both synthetic and real datasets, enabling the evaluation of the QA performance w.r.t. the position of the corresponding answer in a document. Our investigation shows that even a large model suffers from the "perplexity curse", but regularization such as denoising auto-regressive loss can enhance the information extraction from diverse positions. These findings will be (i) a key to improving knowledge extraction from LLMs and (ii) new elements to discuss the trade-off between RAG and fine-tuning in adapting LLMs to a new domain.

We introduce a novel framework of LLM agents named AGILE (AGent that Interacts and Learns from Environments) designed to perform complex conversational tasks with users, leveraging LLMs, memory, tools, and interactions with experts. The agent's abilities include not only conversation but also reflection, utilization of tools, and consultation with experts. We formulate the construction of such an LLM agent as a reinforcement learning problem, in which the LLM serves as the policy model. We fine-tune the LLM using labeled data of actions and the PPO algorithm. We focus on question answering and release a dataset for agents called ProductQA, comprising challenging questions in online shopping. Our extensive experiments on ProductQA and MedMCQA show that AGILE agents based on 13B and 7B LLMs trained with PPO can outperform GPT-4 agents. Our ablation study highlights the indispensability of memory, tools, consultation, reflection, and reinforcement learning in achieving the agent's strong performance.

OpenAI and News Corp, the owner of The Wall Street Journal, MarketWatch, The Sun, and more than a dozen other publishing brands, have struck a multi-year deal to display news from these publications in ChatGPT, News Corp announced on Wednesday. OpenAI will be able to access both current and well as archived content from News Corp's publications and use the data to further train its AI models. Neither company disclosed the terms of the deal, but a report in The Wall Street Journal estimated that News Corp would get 250 million over five years in cash and credits. "The pact acknowledges that there is a premium for premium journalism," News Corp Chief Executive Robert Thomson reportedly said in a memo to employees on Wednesday. "The digital age has been characterized by the dominance of distributors, often at the expense of creators, and many media companies have been swept away by a remorseless technological tide. The onus is now on us to make the most of this providential opportunity."

ChatGPT developer OpenAI has signed a deal to bring news content from the Wall Street Journal, New York Post, the Times and the Sunday Times to the artificial intelligence platform, the companies said on Wednesday. Neither party disclosed a dollar figure for the deal. The deal will give OpenAI access to current and archived content from all of News Corp's publications. The deal comes weeks after the AI heavyweight signed a deal with the Financial Times to license its content for the development of AI models. Other publications, including the New York Times, have taken a different tack: suing OpenAI and Microsoft, the startup's key backer, over the use of its content to train generative AI and large-language model systems.

It seems like every Z-list celebrity and reality star has a podcast these days, but can you tell what makes this one truly unique? Eagle-eyed viewers may have spotted that its co-host, Jake, appears a bit stiff and robotic - and it's not because he's camera shy. Jake is a fully artificial intelligence-generated avatar. Human podcast host Jakob Wredstrøm, who created Jake as a digital clone of himself, says his is the first AI podcast. Jakob Wredstrøm, the creator of the podcast Sound Connections, created an AI version of himself called Jake (pictured).

The last time we saw Imperator Furiosa, in the dystopian chase thriller "Mad Max: Fury Road" (2015), she had just returned from the heat of battle, her face streaked with blood, one eye swollen shut, her body so fatigued and battered that she could hardly stand. Furiosa, played by a stupendous Charlize Theron, had spent several days and nights driving an enormous truck, the War Rig, across miles of open desert, withstanding fiery assaults, a lethal sandstorm, and the surly company of a reluctant ally named Max (Tom Hardy). But triumph, at last, was hers: the vile warlord Immortan Joe (Hugh Keays-Byrne) lay dead at her feet, and hundreds of newly liberated desert dwellers were erupting in celebration. Amid the chaos, Furiosa scanned the crowd for Max and caught him slinking away. For a moment, he looked back and gave her an approving nod--then turned and vanished into the throng.

On May 13th, during a live event, the artificial-intelligence company OpenAI unveiled the next generation of its technology, GPT-4o, the successor to GPT-3. When OpenAI first released its product to the public in late 2022, as the text-based tool ChatGPT, it nearly single-handedly ushered in the A.I. era. The latest version is far more powerful still. The "o" in the name stands for "omni"; the model can communicate seamlessly across various forms of media at once, including text, audio, and video, receiving prompts in one medium and responding in another. It can maintain a memory of everything you tell it.

'IN DISBELIEF': Scarlett Johansson is "angered and in disbelief" by tech company OpenAI over its ChatGPT app's voice, Sky, noting it uses a voice very similar to hers. NVIDIA'S RISE: Wall Street is eagerly awaiting the latest earnings report Wednesday from Nvidia, which has experienced rapid growth amid the boom in artificial intelligence technology. ELECTRIC RUNNING ROBOT: Standing as tall as an average human and powered by a symphony of sensors and processors, Tiangong has the ability to jog at a steady pace, navigate complex terrain and perform tasks with precision. Tiangong represents a future where robots could possibly become our companions, helpers and perhaps even our friends. GOOGLE'S BIG REVEALS: Google's flagship developer conference called I/O just wrapped up with interesting leaps in how the Big Tech giant is planning to change the world.

UPenn Wharton School Associate Professor Ethan Mollick weighs in on the Biden White House's new guidelines for artificial intelligence in the workplace on'Fox News Live.' South Korea's science and information technology minister said on Wednesday the world must cooperate to ensure the successful development of AI, as a global summit on the rapidly evolving technology hosted by his country wrapped up. A separate pledge was signed on Wednesday by 14 companies including Alphabet's Google, Microsoft, OpenAI and six Korean companies to use methods such as watermarking to help identify AI-generated content, as well as ensure job creation and help for socially vulnerable groups. "Cooperation is not an option, it is a necessity," Lee Jong-Ho, South Korea's Minister of Science and ICT (information and communication technologies), said in an interview with Reuters. Han Duck-soo, South Korean Prime Minister, gives a speech during the opening ceremony of the AI Global Forum in Seoul, South Korea, on May 22, 2024. South Korea's science and information technology minister said on Wednesday the world must cooperate to ensure the successful development of AI, as the summit on the rapidly evolving technology hosted by his country wrapped up.