Did you know that soundtrack concerts are among the most popular for touring orchestras? A full third of the Royal Philharmonic Orchestra's first-time audience members are coming to the concert hall via their favourite series and movies – and video games. It is a huge cultural growth area, and one that may have gone unrecognised by the general public. "It is impossible to ignore video game music now," says Tommy Pearson, founder and artistic director of the inaugural London Soundtrack festival. "The sheer creativity and artistry in games is incredible, and it's been fascinating to see so many composers blossom in the genre."

She's not long on charisma or passion but keeps perfect rhythm and is never prone to temperamental outbursts against the musicians beneath her three batons. Meet MAiRA Pro S, the next-generation robot conductor who made her debut this weekend in Dresden. Her two performances in the eastern German city are intended to show off the latest advances in machine maestros, as well as music written explicitly to harness 21st-century technology. The artistic director of Dresden's Sinfoniker, Markus Rindt, said the intention was "not to replace human beings" but to perform complex music that human conductors would find impossible. The Sinfoniker, long known for innovation and political statements, is celebrating its 25th anniversary with the Robotersinfonie at the Hellerau hall in a concert divided into two parts, one purely human and, after the interval, one that is robot-led.

In an era where large language models (LLMs) are increasingly integrated into a wide range of everyday applications, research into these models' behavior has surged. However, due to the novelty of the field, clear methodological guidelines are lacking. This raises concerns about the replicability and generalizability of insights gained from research on LLM behavior. In this study, we discuss the potential risk of a replication crisis and support our concerns with a series of replication experiments focused on prompt engineering techniques purported to influence reasoning abilities in LLMs. We tested GPT-3.5, GPT-4o, Gemini 1.5 Pro, Claude 3 Opus, Llama 3-8B, and Llama 3-70B, on the chain-of-thought, EmotionPrompting, ExpertPrompting, Sandbagging, as well as Re-Reading prompt engineering techniques, using manually double-checked subsets of reasoning benchmarks including CommonsenseQA, CRT, NumGLUE, ScienceQA, and StrategyQA. Our findings reveal a general lack of statistically significant differences across nearly all techniques tested, highlighting, among others, several methodological weaknesses in previous research. We propose a forward-looking approach that includes developing robust methodologies for evaluating LLMs, establishing sound benchmarks, and designing rigorous experimental frameworks to ensure accurate and reliable assessments of model outputs.

Indonesian and Western gamelan ensembles are investigated with respect to performance differences. Thereby, the often exotistic history of this music in the West might be reflected in contemporary tonal system, articulation, or large-scale form differences. Analyzing recordings of four Western and five Indonesian orchestras with respect to tonal systems and timbre features and using self-organizing Kohonen map (SOM) as a machine learning algorithm, a clear clustering between Indonesian and Western ensembles appears using certain psychoacoustic features. These point to a reduced articulation and large-scale form variability of Western ensembles compared to Indonesian ones. The SOM also clusters the ensembles with respect to their tonal systems, but no clusters between Indonesian and Western ensembles can be found in this respect. Therefore, a clear analogy between lower articulatory variability and large-scale form variation and a more exostistic, mediative and calm performance expectation and reception of gamelan in the West therefore appears.

Text-to-SQL enables users to interact with databases using natural language, simplifying the retrieval and synthesis of information. Despite the remarkable success of large language models (LLMs) in translating natural language questions into SQL queries, widespread deployment remains limited due to two primary challenges. First, the effective use of text-to-SQL models depends on users' understanding of the model's capabilities-the scope of questions the model can correctly answer. Second, the absence of abstention mechanisms can lead to incorrect SQL generation going unnoticed, thereby undermining trust in the model's output. To enable wider deployment, it is crucial to address these challenges in model design and enhance model evaluation to build trust in the model's output. To this end, we introduce TrustSQL, a novel comprehensive benchmark designed to evaluate text-to-SQL reliability-defined as a model's ability to correctly handle any type of input question by generating correct SQL queries for feasible questions and abstaining from generating infeasible ones (e.g., due to schema incompatibility or functionalities beyond SQL). We evaluate existing methods using a novel penalty-based scoring metric with two modeling approaches: (1) pipeline-based methods combining SQL generators with infeasible question detectors and SQL error detectors for abstention; and (2) unified methods using a single model for the entire task. Our experimental results reveal that achieving high scores under severe penalties requires significant effort and provide a new perspective on developing text-to-SQL models for safer deployment. TrustSQL is available at https://github.com/glee4810/TrustSQL.

If your first thought, as we ushered in the New Year, was not of fresh starts and resolutions but of the crises looming in 2024 and beyond, the best antidote, culturally speaking, might be to lean into catastrophe. Metrograph has catered to the pessimists among us by curating a series entitled "The Future Looks Bright from Afar" (through Feb. 4), which promises a suite of sci-fi films marked by "grim prognostications" about mankind's trajectory. As it happens, the great new crowd-pleaser of the moment is also a disaster story, albeit one set firmly in the past. "Godzilla Minus One" follows a kamikaze pilot who shirks his duty in the final days of the Second World War--a decision that puts him in the path of the eponymous monster and, years later, leaves him uniquely motivated to stop its rampage through postwar Tokyo. Elevated by emotional and historical specificity as well as set pieces that belie its modest fifteen-million-dollar budget, Takashi Yamazaki's contribution to the Godzilla canon is simultaneously a study in survivor's guilt and a "Jaws"-style blockbuster, complete with the revelation that our protagonists are going to need a bigger boat.

We show that language model finetuning can be improved, sometimes dramatically, with a simple augmentation. NEFTune adds noise to the embedding vectors during training. Standard finetuning of LLaMA-2-7B using Alpaca achieves 29.79% on AlpacaEval, which rises to 64.69% using noisy embeddings. NEFTune also improves over strong baselines on modern instruction datasets. Models trained with Evol-Instruct see a 10% improvement, with ShareGPT an 8% improvement, and with OpenPlatypus an 8% improvement. Even powerful models further refined with RLHF such as LLaMA-2-Chat benefit from additional training with NEFTune. The ability of LLMs to follow detailed instructions is vital to their usefulness. Generative language models are typically trained on raw web data, and then subsequently fine-tuned on a comparatively small but carefully curated set of instruction data. Instruction fine-tuning is crucial to taming the power of LLMs, and the usefulness of a model is largely determined by our ability to get the most out of small instruction datasets. In this paper, we propose to add random noise to the embedding vectors of the training data during the forward pass of fine-tuning. We show that this simple trick can improve the outcome of instruction fine-tuning, often by a large margin, with no additional compute or data overhead. Noisy Embedding Instruction Fine Tuning (NEFTune), while simple, has a strong impact on downstream conversational quality. When a raw LLM like LLaMA-2-7B is finetuned with noisy embeddings, its performance on AlpacaEval improves from 29.8% to 64.7% (Figure 1) - an impressive boost of around 35 percentage points (Touvron et al., 2023b; Dubois et al., 2023). NEFTune leads to this surprising and large jump in performance on conversational tasks, maintaining performance on factual question answering baselines. This technique seems to be a free lunch for LLM fine-tuning. NEFTune leads to massive performance boosts across all of these datasets, showcasing the increased conversational quality of the generated answers. The earliest forms of instruction finetuning such as FLAN and T0 (Sanh et al., 2021; Wei et al., 2021) focused on cross-task generalization in language models. Encoder-decoder language models were finetuned on a broad range of NLP tasks (about 100) and then evaluated on a set of different tasks. This was later scaled up to include thousands of tasks, seeing further improvement over the original FLAN (Chung et al., 2022; Xu et al., 2022). Although these works showed that LLMs could be easily adapted to solve simple and classical NLP tasks, real-world scenarios require LLMs to provide free-form answers to open-ended queries. InstructGPT (Ouyang et al., 2022) was the first model to tackle open-ended queries with impressive performance. OpenAI further trained GPT-3 (Brown et al., 2020) using reinforcement learning from human feedback (RLHF) to align the model.

Having a computer do the work for you has become more and more common over time. But in the entertainment area, where a human is a creator, we want to avoid having too much influence on technology. On the other hand, inspiration is still important; we developed a virtual conductor that can generate an emotionally associated interpretation of known music work. This was done by surveying a set number of people to determine, which emotions were associated with a specific interpretation and instruments. As a result of machine learning this conductor was then able to achieve his goal. Unlike earlier studies of virtual conductors, which would replace the role of a human conductor, this new one is supposed to be an assisting tool for conductors. As a result, starting on a new interpretation will be easier because it streamlines research time and provides a technical perspective that can inspire new ideas. By using this technology as a supplement to human creativity, we can create richer, more nuanced interpretations of musical works.

Luk Kop didn't seem to have the makings of a musical prodigy. He didn't hum made-up tunes to himself as a youngster or shake his head when someone sang flat. He didn't build instruments out of sticks and gourds or blow trumpet solos as a five-year-old. He had a brief moment of fame as a child actor, in the Disney film "Operation Dumbo Drop," but grew into a sullen and ungainly teen. When the composer and instrumentalist Dave Soldier first met him, in Thailand, in 2000, Luk Kop spent most of his time eating grass and hanging around with the other elephants.

For the past 10 years or so, if you lived in a big city and fancied hearing an orchestra play something from Metal Gear Solid or Sonic the Hedgehog instead of the Romantic period, there has been no shortage of options. Touring orchestras have played music from games such as Pokémon, Final Fantasy and Assassin's Creed for appreciative audiences all over the world. The largest such series, Video Games Live, has been running since 2005 and has played over 400 shows in Los Angeles, Beijing, Sydney and elsewhere. But this summer, for the first time, video game music will be part of the BBC Proms season at the Royal Albert Hall in London. A concert on 1 August will feature orchestral selections and adaptations from soundtracks spanning gaming history, including The Legend of Zelda, Shadow of the Colossus and Battlefield 2042.