Media
On the Evaluation of Machine-Generated Reports
Mayfield, James, Yang, Eugene, Lawrie, Dawn, MacAvaney, Sean, McNamee, Paul, Oard, Douglas W., Soldaini, Luca, Soboroff, Ian, Weller, Orion, Kayi, Efsun, Sanders, Kate, Mason, Marc, Hibbler, Noah
Large Language Models (LLMs) have enabled new ways to satisfy information needs. Although great strides have been made in applying them to settings like document ranking and short-form text generation, they still struggle to compose complete, accurate, and verifiable long-form reports. Reports with these qualities are necessary to satisfy the complex, nuanced, or multi-faceted information needs of users. In this perspective paper, we draw together opinions from industry and academia, and from a variety of related research areas, to present our vision for automatic report generation, and -- critically -- a flexible framework by which such reports can be evaluated. In contrast with other summarization tasks, automatic report generation starts with a detailed description of an information need, stating the necessary background, requirements, and scope of the report. Further, the generated reports should be complete, accurate, and verifiable. These qualities, which are desirable -- if not required -- in many analytic report-writing settings, require rethinking how to build and evaluate systems that exhibit these qualities. To foster new efforts in building these systems, we present an evaluation framework that draws on ideas found in various evaluations. To test completeness and accuracy, the framework uses nuggets of information, expressed as questions and answers, that need to be part of any high-quality generated report. Additionally, evaluation of citations that map claims made in the report to their source documents ensures verifiability.
Steve Albini Was an Icon of Punk-Rock Purity--but He Also Showed How You Could Evolve
There's a quote by the legendary producer Steve Albini, whose death, of a heart attack at age 61, was announced earlier Wednesday, that's been rattling around in my head for so many decades that it's been paraphrased beyond recognition, or at least the reach of search engines. It goes, vaguely, like this: There is nothing better than the sound of a drum kit in an empty room. As a producer, Albini, who recorded albums by Nirvana, PJ Harvey, Pixies, the Breeders, and hundreds of others, was renowned as an icon of punk-rock purity, one who approached the process of making albums and the business of selling them with the same uncompromising approach. When Nirvana was overwhelmed by the runaway success of their major-label debut, Nevermind, they turned to Albini for the follow-up, In Utero, hoping that his abrasive approach would help reduce their fan base to a more manageable level. Instead of jumping at the chance to work with the hottest band in the world, Albini responded with a lengthy letter outlining the circumstances under which he would, and would not, make an album with them.
Fox News AI Newsletter: Katy Perry says fake Met Gala photos fooled her mom
'The Big Weekend Show' analyzes the possibilities of artificial intelligence when it comes to influencing voters. NEW YORK, NEW YORK - MAY 02: Katy Perry attends The 2022 Met Gala Celebrating "In America: An Anthology of Fashion" at The Metropolitan Museum of Art on May 02, 2022 in New York City. IT'S SUPERNATURAL: A picture of Perry at the bottom of the Met steps circulated online, leading fans to believe the "Wide Awake" singer was attending the event. In the picture, Perry is wearing an off-white ball gown adorned with roses and moss. GROWING WITH AI: Over 3,000 micro business owners were surveyed by Venture Forward, GoDaddy's international research initiative, in February 2024 about leveraging generative artificial intelligence to compete with large brands and level the playing field across a multitude of industries.
Comparative Study of Recurrent Neural Networks for Virtual Analog Audio Effects Modeling
Simionato, Riccardo, Fasciani, Stefano
Analog electronic circuits are at the core of an important category of musical devices. The nonlinear features of their electronic components give analog musical devices a distinctive timbre and sound quality, making them highly desirable. Artificial neural networks have rapidly gained popularity for the emulation of analog audio effects circuits, particularly recurrent networks. While neural approaches have been successful in accurately modeling distortion circuits, they require architectural improvements that account for parameter conditioning and low latency response. In this article, we explore the application of recent machine learning advancements for virtual analog modeling. We compare State Space models and Linear Recurrent Units against the more common Long Short Term Memory networks. These have shown promising ability in sequence to sequence modeling tasks, showing a notable improvement in signal history encoding. Our comparative study uses these black box neural modeling techniques with a variety of audio effects. We evaluate the performance and limitations using multiple metrics aiming to assess the models' ability to accurately replicate energy envelopes, frequency contents, and transients in the audio signal. To incorporate control parameters we employ the Feature wise Linear Modulation method. Long Short Term Memory networks exhibit better accuracy in emulating distortions and equalizers, while the State Space model, followed by Long Short Term Memory networks when integrated in an encoder decoder structure, outperforms others in emulating saturation and compression. When considering long time variant characteristics, the State Space model demonstrates the greatest accuracy. The Long Short Term Memory and, in particular, Linear Recurrent Unit networks present more tendency to introduce audio artifacts.
Dynamic Data Layout Optimization with Worst-case Guarantees
Rong, Kexin, Liu, Paul, Sonje, Sarah Ashok, Charikar, Moses
Many data analytics systems store and process large datasets in partitions containing millions of rows. By mapping rows to partitions in an optimized way, it is possible to improve query performance by skipping over large numbers of irrelevant partitions during query processing. This mapping is referred to as a data layout. Recent works have shown that customizing the data layout to the anticipated query workload greatly improves query performance, but the performance benefits may disappear if the workload changes. Reorganizing data layouts to accommodate workload drift can resolve this issue, but reorganization costs could exceed query savings if not done carefully. In this paper, we present an algorithmic framework OReO that makes online reorganization decisions to balance the benefits of improved query performance with the costs of reorganization. Our framework extends results from Metrical Task Systems to provide a tight bound on the worst-case performance guarantee for online reorganization, without prior knowledge of the query workload. Through evaluation on real-world datasets and query workloads, our experiments demonstrate that online reorganization with OReO can lead to an up to 32% improvement in combined query and reorganization time compared to using a single, optimized data layout for the entire workload.
An LSTM-Based Chord Generation System Using Chroma Histogram Representations
This paper proposes a system for chord generation to monophonic symbolic melodies using an LSTM-based model trained on chroma histogram representations of chords. Chroma representations promise more harmonically rich generation than chord label-based approaches, whilst maintaining a small number of dimensions in the dataset. This system is shown to be suitable for limited real-time use. While it does not meet the state-of-the-art for coherent long-term generation, it does show diatonic generation with cadential chord relationships. The need for further study into chroma histograms as an extracted feature in chord generation tasks is highlighted.
SVDD Challenge 2024: A Singing Voice Deepfake Detection Challenge Evaluation Plan
Zhang, You, Zang, Yongyi, Shi, Jiatong, Yamamoto, Ryuichi, Han, Jionghao, Tang, Yuxun, Toda, Tomoki, Duan, Zhiyao
The rapid advancement of AI-generated singing voices, which now closely mimic natural human singing and align seamlessly with musical scores, has led to heightened concerns for artists and the music industry. Unlike spoken voice, singing voice presents unique challenges due to its musical nature and the presence of strong background music, making singing voice deepfake detection (SVDD) a specialized field requiring focused attention. To promote SVDD research, we recently proposed the "SVDD Challenge," the very first research challenge focusing on SVDD for lab-controlled and in-the-wild bonafide and deepfake singing voice recordings. The challenge will be held in conjunction with the 2024 IEEE Spoken Language Technology Workshop (SLT 2024).
ATDM:An Anthropomorphic Aerial Tendon-driven Manipulator with Low-Inertia and High-Stiffness
Xu, Quman, Li, Zhan, Li, Hai, Yu, Xinghu, Yang, Yipeng
Aerial Manipulator Systems (AMS) have garnered significant interest for their utility in aerial operations. Nonetheless, challenges related to the manipulator's limited stiffness and the coupling disturbance with manipulator movement persist. This paper introduces the Aerial Tendon-Driven Manipulator (ATDM), an innovative AMS that integrates a hexrotor Unmanned Aerial Vehicle (UAV) with a 4-degree-of-freedom (4-DOF) anthropomorphic tendon-driven manipulator. The design of the manipulator is anatomically inspired, emulating the human arm anatomy from the shoulder joint downward. To enhance the structural integrity and performance, finite element topology optimization and lattice optimization are employed on the links to replicate the radially graded structure characteristic of bone, this approach effectively reduces weight and inertia while simultaneously maximizing stiffness. A novel tensioning mechanism with adjustable tension is introduced to address cable relaxation, and a Tension-amplification tendon mechanism is implemented to increase the manipulator's overall stiffness and output. The paper presents a kinematic model based on virtual coupled joints, a comprehensive workspace analysis, and detailed calculations of output torques and stiffness for individual arm joints. The prototype arm has a total weight of 2.7 kg, with the end effector contributing only 0.818 kg. By positioning all actuators at the base, coupling disturbance are minimized. The paper includes a detailed mechanical design and validates the system's performance through semi-physical multi-body dynamics simulations, confirming the efficacy of the proposed design.
Attention-Driven Training-Free Efficiency Enhancement of Diffusion Models
Wang, Hongjie, Liu, Difan, Kang, Yan, Li, Yijun, Lin, Zhe, Jha, Niraj K., Liu, Yuchen
Diffusion Models (DMs) have exhibited superior performance in generating high-quality and diverse images. However, this exceptional performance comes at the cost of expensive architectural design, particularly due to the attention module heavily used in leading models. Existing works mainly adopt a retraining process to enhance DM efficiency. This is computationally expensive and not very scalable. To this end, we introduce the Attention-driven Training-free Efficient Diffusion Model (AT-EDM) framework that leverages attention maps to perform run-time pruning of redundant tokens, without the need for any retraining. Specifically, for single-denoising-step pruning, we develop a novel ranking algorithm, Generalized Weighted Page Rank (G-WPR), to identify redundant tokens, and a similarity-based recovery method to restore tokens for the convolution operation. In addition, we propose a Denoising-Steps-Aware Pruning (DSAP) approach to adjust the pruning budget across different denoising timesteps for better generation quality. Extensive evaluations show that AT-EDM performs favorably against prior art in terms of efficiency (e.g., 38.8% FLOPs saving and up to 1.53x speed-up over Stable Diffusion XL) while maintaining nearly the same FID and CLIP scores as the full model. Project webpage: https://atedm.github.io.
Locally Differentially Private In-Context Learning
Zheng, Chunyan, Sun, Keke, Zhao, Wenhao, Zhou, Haibo, Jiang, Lixin, Song, Shaoyang, Zhou, Chunlai
Large pretrained language models (LLMs) have shown surprising In-Context Learning (ICL) ability. An important application in deploying large language models is to augment LLMs with a private database for some specific task. The main problem with this promising commercial use is that LLMs have been shown to memorize their training data and their prompt data are vulnerable to membership inference attacks (MIA) and prompt leaking attacks. In order to deal with this problem, we treat LLMs as untrusted in privacy and propose a locally differentially private framework of in-context learning (LDP-ICL) in the settings where labels are sensitive. Considering the mechanisms of in-context learning in Transformers by gradient descent, we provide an analysis of the trade-off between privacy and utility in such LDP-ICL for classification. Moreover, we apply LDP-ICL to the discrete distribution estimation problem. In the end, we perform several experiments to demonstrate our analysis results.