Irregularly sampled multivariate time series (ISMTS) are prevalent in reality. Most existing methods treat ISMTS as synchronized regularly sampled time series with missing values, neglecting that the irregularities are primarily attributed to variations in sampling rates. In this paper, we introduce a novel perspective that irregularity is essentially relative in some senses. With sampling rates artificially determined from low to high, an irregularly sampled time series can be transformed into a hierarchical set of relatively regular time series from coarse to fine. We observe that additional coarse-grained relatively regular series not only mitigate the irregularly sampled challenges to some extent but also incorporate broad-view temporal information, thereby serving as a valuable asset for representation learning. Therefore, following the philosophy of learning that Seeing the big picture first, then delving into the details, we present the Multi-Scale and Multi-Correlation Attention Network (MuSiCNet) combining multiple scales to iteratively refine the ISMTS representation. Specifically, within each scale, we explore time attention and frequency correlation matrices to aggregate intra- and inter-series information, naturally enhancing the representation quality with richer and more intrinsic details. While across adjacent scales, we employ a representation rectification method containing contrastive learning and reconstruction results adjustment to further improve representation consistency. MuSiCNet is an ISMTS analysis framework that competitive with SOTA in three mainstream tasks consistently, including classification, interpolation, and forecasting.

This paper introduces a new large-scale music dataset, MusicNet, to serve as a source of supervision and evaluation of machine learning methods for music research. MusicNet consists of hundreds of freely-licensed classical music recordings by 10 composers, written for 11 instruments, together with instrument/note annotations resulting in over 1 million temporal labels on 34 hours of chamber music performances under various studio and microphone conditions. The paper defines a multi-label classification task to predict notes in musical recordings, along with an evaluation protocol, and benchmarks several machine learning architectures for this task: i) learning from spectrogram features; ii) end-to-end learning with a neural net; iii) end-to-end learning with a convolutional neural net. These experiments show that end-to-end models trained for note prediction learn frequency selective filters as a low-level representation of audio.

More broadly, we hope that MusicNet can be a resource for more creative tasks. Automatic music transcription, inferring a musical score from a recording, is a long-standing open problem in the music information retrieval community. Music streaming services traditionally make recommendations based on collaborative filtering and metadata (e.g. Recently, some services have begun to incorporate audio features into their recommendation engines. Features learned from the MusicNet labels might be useful for recommendation.

Are consciousness and emotion essential components for creating music? Johann Sebastian Bach never completed his 18th-century work "The Art of Fugue," but now a computer might do it for him. University of Washington researchers on Wednesday released MusicNet, a large-scale classical music dataset aimed at helping machines understand the basic structure of classical music -- and even predict the next notes in a recording. The publicly available dataset includes 330 classical music recordings, along with more than a million annotated markers, verified by trained musicians, that indicate the timing of each note, the instrument that plays it and the note's position in a composition's metrical structure. "At a high level, we're interested in what makes music appealing to the ears, how we can better understand composition, or the essence of what makes Bach sound like Bach," Sham Kakade, a UW associate professor of computer science, engineering and statistics, said in a statement.

What makes Bach sound like Bach? MusicNet is a new publicly available dataset from UW researchers that labels each note of 330 classical compositions in ways that can teach machine learning algorithms about the basic structure of music.Yngve Bakken Nilsen, flickr The composer Johann Sebastian Bach left behind an incomplete fugue upon his death, either as an unfinished work or perhaps as a puzzle for future composers to solve. A classical music dataset released Wednesday by University of Washington researchers -- which enables machine learning algorithms to learn the features of classical music from scratch -- raises the likelihood that a computer could expertly finish the job. MusicNet is the first publicly available large-scale classical music dataset with curated fine-level annotations. It's designed to allow machine learning researchers and algorithms to tackle a wide range of open challenges -- from note prediction to automated music transcription to offering listening recommendations based on the structure of a song a person likes, instead of relying on generic tags or what other customers have purchased. "At a high level, we're interested in what makes music appealing to the ears, how we can better understand composition, or the essence of what makes Bach sound like Bach. It can also help enable practical applications that remain challenging, like automatic transcription of a live performance into a written score," said Sham Kakade, a UW associate professor of computer science and engineering and of statistics.