Automatic Music Transcription (AMT), aiming to get musical notes from raw audio, typically uses frame-level systems with piano-roll outputs or language model (LM)-based systems with note-level predictions. However, frame-level systems require manual thresholding, while the LM-based systems struggle with long sequences. In this paper, we propose a hybrid method combining pre-trained roll-based encoders with an LM decoder to leverage the strengths of both methods. Besides, our approach employs a hierarchical prediction strategy, first predicting onset and pitch, then velocity, and finally offset. The hierarchical prediction strategy reduces computational costs by breaking down long sequences into different hierarchies. Evaluated on two benchmark roll-based encoders, our method outperforms traditional piano-roll outputs 0.01 and 0.022 in onset-offset-velocity F1 score, demonstrating its potential as a performance-enhancing plug-in for arbitrary roll-based music transcription encoder.

Instrument playing techniques (IPTs) constitute a pivotal component of musical expression. However, the development of automatic IPT detection methods suffers from limited labeled data and inherent class imbalance issues. In this paper, we propose to apply a self-supervised learning model pre-trained on large-scale unlabeled music data and finetune it on IPT detection tasks. This approach addresses data scarcity and class imbalance challenges. Recognizing the significance of pitch in capturing the nuances of IPTs and the importance of onset in locating IPT events, we investigate multi-task finetuning with pitch and onset detection as auxiliary tasks. Additionally, we apply a post-processing approach for event-level prediction, where an IPT activation initiates an event only if the onset output confirms an onset in that frame. Our method outperforms prior approaches in both frame-level and event-level metrics across multiple IPT benchmark datasets. Further experiments demonstrate the efficacy of multi-task finetuning on each IPT class.

With the advancements in deep learning, deep neural networks have been increasingly used in more recent work [8, 9]. In [10], a Instrument playing technique (IPT) is a key element in enhancing convolutional recurrent neural network (CRNN) based model was the vividness of musical performance. As shown by the Guzheng proposed to classify IPTs in audio sequences concatenated by cello numbered musical notation (a musical notation system widely used notes from 5 sound banks. To alleviate the computational redundancy in China) in Fig.1, a complete automatic music transcription (AMT) caused by the sliding window in [10], Wang et al. [11] proposed system should contain IPT information in addition to pitch and onset a fully convolutional network (FCN) based end-to-end method information. IPT detection aims to classify the types of IPTs and to detect IPTs in segments concatenated by isolated Erhu notes. In locate the associated IPT boundaries in audio. IPT detection and [12], an additional onset detector was used, and its output was fused modeling can be utilized in many applications of music information with IPT prediction in a post-processing step to improve the accuracy retrieval (MIR), like performance analysis [1] and AMT [2]. of IPT detection from monophonic audio sequences concatenated by The research on IPT detection is still in its early stage.