Remastering Divide and Remaster: A Cinematic Audio Source Separation Dataset with Multilingual Support
Watcharasupat, Karn N., Wu, Chih-Wei, Orife, Iroro
–arXiv.org Artificial Intelligence
Cinematic audio source separation (CASS) is a relatively new subtask of audio source separation, concerned with the separation of a mixture into the dialogue, music, and effects stems. To date, only one publicly available dataset exists for CASS, that is, the Divide and Remaster (DnR) dataset, which is currently at version 2. While DnR v2 has been an incredibly useful resource for CASS, several areas of improvement have been identified, particularly through its use in the 2023 Sound Demixing Challenge. In this work, we develop version 3 of the DnR dataset, addressing issues relating to vocal content in non-dialogue stems, loudness distributions, mastering process, and linguistic diversity. In particular, the dialogue stem of DnR v3 includes speech content from more than 30 languages from multiple families including but not limited to the Germanic, Romance, Indo-Aryan, Dravidian, Malayo-Polynesian, and Bantu families. Benchmark results using the Bandit model indicated that training on multilingual data yields significant generalizability to the model even in languages with low data availability. Even in languages with high data availability, the multilingual model often performs on par or better than dedicated models trained on monolingual CASS datasets.
arXiv.org Artificial Intelligence
Jul-9-2024
- Country:
- South America
- Oceania > Australia
- Queensland (0.04)
- North America
- Central America (0.04)
- Puerto Rico (0.04)
- United States
- Georgia > Fulton County
- Atlanta (0.04)
- California > Santa Clara County
- Los Gatos (0.04)
- Georgia > Fulton County
- Europe
- Spain (0.04)
- Ukraine (0.04)
- Italy (0.04)
- Faroe Islands (0.04)
- Central Europe (0.04)
- United Kingdom > England
- Greater London > London (0.04)
- Cambridgeshire > Cambridge (0.04)
- France > Provence-Alpes-Côte d'Azur
- Bouches-du-Rhône > Marseille (0.05)
- Asia
- Malaysia (0.14)
- Southeast Asia (0.04)
- Myanmar (0.04)
- China (0.04)
- Japan (0.04)
- Indonesia > Sumatra (0.04)
- Nepal (0.04)
- Cambodia (0.04)
- India
- Telangana (0.04)
- Tamil Nadu (0.04)
- Maharashtra (0.04)
- Kerala (0.04)
- Karnataka (0.04)
- Gujarat (0.04)
- Andhra Pradesh (0.04)
- Africa
- South Africa (0.04)
- Lesotho (0.04)
- Botswana (0.04)
- Nigeria > Lagos State
- Lagos (0.04)
- Genre:
- Research Report (0.40)
- Industry:
- Leisure & Entertainment (1.00)
- Media > Film (0.47)
- Technology: