Automatic Multitrack Mixing With A Differentiable Mixing Console Of Neural Audio Effects

Applications of deep learning to automatic multitrack mixing are largely unexplored. This is partly due to the limited available data, coupled with the fact that such data is relatively unstructured and variable. To address these challenges, we propose a domain-inspired model with a strong inductive...

Full description

Saved in:
Bibliographic Details
Main Authors: Steinmetz, Christian J., Pons, Jordi, Pascual, Santiago, Serra, Joan
Format: Conference Proceeding
Language:English
Subjects:
automatic mixing
Conferences
Convolution
Deep learning
Intelligent music production
Multiple signal classification
Music
Speech processing
Task analysis
temporal convolutional network
Online Access:Request full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Applications of deep learning to automatic multitrack mixing are largely unexplored. This is partly due to the limited available data, coupled with the fact that such data is relatively unstructured and variable. To address these challenges, we propose a domain-inspired model with a strong inductive bias for the mixing task. We achieve this with the application of pre-trained sub-networks and weight sharing, as well as with a sum/difference stereo loss function. The proposed model can be trained with a limited number of examples, is permutation invariant with respect to the input ordering, and places no limit on the number of input sources. Furthermore, it produces human-readable mixing parameters, allowing users to manually adjust or refine the generated mix. Results from a perceptual evaluation involving audio engineers indicate that our approach generates mixes that outperform baseline approaches. To the best of our knowledge, this work demonstrates the first approach in learning multitrack mixing conventions from real-world data at the waveform level, without knowledge of the underlying mixing parameters.
ISSN:2379-190X
DOI:10.1109/ICASSP39728.2021.9414364