Seperating a mixed audio source from one channel is very difficult. However, recent years AI has allowed us to effectively perform this task. For this project I would like to specifically un-mix vocals from a guitar track. In this project I have decided to experiment with using Deep Learning techniques for audio source separation. A Deep Neural Network architecture and approach is is based off of "A Multi-Source Separation Approach Based on DOA Cue and DNN" by Zhang et al. In this approach a time-frequency mask in binary representation of a guitar and vocal track is used as the target for the deep neural network to train on.
Short-time Fourier Transform Representation of the Instrument and Vocals unmixed
The training and validation dataset consists of a vocal and guitar sample at 16kHz sampling rate. The guitar and vocal tracks are mixed and used as the training and validation sets fed into the neural network. A time frequency representation of the training and validation sets will be taken. An FFT size of 256 using a Hanning window with length 256 is used in the STFT analysis. From the time frequency representation a binary mask is made and then both the mask and the original is fed into a neural network with the Deep Neural architecture presented by Zhang et al. The architecture consists of four fully connected layers(in the paper a convolutional layer was used however I decided to replace the convolutional layer with a fully connected layer), three batch normalization layers, dropout layers and Relu layers, and one sigmoid layer followed by a regression layer at the end of the network. Each dropout layer is assigned a weight of 0.1. The network utilizes the Adam optimizer running 4 epochs with a batch size of 64. L2 regularization was increased to prevent overfitting. The Deep Neural Network is trained using both the binary mask and time frequency representation and was based off minimizing the root mean squared error.
Snippet of the Deep Neural Network layers
Dataset Training used to find the least Root Mean Squared Error
This trained neural network now can be used to predict the binary mask of any incoming vocal and guitar track to separate them into two channels. The output sound file using just the simple Neural Network reveals artifacts in the output data. Thus a final mask, being the previous mask with a threshold, is created to provide a cleaner output.
Instrument output with final predicted mask layer
Vocal output with final predicted mask layer