Music & Voice Separation

1) New algorithm MVSep Wind was added. It's used for extraction of wind instruments from music tracks. Wind instruments include 2 categories of instruments: brass and woodwind. More specifically, we included in wind instruments: flute, saxophone, trumpet, trombone, French horn, clarinet, oboe, harmonica, bagpipes, bassoon, tuba, kazoo, piccolo, flugelhorn, ocarina, shakuhachi, melodica, reed, didgeridoo, mussette, gaida. We prepared two models based on SCNet and MelBand Roformer architectures. Quality metrics are given in the table below:

Listen to: user demos

2) A new model for drums based on the SCNet neural network has been prepared. It gives the best results among all other models. We have also updated the ensembles that include drums. See the results in the table below:

Listen to: user demos

3) Added new algorithm MVSep Strings. It is a model based on MDX23C architecture for separating music into bowed string instruments and everything else. SDR metric: 3.84. We plan to prepare other architectures later.

Listen to: user demos

4) Added a new experimental algorithm for Phantom Center extraction by wesleyr36. According to the author, the algorithm extracts the phantom center from stereo sound, i.e. content that is the same for both channels and is perceived as being in the middle.

Listen to: user demos

5) Added 2 new Mel Roformer variations for vocal extraction: ver 2024.10 (SDR vocals: 11.28, SDR instrum: 17.59) - which improved the result on Multisong Leaderboard. And unwa Instrumental (SDR vocals: 10.24, SDR instrum: 16.54) - which, although noisy, gives a more complete picture for the instrumental part.

Listen to: user demos 1, user demos 2

6) Added new models SCNet and MelBand Roformer, trained on DnR v3 dataset. They are designed for "cinematic" separation of tracks into stems: speech, music and sfx. The metrics turned out better than those of the similar model Bandit v2. You can see the metrics in the table below:

Listen to: user demos

7) Added a new model for removing the reverb effect from Sucial. The model works only with vocals. It is available for selection in the algorithm "Reverb Removal (noreverb)" under the name "Reverb removal by Sucial (MelRoformer)"

Listen to: user demos

8) The "MVSep Multichannel BS (vocals, instrumental)" algorithm, which has proven itself excellent for separating multichannel tracks (Surround 5.1/7.1), was moved from the experimental section to the HQ Models section. We also added the top vocal model MelBand Roformer for selection. The peculiarity of this algorithm is that when using it, the number of channels does not decrease after separation, and the Sample Rate remains identical to the original.

9) Medley Vox algorithm has been added. Medley Vox initially is a dataset for testing algorithms for separating multiple singers within a single music track. Also, the authors of Medley Vox proposed a neural network architecture for separating singers. However, unfortunately, they did not publish the weights. Later, their training process was repeated by Cyru5, who trained several models and published the weights in the public domain. Now the trained neural network is available on MVSep. The algorithm works with low Sample Rate audio, but can be useful in some cases.

10) A large set of new datasets was released to test the quality of the models:
- Piano: https://mvsep.com/quality_checker/leaderboard/piano/
- Guitar: https://mvsep.com/quality_checker/leaderboard/guitar/
- Medley Vox: https://mvsep.com/quality_checker/leaderboard/medley/
- Strings: https://mvsep.com/quality_checker/leaderboard/strings/
- Wind: https://mvsep.com/quality_checker/leaderboard/wind/
- DNR v3 Test: https://mvsep.com/quality_checker/leaderboard/dnr_v3/
- Super Resolution Checker for Music: https://mvsep.com/quality_checker/leaderboard/super_res_music/