Music/voice separation

Info: site splits the audio track into separate components: voice, music, drums, guitar, piano, etc. Examples of a track divided into two parts (voice and music) can be seen in the video below. You can also check results of separation on our demo-page.

News [2022.07.29]

1. An experimental MVSep DNR algorithm has been added to the site, which divides tracks into 3 parts: music, special effects and voice. The algorithm was trained on the "Divide and Remaster" dataset. Quality Metrics:
   SDR DNR for music: 6.17
   SDR DNR for sfx: 7.26
   SDR DNR for speech: 14.13
   The algorithm is not well suited for ordinary music, but it does a good job when you need clean the voice of the speaker from extraneous noise in the background.
   Examples of the MVSep DNR algorithm
   

News [2022.07.25]

1. We created independent synthetic dataset to compare different music source separation algorithms. We published dataset here as well as automatic judging test system. Also leaderboard of best algorithms is available.
   
2. New MDX-B UVR vocal model was added. It's latest reelease from UVR Team. You have ability to choose it during selecting MDX-B algorithm in form.
   

News [2022.07.07]. Recent changes on MVSep:

1. New models from Ultimate Vocal Remover based on demucs3 architecture were added. It's available by name UVR Demucs in algorithm list.
   

Quality metrics for algorithms including UVD Demucs can be found here.

News [2022.04.18]. Recent changes on MVSep:

1. New algorithm Danna Sep was added. It's algorithm which got 3rd place on Leaderboard A in Sony Music Demixing Challenge
2. New algorithm Byte Dance was added. This algorithm took second place in the vocals category on Leaderboard A in the Sony Music Demixing Challenge. It's trained only on the MUSDB18HQ data and has potential in the future if more training data is added.

Quality metrics for these and other algorithms can be found here.

News [2022.02.24]. Recent changes on MVSep:

1. New UVR models: Piano, Bass, Drums and several different Vocal models were added. Possibility to set aggressivness was added for UVR models.
2. New remote GPU servers were added to process queue. Size of queue must be reduced.
3. Instrumental stem was added for spleeter (vocals, drums, bass, other) and spleeter (vocals, drums, bass, piano, other).

News [2021.12.23]. Recent changes on MVSep:

1. Added the ability to select lossless encoding of the created audio-files. Previously, it was possible to use only MP3. Now we added output to WAV and FLAC.
2. Added the output of the general instrumental track for all main algorithms: MDX, Demucs3 and Unmix.
3. Added translation of the site into Polish and Indonesian.
4. Added an automatic script to reset the GPU in case of errors. There should be no longer large server downtime.

Unfortunately, all the highest quality algorithms work very slow. Large queues are periodically formed because of that. We think what to do with this.

News [2021.11.12]: We have three big news:

1. We had to move to a new server due to lack of space on the old one. Positive effect - the video card has been changed to a more powerful one with more memory. As a result, the waiting queues have decreased and there are fewer errors associated with a lack of GPU memory. The downside is that server costs have doubled.

2. A new algorithm has been added Ultimate Vocal Remover (UVR). It splits the track into two parts, music and vocals. UVR usually does it better than spleeter. There are a lot of models and different settings in the original UVR. We have chosen one of the best models and optimal settings. Perhaps later, a flexible choice of settings for the algorithm will be added.

3. The winner of the Music Demuxing Challenge has finally released his code. We added its models to the site under the names Demux3 Model A and Demux3 Model B. Demux3 Model B gives a better result, and works better for bass and drums comparing to other models, but is slightly inferior in vocals to the MDX-B algorithm.

Below is an updated table comparing the quality of algorithms (data for UVR are not available). The values ​​in the table are calculated on private Music Demuxing Challenge dataset (available only to organizers). The higher the value, the better the algorithm works.

Algorithm	Quality (Bass)	Quality (Drums)	Quality (Other)	Quality (Vocals)	Example
Spleeter (4 stems)	5.774	5.845	4.321	6.939	Example
UmxXL	6.619	6.838	4.891	7.732	Example
MDX A	7.232	7.173	5.636	8.901	Example
MDX B (Orig)	7.495	7.554	5.533	8.896	---
MDX B (UVR)	7.495	7.554	5.533	9.482	Example
Ultimate Vocal Remover HQ	---	---	---	---	Example
Demucs 3 Model A	8.115	8.037	5.193	7.968	Example
Demucs 3 Model B	8.856	8.850	5.978	8.756	Example

News [2021.10.19]: Two new algorithms have been added to mvsep.com for separate tracks: MDX A and MDX B. These models were created by the participants in the Music Demuxing Challenge who took second place. Their solution code and neural network models were made publicly available. We are still waiting for the first place solution. But even these models significantly outperform Spleeter and UmxXL in competition metrics (see the table above), but slower in speed. MDX A differs from MDX B in that the first algorithm did not use external data for training, so the results are slightly worse than MDX B. Later, the enthusiasts of the UVR project improved the vocal separation model, getting a better value for the quality metric (8.896 -> 9.482).

News [2021.08.30]: There are some useful updates on mvsep.com:

• Updated software and site code. Splitting tracks is faster and more stable. Our backend crashes are less and less common.
• Added a new splitting algorithm called UnMix. The algorithm has 4 models "umxXL", "umxHQ", "umxSD", "umxSE". The highest quality is the first "umxXL". According to the first tests, the voice separates a little worse than the spleeter, but the instruments are better. In any case, a large field is now open for experimenting with tracks.
• The page with the split results has been redesigned: an original track has been added, it is convenient to compare from one page. Added information on sharing settings, displays information on the uploaded file, ID3 tags and an image (if any).

Examples of separation based on the new algorithm:
umxXL: Monk Turner Fascinoma - Its Your Birthday
umxHQ: Robin Grey - These Days
umxSD: Brad Sucks - Total Breakdown
umxSE: Paper Navy - Swan Song

And finally, some statistics. About 600-750 tracks are divided on the site per day. And for all the time, more than 300,000 tracks have been split. Moving towards a million.

Other projects: blur faces on photo and video - hide-face.com

×