Music/voice separation

Info: site splits the audio track into separate components: voice, music, drums, guitar, piano, etc. Examples of a track divided into two parts (voice and music) can be seen in the video below. You can also check results of separation on our demo-page.

News [2021.10.19]: Two new algorithms have been added to mvsep.com for separate tracks: MDX A and MDX B. These models were created by the participants in the Music Demuxing Challenge who took second place. Their solution code and neural network models were made publicly available. We are still waiting for the first place solution. But even these models significantly outperform Spleeter and UmxXL in competition metrics (see the table below), but slower in speed. MDX A differs from MDX B in that the first algorithm did not use external data for training, so the results are slightly worse than MDX B.
The values ​​in the table are calculated on private competition dataset (available only to organizers). The higher the value, the better the algorithm works.

Algorithm	Quality (Bass)	Quality (Drums)	Quality (Other)	Quality (Vocals)	Example
Spleeter (4 stems)	5.774	5.845	4.321	6.939	Example
UmxXL	6.619	6.838	4.891	7.732	Example
MDX A	7.232	7.173	5.636	8.901	Example
MDX B (Orig)	7.495	7.554	5.533	8.896	---
MDX B (MVSep)	7.495	7.554	5.533	9.482	Example

News [2021.08.30]: There are some useful updates on mvsep.com:

• Updated software and site code. Splitting tracks is faster and more stable. Our backend crashes are less and less common.
• Added a new splitting algorithm called UnMix. The algorithm has 4 models "umxXL", "umxHQ", "umxSD", "umxSE". The highest quality is the first "umxXL". According to the first tests, the voice separates a little worse than the spleeter, but the instruments are better. In any case, a large field is now open for experimenting with tracks.
• The page with the split results has been redesigned: an original track has been added, it is convenient to compare from one page. Added information on sharing settings, displays information on the uploaded file, ID3 tags and an image (if any).

Examples of separation based on the new algorithm:
umxXL: Monk Turner Fascinoma - Its Your Birthday
umxHQ: Robin Grey - These Days
umxSD: Brad Sucks - Total Breakdown
umxSE: Paper Navy - Swan Song

And finally, some statistics. About 600-750 tracks are divided on the site per day. And for all the time, more than 300,000 tracks have been split. Moving towards a million.

Other projects: blur faces on photo and video - hide-face.com

×