Muscriptor
MuScriptor is an open-weight model for general-purpose, multi-instrument automatic music transcription, developed by Kyutai and Mirelo AI. It converts a music recording into a MIDI file: for every note it finds pitch, start and end time and the instrument that plays it. MVSep uses the largest and most accurate version of the model (MuScriptor large, ~1.3B parameters, decoder-only Transformer).
The model recognizes 34 groups of melodic instruments plus drums. Piano, guitars, bass and drums are transcribed best, because they are the most frequent in the training data. Tempo and meter of the track are detected automatically and written into the MIDI file.
Options
- Quantize notes - snap notes to the detected beat grid. Useful for notation editors and DAWs. Works best for recordings with a steady tempo; for rubato or live recordings without a fixed tempo, keep this option off.
- Instruments - optionally restrict transcription to selected instruments (e.g. only piano, bass and drums). Notes of other instruments will not appear in the result. Leave empty to let the model detect all instruments itself.
Supported instruments
acoustic piano, electric piano, chromatic percussion (celesta, glockenspiel, vibraphone, marimba, xylophone, bells), organ (incl. accordion and harmonica), acoustic guitar, clean electric guitar, distorted electric guitar, acoustic bass, electric bass, violin, viola, cello, contrabass, orchestral harp, timpani, string ensemble, synth strings, voice / choir, orchestra hit, trumpet, trombone, tuba, french horn, brass section, soprano & alto sax, tenor sax, baritone sax, oboe, english horn, bassoon, clarinet, flutes (incl. piccolo, recorder, pan flute), synth lead, synth pad, drums.
Limitations
- Note velocity (loudness) is not predicted — all notes have the same velocity.
- Drums are transcribed as onsets only (hits without duration).
- Within a group, the exact instrument is not distinguished (e.g. any electric bass is written as one bass program).
Quality
F1 scores (%) reported by the authors on their held-out test set of 372 tracks (instrument-agnostic, computed with mir_eval):
| Model | Onset F1 | Frame F1 | Offset F1 | Drums F1 | Multi-instrument F1 |
|---|---|---|---|---|---|
| YourMT3+ (baseline) | 32.5 | 45.5 | 17.8 | 41.4 | 21.9 |
| MuScriptor large | 60.4 | 72.4 | 48.6 | 49.6 | 47.8 |
Links
- Original page: github.com/muscriptor/muscriptor
- Model weights: huggingface.co/MuScriptor/muscriptor-large
- Paper: MuScriptor: An Open Model for Multi-Instrument Music Transcription (ISMIR 2026)
- Online demo by the authors: muscriptor.kyutai.org
© Kyutai, Mirelo AI. Authors: Rouard, Krause, Roebel, Simon-Gabriel, Défossez
