Key contributions
01
Combinatorial music generation
The paper introduces this task: a model generates short note sequences from metadata, and the sequences are combined into a complete song.
02
Composer-made dataset
Professional composers wrote every sample by hand, following written guidelines to keep the data consistent.
03
12 types of metadata
These include BPM, key, instrument, and nine others. Two are unique to ComMU: track role and chord progression with extended chord qualities such as sus4 and maj7.
How it works
Stage 1
Generate note sequences
A model generates a note sequence for each set of metadata.
Stage 2
Combine into a song
For now, composers arrange the generated note sequences across tracks and time to complete a song. Automating this stage is future work.
Metadata
- BPM
- 80
- Key
- A minor
- Time signature
- 4/4
- Number of measures
- 4
- Genre
- cinematic
- Rhythm
- standard
- Track role
- main melody
- Pitch range
- mid
- Instrument
- violin
- Min/max velocity
- 2, 127
- Chord progression
- C-Am-F-G
Note sequence
aSLwZ5Ts
Combination
| t1 | t2 | t3 | ||
|---|---|---|---|---|
| Main melody | 1 | 4 | … | |
| Sub melody | … | |||
| Accompaniment | 2 | 2 | 2 | … |
| Bass | … | |||
| Riff | 3 | 3 | … | |
| Pad | 5 | … |
Each number refers to a note sequence generated in Stage 1
Dataset
Over six months, 14 professional composers worked in two teams. Team A analyzed reference music and wrote composition guidelines, and Team B composed samples following them.
Team A
- 01Collect reference music by genre and mood
- 02Analyze the reference music
- 03Write composition guidelines with metadata
Team B
- 04Compose MIDI samples following the guidelines
ComMU dataset
11,144
MIDI samples with corresponding metadata
Track role
Each sample is labeled with its part in an arrangement: main melody, sub melody, accompaniment, bass, pad, or riff. With instrument labels alone, a model knows to use piano and guitar but not what each one plays. Adding track role lets it assign accompaniment to the piano and melody to the guitar.
Chord progression
Each sample comes with the sequence of chords it uses. Besides major, minor, diminished, augmented, and dominant, ComMU includes sus4, maj7, half-diminished, and min7. These extended chord qualities let the model learn more melodies for the same progression, improving the harmony and diversity of the music it generates.
Genre
The dataset covers two genres often used in background music. New age is melody-driven, built on keyboards and small acoustic instruments. Cinematic is orchestral, with string ensembles carrying the melody and accompaniment.
Results
Fidelity
In a survey, 40 anonymous composers compared generated and human-composed samples made from the same metadata and chose the one they preferred. Generated samples won about 42% of the time.
Human-composed
≈58%
Generated
≈42%
0255075100%Values read from the paper's chart
Controllability and diversity
Controllability is the proportion of generated notes that follow the pitch range, velocity range, and key or chord specified in the metadata. Diversity is the average pairwise distance between note sequences generated from the same metadata.
- Pitch control
- 0.84
- Velocity control
- 0.91
- Harmony control
- 0.99
- Diversity
- 0.32
Sampled with top-k 32, temperature 0.95
