| 1 |
.TH MIDISTATS 1 "18 March 2024" |
| 2 |
.SH NAME |
| 3 |
\fBmidistats\fP \- program to summarize the statistical properties of a midi file |
| 4 |
.SH SYNOPSIS |
| 5 |
This is a long manual because the program extracts many different parameters |
| 6 |
from the midi file depending upon the options selected. This manual |
| 7 |
attempts to describe these parameters and how they are computed. |
| 8 |
These parameters are formatted in a way so they can be read by other |
| 9 |
applications such midiexplorer.tcl, runstats.tcl, and numerous Python |
| 10 |
scripts. |
| 11 |
|
| 12 |
Many of the options were designed specifically for analyzing the |
| 13 |
percussion track of the midi file. They are described in a separate |
| 14 |
section below (Other options). |
| 15 |
|
| 16 |
|
| 17 |
.SH DESCRIPTION |
| 18 |
If you run midistats without any options other than the name of |
| 19 |
the midi input file, it will produce a table of values described |
| 20 |
here. Each line of output starts with the name of the variable or |
| 21 |
variable array and the associated values. |
| 22 |
.PP |
| 23 |
ntrks indicates the number of tracks in the midi file. |
| 24 |
.PP |
| 25 |
ppqn the number of midi pulses per quarter note. |
| 26 |
.PP |
| 27 |
keysig the key signature, followed by a major/minor flag, the number |
| 28 |
of sharps (positive) or flats (negative) in the key, and the beat number |
| 29 |
where the key signature was found. |
| 30 |
.PP |
| 31 |
trk is followed by the track number for which the following information |
| 32 |
applies. |
| 33 |
.PP |
| 34 |
program is followed by the channel number and the General Midi Program |
| 35 |
number. |
| 36 |
.PP |
| 37 |
trkinfo is an array of 19 numbers which indicates the statistical properties |
| 38 |
of the track of interest. The following data is given: |
| 39 |
0 the channel number, |
| 40 |
1 the first program assigned to this channel, |
| 41 |
2 the number of notes for this channel counting any chords as one note, |
| 42 |
3 the total number of notes for this for this channel, |
| 43 |
4 the sum of the MIDI pitches for all the notes, |
| 44 |
5 the sum of the note durations in MIDI pulse units, |
| 45 |
6 the number of pitchbend messages, |
| 46 |
7 the number of control parameter messages, |
| 47 |
8 the number of pressure messages. |
| 48 |
9 the number of distinct rhythm patterns for each channel |
| 49 |
10 the number of pulses the channel was inactive |
| 50 |
11 the minimum pitch value |
| 51 |
12 the maximum pitch value |
| 52 |
13 the minimum note length in pulses |
| 53 |
14 the maximum note length in pulses |
| 54 |
15 the number of gaps in the channel |
| 55 |
16 the entropy of the pitch class histogram for that channel |
| 56 |
17 the number of notes whose pitch were the same as the previous note |
| 57 |
18 the number of notes whose pitch changed by less than 4 semitones |
| 58 |
19 the number of notes whose pitch changed by 4 or more semitones |
| 59 |
(In event of a chords the maximum pitches are compared.) |
| 60 |
.PP |
| 61 |
After processing all the individual tracks, the following information |
| 62 |
applies to the entire midi file. |
| 63 |
.PP |
| 64 |
npulses is the length of the longest midi track in midi pulse units |
| 65 |
.PP |
| 66 |
tempocmds specifies the number of times the tempo is changed in this |
| 67 |
file. |
| 68 |
.PP |
| 69 |
pitchbends specifies the total number of pitchbends in this file. |
| 70 |
.PP |
| 71 |
pitchbendin c n specifies the number of pitchbends n in channel c |
| 72 |
.PP |
| 73 |
progs is a list of all the midi programs addressed |
| 74 |
.PP |
| 75 |
progsact the amount of activity for each of the above midi programs. |
| 76 |
The activity is the sum of the note durations in midi pulse units. |
| 77 |
.PP |
| 78 |
progcolor: is a 17 dimensional vector where each component maps into |
| 79 |
a specific group of MIDI programs. Some of these groups are, keyboard |
| 80 |
instruments, brass instruments, wind instruments, and etc. More information |
| 81 |
can be found in the midiexplorer documentation. |
| 82 |
.PP |
| 83 |
drums is a list of all the percussion instruments (channel 9) that were |
| 84 |
used. |
| 85 |
.PP |
| 86 |
drumhits indicates the number of notes for each of the above percussion |
| 87 |
instruments. |
| 88 |
.PP |
| 89 |
pitches is a histogram for the 11 pitch classes (C, C#, D ...B) |
| 90 |
that occur in the midi file. |
| 91 |
.PP |
| 92 |
key indicates the key of the music, the number of sharps (positive) or |
| 93 |
flats (negative) in the key signature, and a measure of the confidence |
| 94 |
in this key signature. The key was estimated from the above pitch histogram |
| 95 |
by convolving with Craig Sapp's model. The peak of rmaj or rmin (below) |
| 96 |
indicates the key. A correlation less than 0.4 indicates that the pitch |
| 97 |
histogram does not follow the histogram of a major or minor scale. |
| 98 |
(It may be the result of a mixture of two key signatures.) |
| 99 |
.PP |
| 100 |
rmaj the cross correlation coefficients with Craig Sapp's major key model |
| 101 |
for each of the 11 keys (C, C#, D, ...,B). |
| 102 |
.PP |
| 103 |
rmaj the cross correlation coefficients with Craig Sapp's minor key model |
| 104 |
for each of the 11 keys (C, C#, D, ...,B). |
| 105 |
.PP |
| 106 |
pitchact is a similar histogram but is weighted by the length of |
| 107 |
the notes. |
| 108 |
.PP |
| 109 |
chanvol indicates the value of the control volume commands in the |
| 110 |
midi file for each of the 16 channels. The maximum value is 127. |
| 111 |
It scales the loudness of the notes (velocity) by its value. |
| 112 |
.PP |
| 113 |
chnact returns the amount of note activity in each channel. |
| 114 |
.PP |
| 115 |
trkact returns the number of notes in each track. |
| 116 |
.PP |
| 117 |
totalrhythmpatterns is the total number of bar rhythm patterns for |
| 118 |
all channels except the percussion channel. |
| 119 |
.PP |
| 120 |
collisions. Midistats counts the bar rhythm patterns using a hashing |
| 121 |
function. Presently collisions are ignored so occasionally two |
| 122 |
distinct rhythm patterns are counted as one. |
| 123 |
.PP |
| 124 |
Midistats prints a number of arrays which may be useful in |
| 125 |
determining where the music in the track is a melody line or |
| 126 |
chordal rhythmic support. These arrays indicate the properties |
| 127 |
for each of the 16 channels. (The percussion channel 9 contains |
| 128 |
zeros.) In the case same channel occurs in several tracks, these |
| 129 |
numbers are the totals for all track containing that channel. |
| 130 |
Here is a description of these properties. |
| 131 |
.PP |
| 132 |
programs: channel to midi program mapping |
| 133 |
.PP |
| 134 |
cnotes: the total number of notes in each channel |
| 135 |
.PP |
| 136 |
nnotes: the number of notes in each channel not including |
| 137 |
those playing in the same time interval. |
| 138 |
.br |
| 139 |
nzeros: the number of notes whose previous note was the same pitch |
| 140 |
.br |
| 141 |
nsteps: the number of notes whose pitch difference with the previous |
| 142 |
note was less than 4 semitones. |
| 143 |
.br |
| 144 |
njumps: the number of notes whose pitch difference with the previous |
| 145 |
note was 4 or more semitones. |
| 146 |
.br |
| 147 |
rpats: the number of rhythmpatterns for each channels. This is a |
| 148 |
duplication of data printed previously. |
| 149 |
.br |
| 150 |
pavg: the average pitch of all the notes for each channel. |
| 151 |
.br |
| 152 |
spread: the percentage of the track that each channel is active. |
| 153 |
.PP |
| 154 |
If some of the channels appear in more than one track, then |
| 155 |
some of the above values may be incorrect. |
| 156 |
.PP |
| 157 |
In addition the midistats may return other codes that describe |
| 158 |
other characteristics. They include |
| 159 |
|
| 160 |
unquantized - the note onsets are not quantized |
| 161 |
.br |
| 162 |
triplets - 3 notes played in the time of 2 notes are present |
| 163 |
.br |
| 164 |
qnotes - the rhythm is basically simple |
| 165 |
.br |
| 166 |
clean_quantization - the note onsets are quantized into 1/4, 1/8, 1/16 time units. |
| 167 |
.br |
| 168 |
dithered_quantization - small variations in the quantized note onsets. |
| 169 |
.br |
| 170 |
Lyrics - lyrics are present in the meta data |
| 171 |
.br |
| 172 |
programcmd - there may be multiple program changes in a midi channel |
| 173 |
|
| 174 |
|
| 175 |
|
| 176 |
.SH Other options |
| 177 |
|
| 178 |
It is recommended that you only select one of the options |
| 179 |
described here as the program was not designed to handle a |
| 180 |
multiple number of options. |
| 181 |
|
| 182 |
.PP |
| 183 |
If you run midistats with the -CSV option, it will return the |
| 184 |
results in a form of comma separated values that can be loaded |
| 185 |
into a Python panda dataframe. Each line refers to one of the |
| 186 |
16 midi channels. The following Python 3 code illustrates |
| 187 |
how you would load the midistats output into a dataframe. |
| 188 |
|
| 189 |
.br |
| 190 |
import pandas as pd |
| 191 |
.br |
| 192 |
import io |
| 193 |
.br |
| 194 |
import subprocess |
| 195 |
.br |
| 196 |
cmd = ("midistats", "-CSV" , inputmidifilepath) |
| 197 |
.br |
| 198 |
process = subprocess.Popen(cmd, stdout=subprocess.PIPE) |
| 199 |
.br |
| 200 |
csv = io.StringIO(process.stdout.read().decode()) |
| 201 |
.br |
| 202 |
df = pd.read_csv(csv) |
| 203 |
|
| 204 |
where inputmidifilepath is the path to the midi file that |
| 205 |
you are using. (eg. 'clean_midi/Zero/Chi sei.mid') |
| 206 |
|
| 207 |
.PP |
| 208 |
The MIDI file devotes channel 9 to the percussion instruments |
| 209 |
and over 60 percussion instruments are defined in the MIDI |
| 210 |
standard. Though there is a lot of diversity in the percussion |
| 211 |
track, for most MIDI files only the first 10 or so percussion |
| 212 |
instruments are important in defining the character of the track. The |
| 213 |
program Midiexplorer has various tools for exposing the percussion |
| 214 |
channel which are described in the documentation. The goal |
| 215 |
here is to find the essential characteristics of the percussion |
| 216 |
track which distinguishes the MIDI files. This is attempted |
| 217 |
in the program midistats. Here is a short description. |
| 218 |
|
| 219 |
|
| 220 |
.br |
| 221 |
|
| 222 |
A number of experimental tools for analyzing the percussion channel |
| 223 |
(track) were introduced into midistats and are accessible through |
| 224 |
the runtime arguments. When these tools are used in a script which |
| 225 |
runs through a collection of midi files, you can build a database |
| 226 |
of percussion descriptors. |
| 227 |
|
| 228 |
.SH OPTIONS |
| 229 |
.PP |
| 230 |
-corestats |
| 231 |
.br |
| 232 |
outputs a line with 5 numbers separated by tabs. eg |
| 233 |
.br |
| 234 |
1 8 384 4057 375 |
| 235 |
.br |
| 236 |
It returns the number of tracks, the number of channels, the |
| 237 |
number of divisions per quarter note beat (ppqn), |
| 238 |
the number of note onsets in the midi file, and the maximum |
| 239 |
number of quarter note beats in midi file. |
| 240 |
|
| 241 |
|
| 242 |
.PP |
| 243 |
-pulseanalysis |
| 244 |
.br |
| 245 |
counts the number of note onsets as a function of its onset time |
| 246 |
relative to a beat, grouping them into 12 intervals and returns |
| 247 |
the result as a discrete probability density function. Generally, |
| 248 |
the distribution consists of a couple of peaks corresponding |
| 249 |
to quarter notes or eigth notes. If the distribution is flat, |
| 250 |
it indicates that the times of the note occurrences have not been |
| 251 |
quantized into beats and fractions. Here is a sample output. |
| 252 |
.br |
| 253 |
0.349,0.000,0.000,0.160,0.000,0.000,0.298,0.000,0.000,0.191,0.000,0.000 |
| 254 |
|
| 255 |
.PP |
| 256 |
-panal |
| 257 |
.br |
| 258 |
Counts the number of note onsets for each percussion instrument. The first |
| 259 |
number is the code (pitch) of the instrument, the second number is the |
| 260 |
number of occurrences. eg. |
| 261 |
.br |
| 262 |
35 337 37 16 38 432 39 208 40 231 42 1088 46 384 49 42 54 1104 57 5 70 1040 85 16 |
| 263 |
|
| 264 |
.PP |
| 265 |
-ppatfor n |
| 266 |
.br |
| 267 |
where n is the code number of the percussion instrument. Each beat |
| 268 |
is represented by a 4 bit number where the position of the on-bit |
| 269 |
indicates the time in the beat when the drum onset occurs. The bits |
| 270 |
are ordered from left to right (higher order bits to lower order |
| 271 |
bits). This is the order of bits that you would expect in a |
| 272 |
time series. |
| 273 |
Thus 0 indicates that there was no note onset in that beat, 1 indicates |
| 274 |
a note onset at the end of the beat, 4 indicates a note onset |
| 275 |
in the middle of the beat, and etc. The function returns a string |
| 276 |
of numbers ranging from 0 to 7 indicating the presence of note onsets |
| 277 |
for the selected percussion instrument for the sequence of beats |
| 278 |
in the midi file. Here is a truncated sample of the output. |
| 279 |
.br |
| 280 |
11 71 c1 1 11 51 f1 c1 1 11 51 e1 1 11 d1 c1 31 1 1 1 21 b1 d1 d1 51 |
| 281 |
.br |
| 282 |
51 51 51 51 41 71 c1 11 1 11 41 11 1 11 41 11 1 11 41 11 31 51 41 11 |
| 283 |
|
| 284 |
.PP |
| 285 |
-ppat |
| 286 |
.br |
| 287 |
midistats attempts to find two percussion instruments in the midi file |
| 288 |
which come closest to acting as the bass drum and snare drum. |
| 289 |
If it is unsuccessful, it returns a message of its failue. Otherwise, |
| 290 |
encodes the position of these drum onsets in a 8 bit byte for each |
| 291 |
quarter note beat in the midi file. The lower (right) 4 bits encode the |
| 292 |
bass drum and the higher (left) 4 bits encode the snare drum in the |
| 293 |
same manner as described above for -ppatfor. The integers are printed |
| 294 |
in hexadecimal. |
| 295 |
.br |
| 296 |
02 88 20 02 a0 08 80 02 82 08 80 02 82 08 80 02 82 08 80 02 82 08 |
| 297 |
.br |
| 298 |
80 02 82 08 80 02 82 08 80 02 82 08 20 a0 0a 08 80 02 82 08 80 02 |
| 299 |
|
| 300 |
.PP |
| 301 |
-ppathist |
| 302 |
.br |
| 303 |
computes and displays the histogram of the values that would appear |
| 304 |
when running the -ppat. eg. |
| 305 |
.br |
| 306 |
bass 35 337 |
| 307 |
.br |
| 308 |
snare 38 432 |
| 309 |
.br |
| 310 |
1 (0.1) 64 32 (2.0) 8 33 (2.1) 136 144 (9.0) 8 145 (9.1) 136 |
| 311 |
.br |
| 312 |
The bass percussion code, the number of onsets, and the snare |
| 313 |
percussion code and the number of onsets are given in the |
| 314 |
first two lines. In the next line the number of occurrences of |
| 315 |
each value in the -ppat listing is given. The number in parentheses |
| 316 |
splits the two 4-bit values with a period. Thus 33 = (2*16 + 1). |
| 317 |
|
| 318 |
.PP |
| 319 |
-pitchclass |
| 320 |
.br |
| 321 |
Returns the pitch class distribution for the entire midi file. |
| 322 |
|
| 323 |
.PP |
| 324 |
-nseqfor n |
| 325 |
.br |
| 326 |
Note sequence for channel n. This option produces a string of bytes |
| 327 |
indicating the presence of a note in a time unit corresponding to |
| 328 |
an eigth note. Thus each quarter note beat is represented by two |
| 329 |
bytes. The pitch class is represented by the line number on the |
| 330 |
staff, where 0 is C. Thus the notes on a scale are represented |
| 331 |
by 7 numbers, and sharps and flats are ignored. The line number is |
| 332 |
then converted to a bit position in the byte, so that the pitch |
| 333 |
classes are represented by the numbers 1,2,4,8, and etc. A chord |
| 334 |
of consisting of two note onsets would set two of the corresponding |
| 335 |
bits. If we were to represent the full chromatic scale consisting |
| 336 |
of 12 pitches, then we would require two-byte integers or |
| 337 |
twice of much memory. |
| 338 |
.br |
| 339 |
Though the pitch resolution is not sufficient to distinguish |
| 340 |
major or minor chords, it should be sufficient to be identify some |
| 341 |
repeating patterns. |
| 342 |
.PP |
| 343 |
-nseq |
| 344 |
.br |
| 345 |
Same as above except it is applied to all channels except the |
| 346 |
percussion channel. |
| 347 |
.br |
| 348 |
.PP |
| 349 |
-nseqtokens |
| 350 |
Returns the number of distinct sequence elements for each channel. |
| 351 |
The channel number and number of distinct elements separated by |
| 352 |
a comma is returned in a tab separated list for all active channels |
| 353 |
except the percussion channel. Here is an example. |
| 354 |
.br |
| 355 |
2,3 3,4 4,11 5,6 6,3 7,3 8,6 9,3 11,2 12,1 |
| 356 |
.br |
| 357 |
|
| 358 |
-ver (version number) |
| 359 |
|
| 360 |
|
| 361 |
.SH AUTHOR |
| 362 |
Seymour Shlien <[email protected]> |
| 363 |
|
| 364 |
|
| 365 |
|