Pedro J. Ponce de León

1. Hybrid-cooperative systems for automatic composition and musical stye perception modeling Pedro J. Ponce de León Teaching assistant at Departamento de Lenguajes y Sistemas Informáticos Universidad de Alicante Promotor: Prof. José Manuel Iñesta Computer Music Laboratory Grupo de Reconocimiento de Formas e Inteligencia Artificial http://grfia.dlsi.ua.es

2. Outline • Background • First part of research project • Musical Style characterization by statistical descriptors • Second part of research project • Genetic Programming • Application to assisted composition

3. My Background • Computer Science Engineering • Msc Thesis: • Musical style identification using self-organising maps (Dec. 2001) • Music (30% academic, 70% self-teaching) • Jazz performance (sax) • Last three months: • Stay at IRCAM within the Music Representations Team (supervisor: Carlos Agon) • Start of second half of my Phd thesis research about two month ago

4. Part I: Music Style Analysis from Symbolic Representations • Universe • Sources Monophonic melodies of two different styles (MIDI file tracks): • Time signature: 4/4 • Resolution: 48 pulses/bar • Jazz standards (65 tunes) • Classical Music (43 pieces) (about 10000 bars) • wd-space • Melody fragments are extracted using a sliding window across a melody. Each fragment becomes a sample. • Parameters: w width of window in bars d shift towards end of melody for next window • Each <w,d> pair yields a different dataset. • Some results at ICMC2004

5. Part I: Music Style Analysis from Symbolic Representations • Statistical description of monophonic melodies from symbolic data • Descriptors • Absolute description model (33 descriptors) • Overall features (3): number of notes, score silences, interpretation silences. • Pitch features (4): • Note duration (4): • Silence duration (4): • Intervals (4): • IOI (4): • Non diatonic notes (3): num., avg. and std. dev. • Syncopation (1): estimated number of syncopes • Normality descriptors (6): D’Agostino statistic for pitches, durations, etc. min., max., avg., std. dev. • Relative description model (28 descriptors) • min. and max. descriptors replaced by range descriptor (max-min) • Avg. descriptors relative to min. (avg-min for pitches, avg/min for durations and IOIs) • Better results than absolute model

6. Part I: Music Style Analysis from Symbolic Representations • Feature space transformation: PCA • 75% of total variance in three first PC for every  tested (97% with ten first PC) • Most contributing descriptors in components (first PC) •  = 1,2,4 note duration •  = 4,8 counters for notes and silences •  = 16,32,64,  silence duration, IOI • No pitch descriptors among three first PC!!

7. Part I: Music Style Analysis from Symbolic Representations Classification techniques Supervised learning Non parametric k-Nearest Neighbours (k-NN) Parametric Bayesian classifier Unsupervised learning Connectionist Self-organising map (SOM) Best results: On average (all <w,d> pairs) Bayes, model 12

8. Part I: Music Style Analysis from Symbolic Representations • Further research topics • Other classifiers • Feed-forward neural networks • Support vector machines • Voting systems (multiclassifiers) • Several classifiers on one fragment • All fragments from the same song • Multiple <w,d> on the same song • Other corpus • XML sources • Suggestions are welcome...

9. PART II: Style oriented composition/generation systems • Investigate the use of Genetic Programming techniques for melody generation. • Genetic Programming (GP) • Genetic Algorithm (GA) where individuals are not data but algorithms generating data. • OpenMusic (OM) • Visual Object Oriented programming environment (IRCAM) • Extended CLOS language (Common Lisp Object System) • Libraries of musical objects and functions • Very recent applications • 1st Workshop: EVOMUSART 2003 • Next: 3rd Workshop: EVOMUSART 2005

10. Genetic Programming (GP) • Primary source • Genetic Programming. On the programming of computers by means of natural selection. John R. Koza, 1992. • Structures undergoing adaptation • Hierarchically structured computer programs (trees) • Can be represented as LISP expressions • GP search space • Combinations of ‘primitive’ functions in a function set F • F = {f1, f2, ..., fNf} • And terminals in • T = {a1, a2, ..., aNt}

11. Genetic Programming (GP) • Initial structures generated randomly • Measure of program fitness • Dependent on the problem • Usually in the form of fitness cases • Representative situations of whole domain numerous enough to give a range of fitness values. • The most common fitness measures: • Error or Root Mean Square Error • (Sum of distances, for every fitness case, between the result obtained and the correct solution)

12. Genetic Programming (GP) • GP primary operators • Darwinian reproduction (survival of the fittest) • Crossover (sexual recombination) • Parents: (A (B 1) (C 2 3)) -- (A (B 4 (D 5)) (F (G 6) (I 7)) ) • Offsprings: • (A (F (G 6) (I 7))(C 2 3)) • (A (B 4 (D 5)) (B 1) ) • GP termination criterion • G : Maximum number of generations to reach (generational predicate) • Problem especific success predicate satisfied • GP result designation • Best-so-far individual of a complete run • Best-so-far subset of population • Entire last population

13. OMGP: GP for generating style oriented musical material • GP usually finds sub-optimal or approximate solutions • Application to assisted composition. Key idea: • We have a corpora of musical entities and a set of transformation functions to apply to these objects • We have a way of defining a fitness function for what we want to obtain. • We want to generate new musical material derived from the corpora that fits our needs. • The sub-optimal/approximative nature of evolved solutions in GP seems adecuate to this context.

14. OMGP Function and Terminal sets • Monophonic or Polyphonic? • Depends on the Function and Terminal sets • Terminal set • OM Musical Objects extracted from a given corpora [+ Random constants]: From single notes to whole scores. • Function set • OM transformation functions: • Input: One or more Musical Objects • Output: One Musical Object • Examples: • Transposition • Retrograde • Interpolation • Concatenation • Random numbers generators (non-determinism)

15. OMGP fitness function (fitness cases) • First approach: • Fitness cases = prototypes of what the user wants • Deterministic programs: • Evaluate once against each fitness case • Non-deterministic programs: • Evaluate K times against each fitness case • Optional: Penalize determinism (non-deterministic programs would in general obtain average scores)

16. OMGP fitness function (fitness cases) • Issues • Non-deterministic program evaluation computationally would make GP even more computationally expensive. • Fitness measure of program p: • fit(p) = w(x) * d(output(p), x) • d(·,·) : Edit distance to prototypes • w(·) : Weight function for prototypes • What if we want to generate lengthy segments from short prototypes? • Lengthy individuals would have bad fitness • Need some global fitnessmeasure

17. OMGP global fitness function • Second (global fitness) approach • Use (multi)classifiers from the first part of my research to evaluate program candidates • Classifiers are trained on a given corpora (we don’t use prototypes here) • The statistical descriptors provide a way of obtaining global characteristics of musical objects (monophonic so far) • Additional constraints for the fitness function • Would need a way to specify constraints about • Length • Range • Key • Meter • Resolution • Use of microtones • Etc...

18. OMGP example

19. THANKS! Suggestions/criticism welcome • Does this approach make sense to you? • It is not late to change my mind!!