11 Applications of Machine Learning to Music Research: Empirical Investigations into the Phenomenon of Musical Expressio

1. 11 Applications of Machine Learning to Music Research: Empirical Investigations into the Phenomenon of Musical Expression 99419-811 이 인 복

2. Introduction • Expressive music performance • Why music? • A set of difficult learning task • weak(imprecise, incomplete) domain knowledge • the notion of musical knowledge

3. Expressive music performance • “Shaping” a piece of music • not exactly as given in the written score • continuously varying certain musical parameters • dynamics(variations of loudness) • rubato, expressive timing(variations of local tempo) • Input - melodies (sequences of notes) • loudness(dynamics dimension), tempo • Given new pieces, how loud? How fast?

5. The nature and importance of background knowledge • The task is to learn to ‘draw’ ‘correct’, ‘sensible’ curves above new melodies. • One symbol alone does not uniquely determine the numeric value. • It is not at all clear what the relevant context is. • Humans possess additional knowledge about the meaning of the symbols. • Expression is not arbitrary but highly correlated with the structure of music.

6. Approach I : Learning at the note level • Learning proceeds at the level of notes. • The goal is to learn rules that determine the precise degrees of loudness and tempo to be applied to each note in a piece. • Distinguish two classes of notes : rise and fall • crescendo, decrescendo • accelerando, ritardando

7. The Qualitative domain theory • Knowledge about relevant musical structure is needed. • Two major components • model of structural hearing • set of programs that perform a structural analysis of a given melody and explicitly annotate the melody with various musical structures that are perceived by human listeners. • qualitative dependency network • intuitions concerning possible relations between structural aspects of the music and appropriate expressive performance decisions

8. IBL-SMART(1/3) • Two major component • symbolic learning component • learns to distinguish between the symbolic target concepts(e.g. crescendo and decrescendo) • utilize domain knowledge in the form of a quantitative model • instance-based component • stores the instances with their precise numeric attributes • predict the target value for some new note by numeric interpolation over known instances

9. IBL-SMART(2/3) • Each rule learned by the symbolic component describes a subset of instances • These are assumed to represent a subtype of the target concept(e.g. some particular type of crescendo situations) • All the instances covered by a rule are given to the instance-based learner to be stored together in a separate instance space.

10. IBL-SMART(3/3) • Predicting the target value for some new note in a new piece involves matching the note against the symbolic rules. • Using only those numeric instance spaces(interpolation tables) for prediction whose associated rules are satisfied by note.

11. Experiment • J S Bach’s Notenbuchlein fur Anna Magdalena Bach • Played on an electronic piano and recorded through a MIDI interface. • Two part • learning with the second half • tested with the first half

12. Learning at the structure level • The note level is not really appropriate from a musical point of view. • Lacked a certain smoothness • performers tend to comprehend music in terms of higher-level abstract forms like phrase • Alternative approaches are needed.

13. Learning at the structure level • Tries to learn expression rules directly at the level of musical structures. • Transforms the training examples and the entire learning problem to a musically plausible abstraction level. • Proceeds in two stages.

14. Learning at the structure level • The system first performs a musical analysis of the given melody. • Analysis routines identify various structures in the melody that might be heard as units or ‘chunks’ by a listener or musician. • In the second step, the abstract target concepts for the learner are identified. • Tries to find prototypical shapes in the given expression curves that can be associated with these structures. • Even_level, ascending,descending, asc_desc, desc_asc

15. Learning at the structure level • The results <musical structure, expressive shape> are passed on to IBL-SMART.

16. An experiment • experiments with waltzes by Chopin • The results look and sound musically convincing.

17. A machine learning analysis of real artistic performances • Real data - performances of a complete piece by internationally famous pianists. • tested with Schumann’s “Traumerei” • by Claudio Arrau, Vladimir Ashkenazy, Alfred Brendel • showed considerable agreement in the overall • Different results • Vladimir Horowitz’s performance decisions can’t be so easily related to by obvious structural features of the music.

18. Quantitative analysis • A precise quantitative evaluation of the results is not possible. • Simply counting the number of matching decisions is far too simplistic. • Apply simple weighting scheme

19. Useful qualitative results for musicology • While abstraction to the structure level generally provides better results for various types of classical music, for other styles like jazz the note level is more adequate. • Ritardando(Note, X) :- interval_prev(Note, I), at_least(I, maj6), dir_prev(Note, up). • Increase the duration(by a certaion amount X) of all notes that terminate an upward melodic leap of at least a major sixth

20. Conclusion • Music is in many ways ‘softer’ • many aspects ar not quantifiable • difficult to perform precise experiments • Machine learning can make useful qualitative contributions • thorough analysis of the application domain