MSc Project Musical Instrument Identification System MIIS

1. MSc ProjectMusical Instrument Identification System MIIS Xiang LI ee05m216 Supervisor: Mark Plumbley

2. Musical instrument identification plays an important role in musical signal indexing and database retrieval. People can search music by the musical instruments instead of the type or the author For instance, user is able to query ‘find piano solo parts of a musical database’. Motivation of MIIS

3. Bass Drum Piano Saxophone Introduction Identification results Musical Mixtures Musical instruments

4. Structure of MIIS Estimated Sources DUET algorithm Separation Feature Extraction Classification Input Mixture X(n) Classification Results Functional Components DUET algorithm: Separate the input musical mixture into sources Feature Extraction: Extract features of each source Classification: Implement classifier on testing source and find out the class it belongs to

5. DUET algorithm • Time-Frequency representation: and are representations in time-frequency domain, i.e. Short-time Fourier Transform, Modified Cosine Discrete Transform. • Mixing parameters computation: Time-frequency points are labeled with • Mask construction: Mask equals deciding set ,which could be achieved by grouping the time-frequency point with the same label • Source estimation is the time-frequency representation of onesource. • Time-domain conversion Convert each to in time domain

6. Feature Extraction • Mel-Frequency Cepstral Coefficient (MFCC) Relationship between Mel and Hertz • Spectral Rolloff It is calculated by summing up the power spectrum samples until the desired percentage (threshold) of the total energy is reached. • Bandwidth Defined as the width of the range of frequencies that the signal occupies. • Root Mean Square RMS features are used to detect boundaries among musical instruments • Spectral Centroid Correlates strongly with the subjective qualities of “brightness” or “sharpness”. • Zero Crossing Rate A simple measure of the frequency content of a signal

7. y x Class a Class c X Class b Classification • K-Nearest Neighbor • Nonparametric classifier • Large storage required

8. Experiments • Musical Instruments Database Database :Downloaded from University of Iowa website. Mixtures are composed by isolated notes. Training set:Includes 18 classes musical instruments Testing set:Choose 3 to 5 instruments to generate mixtures The instruments to be tested: Alto Saxophone Bassoon Double Bass Flute Viola

9. For each group, five mixtures are tested and the result of each group is listed as follows: Experiments of three groups

10. Example Original Sources Estimated Sources

11. Results discussion • Without MISS, the recognisation percentage of each source in 18 classes is 1/18 which is about 5.5%. • The worst case in our experiments is group 3 where each mixture consists five sources. The percentage is 48%. • The less sources mixtures have, the higher percentage system performs. More sources introduce more interferences among each other.

12. Conclusion • MISS is a system able to identify each musical instrument in a musical mixture. • Three functional components are introduced: • DUET algorithm • Feature Extraction • Classification • Experiments of three groups, which is fifteen mixtures in total have been tested. Correct percentages are 80%,60%and 48% respectively. • More features could be extracted such as features of MPEG7 A more adaptive mask could help overcoming interferences among sources.