From Web to Map: Exploring the World of Music

1. From Web to Map: Exploring the World of Music Olga Goussevskaia Michael Kuhn Michael Lorenzi Roger Wattenhofer Web Intelligence 2008 Sydney, Australia

2. Music in the old days • Storage media • Vinyl records • Compact cassetts • Compact discs • An Album is stored on a single physical storage medium • Sequence of songs given by album • Album is typically listened to as a whole organization by album Olga Goussevskaia, ETH Zurich @ Web Intelligence 2008

3. Music today • Huge offer, easily available • filesharing, iTunes, Amazon, etc. • Large collections • The entire collection is stored on a single electronic storage medium • Organization by albums (and other lists) is no longer appropriate organize by similarity! Olga Goussevskaia, ETH Zurich @ Web Intelligence 2008

4. Overview • Define music similarity • From Perception to Web • Build a graph of songs • From Web to Map • Embed the graph into Euclidean space • Application prototype: www.musicexplorer.org Olga Goussevskaia, ETH Zurich @ Web Intelligence 2008

5. Music Similarity Similar or different??? • Audio content analysis • Metadata analysis • Collaborative filtering • “people who listen to this song also listen to that song” Olga Goussevskaia, ETH Zurich @ Web Intelligence 2008

6. From Perception to Web • Data from last.fm (20M users) • Top-50 lists (290K lists, 1.5M distinct songs) • Co-occurrence analysis (normalization cosine(si,sj)=nij/(ninj)1/2) • 1012 (O(TB)!) pair-wise similarity values • Building a graph G • Edge weight w(si,sj) = 1/cosine(si,sj) • Sparsening: co-occ ≥ 2, w(si,sj) ≥ threshold • sim(si, sj) = length(shortestPathG(si, sj)) • Still n = 430K, m = 6.3M, and ever growing • How to operate on G? (assuming G is sparse: m=O(n logn)) • Shortest path computation cost: O(m+logn)=O(n logn) • Memory needed to retrieve one value sim(si, sj): O(m)=O(n logn) Order of seconds on a state-of-the-art PC! Need to store the whole G, even if I only have 50 songs in my collection! Olga Goussevskaia, ETH Zurich @ Web Intelligence 2008

7. From Web to Map • Embedding: map vertices of G into points in Euclidean space, s.t. dG/dE (stretch) is “minimized”. • Computation cost of sim(i,j): O(1) time, O(1) memory per item • Embedding algorithms: • Multi Dimensional Scaling (MDS): O(dn2) • Spring embedding (Fruchterman-Reingold): O(n2 + m) • MIS-filtering: O(n log2 Δ) • High-dimensional embedding: O(nl2 + lm) • Landmark MDS (LMDS): O(nld + l3) • Adaptive computation/quality tradeoff • Suitable for dynamic settings Olga Goussevskaia, ETH Zurich @ Web Intelligence 2008

8. Iterative Embedding • Assumption: some links erroneously shortcut certain paths E [# random edges] = X • Repeat (X / f) times • embed G (using e.g. LMDS) • Remove (from G) fraction f of edges with highest stretch dE/dG • Example: Kleinberg graph (20x20 grid, f = 0.003) Spring embedding output After 12 rounds After 30 rounds After 6 rounds Olga Goussevskaia, ETH Zurich @ Web Intelligence 2008

9. Evaluation • Music Taxonomy (www.allmusic.com) • Control set: 7K songs with genre information How well does the resulting map represent music similarity? Genre distance dS= LCA (least common ancestor) Olga Goussevskaia, ETH Zurich @ Web Intelligence 2008

10. Evaluation: Quality Measures • Distance comparison QL: average similarity increase as a function of genre distance ds • Embedding smoothness QR: average # of genre re-occurrences on a random line Avg. similarity of pairs (si,sj) w/ ds(i,j)=h Songs that belong to distant genres should be far away in the embedding. Genre transitions in the embedding should be “smooth”. Olga Goussevskaia, ETH Zurich @ Web Intelligence 2008

11. Evaluation: Iterative Embedding (430K nodes, 10 dimensions) After 30 rounds, f=0.5% LMDS output Olga Goussevskaia, ETH Zurich @ Web Intelligence 2008

12. Evaluation Closest neighbors in 10D Olga Goussevskaia, ETH Zurich @ Web Intelligence 2008

13. Applications: Music Explorer • www.musicexplorer.org • Web service to query coordinates (current DB with 430K titles) • Visualization in 2D • Zoom level according to song popularity • Playlist generation based on trajectories Olga Goussevskaia, ETH Zurich @ Web Intelligence 2008

14. Playlist generation • Interpolation between start and end-point • Smooth transition from one style to the other • In reality: 10 dimensions Olga Goussevskaia, ETH Zurich @ Web Intelligence 2008

15. Music in Euclidean Space • Performance • Similarity computation comes almost for free: O(1) time • Memory footprint is extremly low: O(1) per song • All information can be saved in the file, no server connection required. • Applications • Trajectories (playlists, ...) • Volumes (region of interest, ...) • Notion of direction coordinates are well suited for mobile applications coordinates are well suited for similarity based organization Olga Goussevskaia, ETH Zurich @ Web Intelligence 2008

16. Towards a new world of music? • Euclidean representation • Efficient similarity computation (time and memory) • No server needed: distributed applications • Building blocks for new functionalities: • New scenarios: • Mobile file sharing • P2P overlay based on the map • Innovations at home • “Play anything hip-hip… not this and not closely related songs… go towards Detroit house, be there in an hour” • Automatic DJ (collect feedback from mobiles, generate playlists based on guests regions of interest) Notion of Direction (Browsing) Volumes (Interest Regions) Trajectories (Playlists) Olga Goussevskaia, ETH Zurich @ Web Intelligence 2008

17. Conclusions • Necessary? Olga Goussevskaia, ETH Zurich @ Web Intelligence 2008

18. Thanks for your Attention • Questions? Olga Goussevskaia, ETH Zurich @ Web Intelligence 2008