1. Multimedia Database Management System Wei Tsang Ooi CS731

2. MMDBMS : Querying Interface, Indexing and Buffer Management

3. Why MMDBMS ? • Concurrency Control • Recovery • Privacy • Query Support • Version Control

4. Example of MMDBMS • Digital Library • News-On-Demand • Video-On-Demand • Music Database • Telemedicine • Geographic Information System

5. 000003 000002 000004 000001 600M 530M 450M 510M 30 30 30 30 parag tibor soam wei l5.mpg l4.mpg l6.mpg l7.mpg No Intergration id size fps title filename

6. Semi-intergrated BLOB 000001b70ae9902...

7. Fully Intergrated    index     buffers  storage

8. Nature of Multimedia Data • Large amount of data • Time sensitive • Vague matching

9. Database Components  Query Interface Query Processing storage manager buffer manager index

10. Query Interface andProcessing

11. Problems • Needs support for : • temporal and spatial relations • “natural” interface • fuzzy query • SQL is not suitable.

12. 13 Temporal Relations • Allen 83 starts before equals meets during overlaps finished by finishes overlapped by contains met by started by after

13. Spatio Relations • Papadias, Theodoridis 96 • Topological Relations • Directional Relations

14. disjoint meet overlap covered by inside equal covers contains Topological Relations

15. Directional Relations North NW NE West East SW SE South

16. Spatio-temporal Relations • overlap-after • meet-during • ...

17. Querying • Image • Audio • Music • Sound • Speech • Video

18. Querying Image • Common approach • allow query by sketches (color, shape, texture) or examples. • perform matching by Feature Vectors F = (v1, v2, ... vn) • e.g. Color Histogram

19. Querying Image • Exisiting Systems : • QBIC • VisualSEEK • PhotoBook • Virage • FourEyes

20. Querying Music • Hawley 93 • Input by MIDI Keyboard • Measure relative pitch (U, D, S) • Perform exact match with existing database.

21. Querying Music • Ghias, Logan, Chamberlin & Smith 95 • Input by humming • Extract relative pitch • Perform approximate matching

22. Querying Music • Chou, Chen, & Liu 96 • Query by chord • Represents musics by chord • C Am Em F C Am Em F ... • Perform fuzzy match

23. Querying Music • Chen & Chen 98 • Query by “rhythm” (tempo ?) • Represents musics by rhythm • | ¶¶— | ¶·¶·| ¶¶¶¶ | ... • Perform fuzzy match

24. Querying Sound • Wold, Blum, Keisar & Wheaton 96 • Analyze audio to extract features • loudness, pitch, brightness, bandwidth and harmonicity • Segment the audio to pieces • Feature Vector Matchings

25. Querying Speech • Hauptmann & Witbrock 97 • Informedia • Use speech recorgnition to convert audio to text • segment audio using silence detection • Query by speaking keywords

26. Querying Video • Query by speech • Informedia • Visual approach • VideoQ

27. VideoQ • Chang et. al 97 • User can sketch objects • Specifies • color • texture • shape • motion • duration • camera zoom and pan.

28. VisualQ Example Titanic Sinking Someone Skiing

29. Indexing

30. Indexing • Requirements • support spatio-temporal operations • support fuzzy matches

31. Indexing Images • N-dimentional indices for feature vector • Well studied in DB/CG community • Two examples : • VP-tree • R-tree

32. VP-tree R Q P PQS VWTU R T S U W V

33. VP-tree R Q P U R P T S S Q T VW U W V

34. R-tree

35. Indexing Audio • Audio are modeled as strings • Inexact match is needed • Common indices for string search can be used • Example • PAT-tree

36. PAT-tree ab b c abc c abc c ababc abc babc bc c

37. Indexing Video • Treat time as third dimension • We can use any multidimension indexing structures

38. Buffer Management

39. Buffer Management • Minimizing response time • Ensuring continuity and synchronization • Prefetching & Replacement

40. Glossaries • Presentation • Media Stream • Media Object

41. Buffer Management • Relevance Based • Distance Based • Buffer Consumption

42. Relevance Based • Moser, Kraiß & Klas 95 : L/MRP • Definition: • State = (curr obj, skip) • Relevance : State  Real

43. Relevance Based • Prefetch : • future objects with highest relevance • Replace : • old objects with lowest relevance

44. Example (5,+2) Relevance History Referenced Skip Time Scale

45. Distance Based • Özden, Rastogi & Silberschatz 96 • Definition • distance of a client C is the offset between C and the immediate following client.

46. Distance Based • Replace : • blocks consumed by clients with largest distance

47. Example C1 C2 C3 C4

48. Example C1 C2 C3 C4

49. Example C1 C2 C3 C4

50. Buffer Consumption • Wu & Yu 97 • Definition : • Buffer consumption = amount of buffer used x total time