Mark Sandler Making Metadata Work, 23 June 2014

1. Mark Sandler Making Metadata Work, 23 June 2014

2. Semantic Media – Problem Area TV Productions Music / Radio Productions Consumer: How to find relevant content in large media collections? Producer: How to monetize, how to subvert piracy? Photo Productions Film Productions Source of images: Google

3. Navigation in Content Collections: Previous Approaches Automatic annotations often not as detailed and robust as needed Reason: Automatic methods have no access to knowledge only available during production, so at best does partial reverse engineering User interfaces are not as rich as needed Reason: Metadata does not incorporate relevant external information

4. Semantic Media - Concept 1: Annotation As Part of Production Workflow • Employing knowledge of the production process leads to simplified and hence more robust (automatic/assisted) metadata generation procedures • Integrating additional information usually discarded after production allows for richer annotations • Resulting novel workflow systems facilitate automation and assist content producers as well consumers throughout the content life-cycle

5. Semantic Media - Concept 1: Example Metadata: Where was this picture taken? What is in it? What’s the weather like? Source of image: Wikipedia

6. Semantic Media - Concept 1: Example Metadata: Who are the actors (in this episode)? What are the story lines? Find the scene with crying.

7. Semantic Media - Concept 2: Incorporating Global Knowledge Using Linked Data Technology Managing and exposing enhanced metadata using semantic web and linked data technology allows for uniting various sources of information and thus improving the user experience with richer interfaces

8. Semantic Media - Concept 2: Example BBC Music website + Structured Wikipedia Data = Improved User Experience More about this later…

9. Catfishsmooth: Linked Data Demo Originally by Kurt Jacobsen See also http://musicweb.academiccharts.com

10. Linked Open Data in Sept 2011

11. Goals of the Semantic Media Project • Creating a forum for researchers / developers • Encouraging interdisciplinary research bringing together specialists across the entire ICT sector • Sparking new collaborations between researchers (including industry partners) by funding mini-projects, student exchanges and internships • Encourage leading researchers to develop roadmaps guiding the direction of future research efforts and grant applications • Encourage substantial grant applications: UK & EU

12. Funding - Opportunities and Examples • Exchange of students across working groups and internships / placements • Construction of ontologies appropriate for 3D+t content description (sound, video, objects) • Capturing of motion information in a film/tv set to capture scene-descriptive metadata to associate with the primary media stream (i.e. video) • Fusion of metadata from disparate sources to build a composite metadata stream associated with a single media stream, propagating through the value chain from producer to consumer, e.g: • Metadata from several musical instruments to create a composite harmony stream • Motion metadata streams from several actors in a scene to create a composite action stream • Combining rights-related metadata (e.g. using MPEG Value Chain Ontology [9]), user generated and other tags downstream from creation • Application of temporal logic on (time-structured) media metadata streams [8] • Use of capture-at-source metadata to enhance the production workflow • Ethnographic studies of metadata-enhanced production tools to assess their fitness for purpose

13. Large-Scale capture of Producer-Defined Musical Semantics • Aims • Capture semantics behind parameters in audioproduction software • Map low-level parameters to high-level concepts(timbre, ‘bright’, ‘warm’) • Create infrastructure to semantically annotate produced music (for meta-data based retrieval andresearch purposes) • Technology: • Develop several audio plugins, which capture/output parameter settings using semantic webdata structures • Analyse audio and map parameters toperceptual entities • Project Partners: • Birmingham City University • Queen Mary Univ of London • Birmingham Conservatoire

14. SemanticNews:enriching publishing of news stories • Aims • Contextualise broadcast news and discussionaround it by identifying concepts and linkingthem to additional information available aslinked open data • Demonstrator running at the BBC using‘BBC Question Time programme’ data • Technology: • Named Entity Recognition in BBC subtitles,BBC programme data and surrounding Twitterdiscussions • Linking to external authorities (dbpedia) • Visualisation • Project Partners: • University of Southampton • University of Sheffield • BBC

15. Semantic Linking of Information, Content and Metadata for Early Music (SLICKMEM) • Aims • Link data/meta data from several informationsources about early music • Early  Music  Online (JISC project) • Electronic  Corpus  of  Lute  Music (AHRC) • External sources (e.g. dbpedia) • Create unifying ontology for all available data • Extract Music Features from scanned score datato support content-based search • Link musically similar section using similarityontology • Project Partners: • Goldsmiths  College • City University • BBC • Oxford  eResearch  Centre

16. Tawny Overtone • Aims • Overtone: a fully programmable musiccomposition and synthesis environment • Tawny-OWL: a programmable, interactive environment for the definition of Semantic Webdata schemes (ontologies) • Goal: Integrate Tawny and Overtone to generateontologies appropriate to capture the semanticsbehind an Overtone ‘music programme’ • Project Partners: • University of Newcastle • University of Manchester

17. Second Screen - a fingerprinting driven semantic music recommendation service • Aims • Use finger-printing technology to identify a musicrecording off-the-air using a smartphone/tablet • Use ID to retrieve wide range of artist metadata from multiple internet data sources • Provide an interface to discover more information about the song/artist/related artist/genres • Project Partners: • Queen Mary Univ. of London • MPEG

18. Ongoing Projects Computational Analysis of the Live Music Archive (CALMA) MUSIC - Metadata Used in Semantic Indexes and Charts Content-based analysis (tempo, key, etc) of freely available music content and publication of results as linked data. Merging the Academic Charts Online music meta-data service with linked open data services. • Project Partners: • University of Manchester • Queen Mary Univ. of London • Project Partners: • University of Northampton • Queen Mary Univ. of London • Oxford e-Research Centre • The Internet Archive • Academic Rights Press WhatTheySaid Automatic generation of timelines from speech data, which summarize main concepts andstatements made • Project Partners: • University of Southampton • University College London • BBC

19. Upcoming Projects Semantic Linking of BBC Radio (SLoBR) - Programme Data and Early Music POWkist– Visualising Cultural Heritage Linked Datasets Building a live-demonstrator at the BBC that enriches/contextualizes BBC Radio 3 programme data with EMO/ECOLMinformation Enriched visualization of digitized cultural heritage data (prisoner of war diaries) by integrating linked open data. • Project Partners: • Oxford e-Research Centre • BBC • Project Partners: • University of Aberdeen • Northumbria University • Dot.rural Digital Economy Hub • Goldsmiths College • City University An Argument Workbench - extracting structured arguments from social media Extraction and semantic representation of discussion threads and arguments from comments to articles and news. • Project Partners: • University  of  Aberdeen • University  of  Sheffield • DebateGraph

20. Getting involved • Join our mailing list for announcements and discussions • Have an idea for a feasibility study and put it on our idea-wiki • Helporganizing meetings (maybe focused on a specific subfield) • Help documenting the research landscape by participating in the landscape-wiki • Participate in future meetings, sandpits, tutorials, as well as collaborative grants and paper submissions • Help identifying people who might be interested in this network and invite them (or tell us) • Check our website: semanticmedia.org.ukand contact s.ewert@qmul.ac.uk (sebastianewert)