Xin Lu 1 , Yanwei Pang 1 *, Qiang Hao 1 , Lei Zhang 2 1 Tianjin University * Corresponding Author 2 Microsoft R

1. Visualize Textual Travelogue with Location-Relevant Images Xin Lu 1, Yanwei Pang 1*, QiangHao1, Lei Zhang 2 1 Tianjin University * Corresponding Author 2 Microsoft Research Asia November 3, 2009

2. Outline • Motivation & Challenge • Our Solution • Framework Overview • Data Source • Demo • Conclusions and Future Work

3. Outline • Motivation & Challenge • Our Solution • Framework Overview • Data Source • Demo • Conclusions and Future Work

4. What is a travelogue • What is a travelogue • Text/article that records one's travel experience • Where can we/you find travelogues • Blog, forum, Web2.0 community, etc. • What's the travelogue's difference from other text • User-generated content (UGC), rather than expert's articles large amount and booming informative to other tourists

5. Why We Visualize the Travelogue • Travelogues are huge knowledge resources • People share others’ experience by reading travelogues online • Textual Travelogues • Long and noisy • Probably be written in foreign languages • Travelogue Visualizing • Highlight the useful information • Visualize the useful information

6. Why Travelogue Visualization is difficult • Travelogue De-noising • Location-oriented • Context words • Image Retrieval and Ranking • Semantic Gap between texts and images

7. Outline • Motivation & Challenge • Our Solution • Framework Overview • Data Source • Demo • Conclusions and Future Work

8. log-based model • log- tag model • tag- based model

9. Similarity Measures • Log-Model & Log-tag Model • Log refers to travelogue • Context words Great Wall: ancient times; stable; impregnable pass; No.1 in the world Sanya Bay: sea sight; beach; sea food

10. Similarity Measures • Tag-Model • Tags also are UGC • De-noise tags Topic Space

11. Data Source • 100K travelogues (automatically) • All written in Chinese • Downloaded from Ctrip • GPS data and English Name for the most popular 10K locations • 2500K images (automatically) • Flickr 950K (plenty of tags) • Picasa 300K (little tags) • *Google 1200K (includes snippets of the image)

12. We add 1200K images (includes snippets of the image) from Google • Flickr images are retrieved based on location • Google images are retrieved by “context words+ location”, which makes candidate images sets more relevant to the travelogue

13. http://202.113.2.198 • Images are ranked based on the following three points: • image quality • log-tag similarity • Image diversity

14. Demo

15. Demo

16. Outline • Motivation & Challenge • Our Solution • Framework Overview • Data Source • Demo • Conclusions and Future Work

17. Conclusions and Future Work • Travelogue visualization benefit common people • Travelogue is more easily to understand • People all over the world benefit from others’ experience to plan trips • Future work • Further narrow the semantic gap using visual features • Improve evaluation approaches

18. LBSN’ 2009 Nov.3, 2009, Seattle, WA, USA Thank you!