SlideServe
Télécharger Obtenir Pro
Menu
Accueil Pro
Préférences
ApparenceLight Aide et commentaires
© SlideServe · v2026

Overview of Web Mining Techniques and Applications in the Modern Internet Landscape

tacey avatar
tacey
27 diapositives · 417 Views · Jul 13, 2014

This document explores the critical aspects of web mining, covering the taxonomy of web crawlers, including traditional, periodic, incremental, and focused crawlers. It discusses the importance of PageRank for search engines and introduces concepts related to web usage mining, including personalization techniques, sessionization, and pattern discovery. Furthermore, it addresses challenges in web mining such as user identification and data privacy. The insights into authoritative and hub pages, as well as frequent sequence mining, highlight the evolving methods for optimizing web navigability and user experience.

Overview of Web Mining Techniques and Applications in the Modern Internet Landscape - slide 1
01 / 27Overview of Web Mining Techniques and Applications in the Modern Internet Landscape
1 / 17

Télécharger la présentation

Overview of Web Mining Techniques and Applications in the Modern Internet Landscape
Une image / un lien ci-dessous est fourni (tel quel) pour télécharger la présentation

Politique de téléchargement : le contenu du site vous est fourni TEL QUEL pour votre information et votre usage personnel ; il ne peut être vendu, concédé sous licence ni partagé sur d’autres sites sans l’accord de son auteur.Le contenu vous est fourni TEL QUEL, pour votre information et votre usage personnel uniquement. Téléchargez la présentation en cliquant sur ce lien. Lors du téléchargement, si vous ne parvenez pas à télécharger une présentation, l’éditeur a peut-être supprimé le fichier de son serveur.Pendant le téléchargement, si vous ne parvenez pas à obtenir une présentation, le fichier a peut-être été supprimé par l’éditeur.

F I N

Plus de contenu associé

Transcription de la présentation

  1. Web Mining Issues • Size • >350 million pages • Grows at about 1 million pages a day • Diverse types of data

  2. Web Mining Taxonomy

  3. Crawlers • Robot (spider) traverses the hypertext sructure in the Web. • Collect information from visited pages • Used to construct indexes for search engines • Traditional Crawler – visits entire Web (?) and replaces index • Periodic Crawler – visits portions of the Web and updates subset of index • Incremental Crawler – selectively searches the Web and incrementally modifies index • Focused Crawler – visits pages related to a particular subject

  4. Focused Crawler • Classifier also determines how useful outgoing links are

  5. Focused Crawler

  6. Personalization • Web access or contents tuned to better fit the desires of each user. • Manual techniques identify user’s preferences based on profiles or demographics. • Collaborative filtering identifies preferences based on ratings from similar users. • Content based filtering retrieves pages based on similarity between pages and user profiles.

  7. PageRank • Used by Google • Prioritize pages returned from search by looking at Web structure. • Importance of page is calculated based on number of pages which point to it – Backlinks. • Weighting is used to provide more importance to backlinks coming form important pages.

  8. PageRank (cont’d) • PR(p) = c (PR(1)/N1 + … + PR(n)/Nn) • PR(i): PageRank for a page i which points to target page p. • Ni: number of links coming out of page I • Rank source E: R= cAR+cE

  9. CLEVER • Identify authoritative and hub pages. • Authoritative Pages : • Highly important pages. • Best source for requested information. • Hub Pages : • Contain links to highly important pages.

  10. Web Usage Mining Applications • Personalization • Improve structure of a site’s Web pages • Aid in caching and prediction of future page references • Improve design of individual pages • Improve effectiveness of e-commerce (sales and advertising)

  11. Web Usage Mining Activities • Preprocessing Web log • Cleanse • Remove extraneous information • Sessionize Session: Sequence of pages referenced by one user at a sitting. • Pattern Discovery • Count patterns that occur in sessions • Pattern is sequence of pages references in session. • Similar to association rules • Transaction: session • Itemset: pattern (or subset) • Order is important • Pattern Analysis

  12. Web Usage Mining Issues • Identification of exact user not possible. • Exact sequence of pages referenced by a user not possible due to caching. • Session not well defined • Security, privacy, and legal issues

  13. Web Log Cleansing • Replace source IP address with unique but non-identifying ID. • Replace exact URL of pages referenced with unique but non-identifying ID. • Delete error records and records containing not page data (such as figures and code)

  14. Sessionizing • Divide Web log into sessions. • Two common techniques: • Number of consecutive page references from a source IP address occurring within a predefined time interval (e.g. 25 minutes). • All consecutive page references from a source IP address where the interclick time is less than a predefined threshold.

  15. Episodes • Partially ordered set of pages • Serial episode – totally ordered with time constraint • Parallel episode – partial ordered with time constraint • General episode – partial ordered with no time constraint

  16. DAG for Episode

  17. Longest Common Subseries • Find longest subseries they have in common. • Ex: • X = <10,5,6,9,22,15,4,2> • Y = <6,9,10,5,6,22,15,4,2> • Output: <22,15,4,2> • Sim(X,Y) = l/n = 4/9

  18. Similarity based on Linear Transformation • Linear transformation function f • Convert a value form one series to a value in the second • ef – tolerated difference in results • d – time value difference allowed

  19. Distance between Strings • Cost to convert one to the other • Transformations • Match: Current characters in both strings are the same • Delete: Delete current character in input string • Insert: Insert current character in target string into string

  20. Distance between Strings

  21. Frequent Sequence

  22. Frequent Sequence Example • Purchases made by customers • s(<{A},{C}>) = 1/3 • s(<{A},{D}>) = 2/3 • s(<{B,C},{D}>) = 2/3

  23. Frequent Sequence Lattice

  24. SPADE • Sequential Pattern Discovery using Equivalence classes • Divides lattice into equivalent classes and searches each separately.

  25. SPADE Example • ID-List for Sequences of length 1: • Count for <{A}> is 3 • Count for <{A},{D}> is 2

  26. Q1 Equivalence Classes

  27. SPADE Algorithm