Download
1 / 25
250 likes | 377 Vues
Summarization of XML Documents. K Sarath Kumar. Outline. Motivation System for XML Summarization Ranking Model and Summary Generation Example Summaries Conclusion and Future Work. Motivation. XML Document Collection (eg: IMDB). XML Document.
E N D
Summarization of XML Documents K Sarath Kumar
Outline Motivation System for XML Summarization Ranking Model and Summary Generation Example Summaries Conclusion and Future Work
Motivation XML Document Collection (eg: IMDB) XML Document • Types of XML Document Summaries • Generic summary – summarizes entire contents of the document. • Query-biased summary – summarizes those parts of the document which are relevant to user’s query.
Aims • We aim at summaries which are : • Generated Automatically • Highly constrained by size • Highly Informative • High Coverage • Challenges • Structure is as important as text • Varying text length
System for XML Summarization Summary Size Corpus Statistics SUMMARY GENERATOR RANKING UNIT Ranked Tag units XML Doc Info Unit Generator Tag Ranker Tag Units Summary Text Ranker Text Units Ranked Text units
Information Units of an XML Document • Tag • Regarded as metadata • Can be highly redundant • Can be encoded into Schema DTD • Text • Instance for the tag • Much less redundant • Have different sizes
Ranking Unit I. Tag Ranking Typicality : How salient is the tag in the corpus?E.g.:<title> • Typical tags define the context of the document • Occur regularly in most or all of the documents • Quantified by fraction of documents in which the tag occurs (df) Specialty : Does the tag occur more/less frequent in this document? • Special tags denote a special aspect of the current document • Occurs too many or too few times in the current document than usual • Quantified by deviation from average number of occurrences per document
II. Text Ranking • Two categories of text • Entities • Regular text
Ranking is done based on context of occurrence. • - No redundancy in tag context (E.g.: actor names, genre) • Redundancy in tag context (E.g.: plots, goofs, trivia items) Tag context Document context Corpus context
Let and Correlated tags and text Often find related tag units – siblings of each other E.g.: Actor and Role Inclusion Principle Case 1 : Case 2 :
Generation of Summary Consider the following tag rank table : To generate a summary with 30 tags, 15 actor tags, 9 keyword tags and 6 trivia would be required. Distribute the remaining “tag-budget” by re-normalizing the distribution of available tags
Few Example Summaries Titanic.xml - Summaries
Conclusion • A fully automated XML summary generator • Ranking of tags and text based on the ranking model • Generation of summary from ranked tags & text within memory budget • User Evaluation is underway Future Work • Rewriting the structure of the xml documents during summarization • Possible usage of text summarizers for long text • Query-biased xml summary generation
Appendix Informativeness
RankingModel I. TAG RANKER Mixture Model of Typicality and Specialty • Typicality:How typical is the tag in the corpus?
Specialty : How unusually frequent/infrequent is the tag in the current document compared to an average document of the corpus?
Text with redundancy in tag context Sort terms by frequencies and take top ‘m’ terms as centroid query Relevance : Similarity : Calculated using Maximum marginal relevance(MMR) Finally,
Text without redundancy in tag context Redundancy at tag level : No redundancy at tag level : is set empirically
A Relative Count Matrix is constructed • Given two tags Ti and Tj, the relative importance of Tj with that of higher ranked Tj is calculated by dividing them both by P(Tj|D) (shows how many Tj tags are worth one Ti) • Tj is considered only after P(Ti|D)/P(Tj|D) number of Ti tags have been considered. • Extending the above concept, a matrix with relative counts can be formed.
