1 / 22

Genealogy Name Extraction Using Layout Clues: A Computerized Approach

Explore the process of extracting names from genealogical texts using Natural Language Processing and layout clues. Discover how tools like Stanford Named Entity Recognizer and Apache UIMA Framework aid in this task, along with challenges and future work.

elmer
Télécharger la présentation

Genealogy Name Extraction Using Layout Clues: A Computerized Approach

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Extracting Names Using Layout Clues in Genealogical Books Aaron Stewart David W. Embley March 20, 2010

  2. Problem

  3. Process

  4. Finding Names • Name recognition in genealogical texts • Focus: Lists, Directories

  5. Finding Names Which side was easier? It’s easy for us to spot names… But how does a computer do it?

  6. Finding Names Natural Language Processing Stanford Named Entity Recognizer ? Apache UIMA Framework MEMM CRF

  7. BYU OntoES Ontology Extraction System • Dictionary • Regular Expressions

  8. Part 1: Preprocessing

  9. Ancestry.com Data • Word text • Word bounding boxes • Genres: • Genealogical Books • City Directories • Yearbooks • Newspapers

  10. Page Separator

  11. Line Segment Identifier

  12. RANSAC Margin Finder

  13. Margin Finder – Future Work Key Left Center Right

  14. Margin Finder – Future Work • ABBYY FineReader handles – • Paragraphs • Newspaper columns • But has trouble with – • Hanging indents • Outline indentation (possibly)

  15. Part 2: Pattern Finding

  16. Pattern Finding • Apply baseline name extractor (OntoES) • Apply margin finder and insert markers • Find left and right context for each name • Apply common contexts to extract more names

  17. Pattern Finding 1. Apply baseline name extractor (OntoES)

  18. Pattern Finding 2. Apply margin finder and insert markers LEVEL 1 LEVEL 1 LEVEL 1 LEVEL 1 LEVEL 1 LEVEL 2 LEVEL 1 LEVEL 2

  19. Pattern Finding 3. Find left and right context for each name LEVEL 1 LEVEL 1 LEVEL 1 LEVEL 1 LEVEL 1 LEVEL 2 LEVEL 1 LEVEL 2

  20. Pattern Finding 4. Apply common context patterns to extract more names LEVEL 1 LEVEL 1 LEVEL 1 LEVEL 1 LEVEL 1 LEVEL 2 LEVEL 1 LEVEL 2

  21. Pattern Finding – Sample Results Baseline Results • Precision: 40% • Recall: 31.25% • F1: 35.09% Results of Most Salient Pattern • Precision: 51.52% • Recall: 53.12% • F1: 52.31% Not all results are this good!

  22. Challenges • Evaluation • More aligned data • Annotation tool • Other books • Centered and right-aligned text • Knowing when to apply patterns

More Related