1 / 71

Pen-Centric Shorthand Interfaces

Pen-Centric Shorthand Interfaces. Charles C. Tappert Seidenberg School of CSIS, Pace University. Themes of Presentation. Online Handwriting Recognition and Pen Computing Tutorial Historical Research – undertaken for the Palm-Xerox Patent Infringement Lawsuit

devon
Télécharger la présentation

Pen-Centric Shorthand Interfaces

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Pen-CentricShorthand Interfaces Charles C. Tappert Seidenberg School of CSIS, Pace University

  2. Themes of Presentation • Online Handwriting Recognition and Pen Computing Tutorial • Historical Research – undertaken for the Palm-Xerox Patent Infringement Lawsuit • Recent Research - Enhanced Pen-Centric Shorthand Interfaces can have benefits • DPS dissertation could extend M.S. thesis

  3. Enhanced Pen-Centric Shorthand Interfaces • Can use word/phrase shorthand to speed text input • Can provide critical infrastructure for many pen-centric applications • Can enhance natural pen-centric interactions for many applications • Will have greatest impact on the utility of applications running on small mobile devices

  4. Part 1: Online (Pen-Centric)Handwriting Recognition • Written Languages and Handwriting Properties • The Fundamental Property of Writing • Handwriting Recognition Difficulties • Online(Pen-Centric) Handwriting Recognition • Onlinemore accurate than Offline Recognition • Online Info Can Complicate Recognition Process • Design Tradeoffs / Design Decisions • Computer Problems in English

  5. Written Language and Handwriting Properties • Alphabet • Letters, digits, punctuation, special symbols • Writing is a time sequence of strokes • Stroke – writing from pen down to pen up • Usually complete one character before beginning the next • Spatial order – e.g., in English left to right

  6. Fundamental Property of Writing • Differences between different characters are more significant than differences between different drawings of the same character • This makes handwritten communication possible • Can there be exceptions – say, different characters written identically?

  7. Fundamental Property of Writingin English • Property holds within subalphabets of uppercase, lowercase, and digits, but not across them • “I”, “l”, and “1” written with single vertical stroke • “O” and “0” written similarly with an oval

  8. Handwriting Recognition Difficulties • Shape, size, and slant variation • Similarly shaped characters – U and V • Careless writing • in the extreme, almost illegible writing • Resolving difficult ambiguities requires sophisticatedrecognition algorithms, syntax/semantics

  9. Online(Pen-Centric) Handwriting Recognition • Electronic tablets invented in late 1950s • Digitizer and display in separate surfaces • Pen Computers arrived in 1980s • Combined digitizer and display • Brought input and output into one surface • Immediate feedback via electronic ink • Created paper-like interface

  10. Tablet Digitizers – Dynamic Information • Pen down – indication of inking • X-Y coordinates as function of time • Sampling rate: 100 points per second • Resolution: 200 points per inch

  11. Early Pen-Centric Interface • Different surfaces for input and output • Rand system, about 1959

  12. Pen Computers • IBM vision • Paper-like interface, 1992 • Microsoft Tablet PC • Launched, 2001

  13. Pen-Centric PDAs • Early Palm Pilot • Palm Tungsten T3 and Sony Clié TH55

  14. Online(Pen-Centric) Handwriting Recognition • Machine recognizes the writing as the user writes • Digitizer equipment captures the dynamic information of the writing • Stroke number,order,direction,speed • A stroke is the writing from pen down to pen up

  15. Online(Pen-Centric) more accurate than Offline (Static) Recognition • Can use both dynamic and static information • Can often distinguish between similarly shaped characters • E.g., 5 versus S where the 5 is usually written with two strokes and the S with one stroke

  16. Online Information Can Complicate Recognition Process • Large number of possible variations • E can be written with one, two, three, or four strokes, and with various stroke orders and directions • A four-stroke E has 384 variations (4! stroke orders x 24 stroke directions)

  17. Online Information Can Complicate Recognition Process • Other variations

  18. Online Information Can Complicate Recognition Process • Segmentation ambiguities • character-within-character problem • lowercase d might be recognized as a cl if drawn with two strokes that are somewhat separated from one another

  19. Design Tradeoffs/Decisions • No constraints on the user • Machine recognizes user's normal writing • User severely constrained • Must write in particular style such as handprint • Must write strokes in particular order, direction, and graphical specification

  20. English Writing Styles • Handprint • Uppercase – about 2 strokes per letter • Lowercase- about 1 stroke per letter • Cursive Script • Usually less than 1 stroke per letter • Delayed crossing and dotting strokes

  21. Computer Problems in English • Constrained Handprint • Printing one symbol per box – form filling • Printing on lines – symbols can touch or overlap • Unconstrained Handprint • No lines and symbols can touch or overlap • Cursive Script • Mixed Printing and Cursive

  22. Computer Problems in English

  23. Pencept Commercial Product 1980s

  24. Part 2Shorthand in Pen-Centric PDAs • Famous Uses of Shorthand • Historical Shorthand Alphabets • Pen-Centric Shorthand Alphabets • Pen-Centric Word/Phrase Shorthand • Allegro/Chatroom Shorthand System • M.S. thesis that could be extended into a DPS dissertation

  25. Background • Famous writings throughout history were effectively written in a style of shorthand • Cicero’s orations • Martin Luther’s sermons • Shakespeare’s and George Bernard Shaw’s plays • Samuel Pepys’ diary • Sir Isaac Newton’s notebooks

  26. Historical Shorthand Alphabets • We first review the history of shorthand systems prior to pen computing • Shorthand is “a method of writing rapidly by substituting characters, abbreviations, or symbols for letters, words, or phrases” • Shorthand can be traced back to the Greeks in 400 B.C.

  27. Historical Shorthand Alphabets • We focus on shorthand alphabets that might be appropriate for PDAs • We review two types of shorthand • Geometric shorthand • Small number of basic shapes • Shapes reused in multiple orientations • Non-geometric shorthand shorthand

  28. Historical Shorthand Alphabets • Ancient Greeks – 400 BC • Tironian Alphabet – 63 BC • John Willis’s Stenography – 1602 • Gabelsberger Alphabet – 1834 • Moon Alphabet – 1845

  29. Tironian Alphabet, 63 B.C.Non-Geometric

  30. Stenography Alphabet, 1602

  31. Stenography Alphabet, 1602 • Basic Shapes and Orientations

  32. Gabelsberger Cursive-Style, 1834 Non-Geometric Alphabet

  33. Moon Geometric Alphabet, 1845

  34. Other Historical Shorthand Systems • Phonetic alphabets • Pitman (1837), was popular in UK • Gregg (1888), was popular in USA • Systems for the blind • Braille(1821)

  35. Pen-Centric Shorthand Alphabets • Some of the earliest were for CAD/CAM • symbols represent graphical items and commands • Others developed for text input on small consumer devices like PDAs that have limited computing power • We review geometric and non-geometric shorthands appropriate for small devices

  36. Pen-Centric Shorthand Alphabets • Historical alphabets presented above could be used for machine recognition • symbols drawn with a single stroke (except “K” in Tironian and “+” in Stenography) • In addition to shape and orientation, online systems can use stroke direction to differentiate among symbols

  37. Pen-Centric Shorthand Alphabets • Geometric Pen-Centric Shorthands • Organek – 1991 • Allen – filed 1991, patent 1993 • Goldberg (Xerox) – filed 1993, patent 1997 • Non-Geometric Pen-Centric Shorthands • Graffiti (Palm Computing) – 1995 • Allegro (Papyrus) – 1995

  38. Organek Alphabet, 1991

  39. Organic Alphabet, 1991 Basic Shapes and Orientations One shape in 4 orientations. This gives 8 directions that together with 3 lengths provide 24 symbols. A second wheel provides additional symbols.

  40. Allen patent, filed 1991

  41. Allen patent, filed 1991 Basic Shapes and Orientations

  42. Goldberg patent, filed 1993(“unistroke symbols”)

  43. Goldberg patent, filed 1993 Basic Shapes and Orientations

  44. Goldberg patent, filed 1993 5 Basic shapes 4 Orientations 2 Stroked Directions 40 Possible Symbols • Designed for Speed of Input and Maximum Symbol Separation

  45. Shorthand Alphabet Design • How would you design a shorthand alphabet? • What would be the design criteria?

  46. Design of Graffiti Alphabetfor the Palm Pilot • Small alphabet • Uppercase, digits, special symbols • One stroke per symbol to avoid segmentation difficulty • Separate writing areas for letters and digits to avoid same-shape confusions

  47. Graffiti Alphabet, 1995

  48. Graffiti Mimics Keyboard Input • Character by character input • Mode shifts for • Uppercase • Special characters • Eyes can focus on application’s insertion point rather than on input area

  49. Graffiti Alphabet Design • What was the additional design criterion?

  50. Graffiti Alphabet Design • Designed for ease of learning • 20 letters exactly match the Roman alphabet • 6 remaining ones match partially

More Related