Automating Tactile Graphics Production DUB October 15, 2008

1. Automating Tactile Graphics ProductionDUBOctober 15, 2008 Richard Ladner University of Washington

2. UW Students Zach Lattin Math Major Sangyun Hahn Ph.D. Student CSE 5

3. The Problem graphics math text

4. Outline • Tactual Perception • Text • Math • Graphics

5. Tactile Perception • Resolution of human fingertip: 25 dpi • Tactual field of perception is no bigger than the size of the fingertips of two hands • Color information is replaced by texture information • Visual bandwidth is 1,000,000 bits per second, tactile is 100bits per second

6. Translation Tactile Visual

7. Braille • System to read text by feeling raised dots on paper (or on electronic displays). Invented in 1820s by Louis Braille, a French blind man. a b c z Critical fact: Fixed height and width and the with mother th ch gh Z 3 Mode characters: cap and num.

8. Importance of Braille Literacy • In the US, only 30% of blind adults are employed. • Among Braille readers: • 56% are employed • Are better educated • Read and write more and better • Tactual perception is extremely useful

9. Tiger Embosser • 20 dpi (raised dots per inch) • Prints Braille text andgraphics • Prints dot patterns fortexture

10. Outline • Tactual Perception • Text • Math • Graphics

11. Text

12. Text Translation Text Image Optical Character Recognition (OCR) The constraints do not define a region with any points in common in Quadrant I. When the constraints of a linear programming problem cannot be satisfied simultaneously, then infeasibility is said to occur. This may mean that the constraints have been formulated incorrectly, certain requirements need to be changed, or that additional resources are required before the problem can be solved. Text Speech Synthesis (Jaws) Braille Translation (Duxbury) Braille ,! 3/ra9ts d n def9e a region ) any po9ts 9 -mon 9 ,quadrant ,i4 ,:5 ! 3/ra9ts (a l9e> programm+ pro#m _c 2 satisfi$ simultane\sly1 !n 9f1sibil;y is sd 6o3ur4 ,? may m1n t ! 3/ra9ts h be5 =mulat$ 9correctly1 c]ta9 require;ts ne$ 6be *ang$1 or t a4i;nal res\rces >e requir$ 2f ! pro#m c 2 solv$4 Speech

13. Outline • Tactual Perception • Text • Math • Graphics

14. Math

15. Math Translation Math Image Math OCR (Infty Reader) \begin{eqnarray*} P(0,0) = 396(0) + 270(0) = 0\\ P(15,0) = 396(15) + 270(0) = 5940\\ P(15,5) = 396(15) + 270(5) = 7290\\ P(0,20) = 396(0) + 270(20) = 5400 \end{eqnarray*} Latex Braille Translation (Duxbury) ;,p(0,0) .k #396(0) + #270(0) .k #0 ;,p(15,0) .k #396(15) + #270(0) .k #5940 ;,p(15,5) .k #396(15) + #270(5) .k #7290 ;,p(0,20) .k #396(0) + #270(20) .k #5400 Nemeth Code

16. Math Translation Examples \sum_{i=0}^\infty x^i = \frac{1}{1-x} \frac{-b \pm \sqrt{b^2 - 4ac}}{2a} ?-b+->b^2"-4ac]/2a# .,s;i ;.k #0^,="x^i .k ?1/1-x#

17. Outline • Tactual Perception • Text • Math • Graphics

18. Graphics

19. Current Methods • Crafts • Low tech embossing • Swell paper • Capsule paper • Thermoform • Drawing Software • Copying • Tracing

20. Batch Translation • Batch Processing instead of one figure at a time. • Many figures have similar features • Image and graphics software can do batch processing • OCR and Braille translation can do batch processing • Our software – Removing text from figures.

21. Batches Complex 62 Grid overlap 113 Clean area 83 Clean lines 648 Grid clean 15 No text 41 Overlapped text 94 Radial 53

22. Graphics Translation <LocationInformation> <NumLabels>16</NumLabels> <Resolution>100.000000</Resolution> <ScaleX>1.923077</ScaleX> <ScaleY>1.953125</ScaleY> - <Label> <x1>121</x1> <y1>45</y1> <x2>140</x2> <y2>69</y2> <Alignment>0</Alignment> <Angle>3.141593</Angle> </Label> location file text extract preprocess original scanned image clean image puregraphic textimage

23. Graphics Translation <LocationInformation> <NumLabels>16</NumLabels> <Resolution>100.000000</Resolution> <ScaleX>1.923077</ScaleX> <ScaleY>1.953125</ScaleY> - <Label> <x1>121</x1> <y1>45</y1> <x2>140</x2> <y2>69</y2> <Alignment>0</Alignment> <Angle>3.141593</Angle> </Label> location file y (0,20) x=15 15 10 5 O x 5 10 15 20 20 x+y=20 (15,0) (15,5) pure graphic y (#0,#20) x.k#15 #15 #10 #5 O x #5 #10 #15 #20 #20 x+y.k#20 (#15,#0) (#15,#5) Braille text textimage

24. Available Books • Advanced Mathematical Concepts, Precalculus with ApplicationsGordon-Holliday, et al. 1999 Glencoe/McGraw-Hill • 6.3 minutes per figure • An Introduction to Modern AstrophysicsCarroll and Ostlie 1996 Addison-Wesley • 10.2 minutes per figure • Discrete Mathematical StructuresKolman, Busby and Ross 2003 Prentice Hall • 8.8 minutes per figure

25. TGA Workflow • Advantages • Much faster production • Batch processing instead of one figure at a time • Much tedious work is avoided • Disadvantages • May be of lower quality than custom translation • A lot of technology needs to be mastered

26. One-offs vs. Mass Production 1916 Woods Dual Power 1906 Reo Model T

27. Other UW Projects • WebInSight for Images • automatically determining alternative text for many web images. • Accessmonkey • bringing power to the people through scripting accessibility. • WebAnywhere • a self-voicing, web-based screen reader that can be used on any computer. • WebinSitu • studies seeking a better understanding of how blind people use the web. • NetMobileAccess • using web services to provide mobile accessibility to blind people.

28. Main Contributors • Sangyun Hahn • Dan Comden • Zach Lattin • Jeff Bigham • Anna Cavender • Chandrika Jayant • Many undergraduate students

29. Thanks To