Tools for People with Special Needs

1. Tools for People with Special Needs

2. We talk with tools Image communication table: Text to read and image to understand girl Future

3. Communication tools to use two-message communicator(e.g., for “telling” YES—NO) cartoons serving comminicaton Talking machine communication tool e.g., with images • They are troublesome, everybody needs something else • One needs different tools at home and at the shop

4. Modern tools for controlling: LegMouse moves the cursor • If we can move the cursors, then we can control almost everything • some people can reliably move their legs

5. We have to practice

6. Microworlds for practicing

7. Game for working memory test: “Turn it!”

8. Game for testing visual problems “Erase it!”

9. Understanding • Context is important. All possible piece of information helps • Handwaving is a good example. • Gesture language is has been developed for those, who can not understand the complex sign language • Dogs – real, animated, robotic – have many, easy to understand gestures.

10. Therapeutic possibilities • HeadMouse • webcam • face monitor and tracking • face motion is transformed into cursor motion

11. Zozó’s development • Zozó and the first results • Monitor

12. The first three trials • The first three trials showed reasonable improvement

13. The sequence of 20 trials cursor trajectories target positions on screen Not perfect, but good for the purpose

14. Bencus’ story Highly imprecise cursor control: barely controlled head swings In three months he developed his neck muscles and learnt to move the cursor smoothly… http://people.inf.elte.hu/lorincz/Files/InfoHumanica/Examples.wmv

15. BOdy MOunted UBIquitous personal computer interface BOMUBI TEstre Szerelhető MobilSzemélyes Interface in Hungarian: TESEMOOSIE

16. Components • Technology development is very fast • Miniature tools can be mounted onto and into the body • sensors • robota • drug delivery tools • are in the pipeline

17. Quality difference Traditional chips are replaced by MEMS: Micro-Electro-Mechanical Systems According to the NSF report of 2005: MEMS technológy will enter traditionally conservative markets before 2015. Such markets include chemical industry, oil industry and the medical market.

18. Yo-Joe “embedded” • Tool • Control • Interaction • Yo-Joe is watching Timea, and not the dog! • The dog has a camera in his head, • Yo-Joe looks with the dogs camera, because Yo-Joe has a see through monitor in his glasses

19. Control with your eyes! • Important corner points of the eye should be marked • Important corner points of the eye should be marked • Calibration is necessary – to screen points • Image of the camera can be seen in the upper left corner • Underneath, you see the so called average image • Underneath that, the actual image can be seen • Purple dot (not seen on this screen): actually estimated position • Blue dot: the best position from the 6 (six) possible options • This is the WinGazer, the improved/fixed OpenGazer from Cambridge University

20. Example WiFi Internet Sensor A Sensor B Sensor C Monitoring System machine – human Parent, doctor, friends

21. Tools: mouse: headmouse, eyemouse, legmouse, soundmouse self-organizing wheelchair monitoring system with cricket ultrasonic sensors laptop, computer environment, WiFi, robot dog image processing software, TTS and intelligent software laptops and webcams RF-MEMS „mote”-ok It is for real Cricket RF-MEMSultrasonic distance measuring tool

22. Continuous monitoring and Ambien intelligence • Continuous monitoring is feasible • Presence detection • User recognition by sounds, motion patterns • Pro-active prediction and • A large proportion of the events can be forecasted. Automatic mapping, modeling, and prediction of human networks is feasible • Alex (Sandy) Pentland Physica A: Statistical Mechanics and its Applications, Volume 378, Issue=20 1, 1 May 2007, edited by A. Lőrincz et al., pages 59-67

23. Software tools working in the background during the Researchers’ Night show • TinyOS operation system works on the MEMS devices • Connections • JXTA software handles the P2P network • Survivable Pipeline Protocol self-organizes data-flow over the peers • patent application: Eötvös University & Johnson and Johnson • Intelligence: • Intelligent Socket System for JAVA, C++, C#, etc. software pieces • utilizes prediction by partial matching (PPM) method • Wireless: WiFi and 433 MHz RF communication

24. Personal tools need continuous calibration – during usage Example for HeadMouse • Interfaces require special designs • Usage can be upgraded to calibration if the cursor is “trapped” time-to-time • In this interface, one can have text to be told • application to be opened • image-table • Bliss symbols on the right hand side • The intelligent algorithm recognizes the user and optimizes the actions • ELU—AAC Center—HVISION technology Others are under development

25. Thanks for your attention! People watch one of the three screens (they look forward, but in different directions) and the handicapped person controlling the show (one person looks to the right at this moment)