650 likes | 815 Vues
H A N D T A L K. An Interactive Web-Based Sign Language Modeler. The Proponents. Natalie Rubnan Que Maria Melissa Mejia Marjorie Joy Pantig Jackson Velasco. Overview of Research. HandTalk – translation of sign language coupled with an instant messenger
E N D
H A N D T A L K An Interactive Web-Based Sign Language Modeler
The Proponents . . . • Natalie Rubnan Que • Maria Melissa Mejia • Marjorie Joy Pantig • Jackson Velasco
Overview of Research • HandTalk – translation of sign language coupled with an instant messenger • Motion Capture – realistic hand movement • ASL (American Sign Language) – widely used language • Limited to fingerspelling and goes back to open palm position
Overview of Research • Research . . . • 3D Graphics on the Web • Animation Techniques • Web Application Implementation
Overview of Research • Research Objectives Met • Study basic sign language • Review sign language related papers , software, theses • Study formulas, algorithms about hand modeling • Study joints of the hand • Study use of motion capture data • Investigate animation techniques • Study web application issues • Apply HCI concepts in system design
Overview of Research • System Objectives Met • Display a 3D hand model on a web page in WWW. • Provide function to choose which hand • Provide functions for modeling transformations on the model • Provide a function that would accept user text input • Provide a means for the user to choose whom the receiver of the text input is • Provide a function that would parse input and divide into token stream • Provide a function that would accept token stream and retrieve motion data
Overview of Research • System Objectives Met • Provide a function that would enable 3D hand model to perform fingerspelling motion of user input • Provide an option to set speed of translation • Provide a means for the user to identify himself in system • Provide a means for the user to keep track of his/her contacts list • Provide a means for the user to keep track of the messages he/she received
Overview of Research • General Objective: “To research on the design and implementation of an interactive web-based sign language modeler.”
Overview of Proposed System • Three Major Modules: • Hand Modeler • Sign Modeler • Sign Language Messenger
Overview of Proposed System • Overall Structure of the System
Maybe we could make a Flash movie here that shows how system works during operation
Proposed System…Hand Modeler • Hand Modeler – hand model generation • Problem: Which 3D graphics tool? • Choices: • VRML • Microsoft .NET Technology • Macromedia Director Shockwave
Proposed System…Hand Modeler • Chosen Solution . . . • Macromedia Director Shockwave • Wide range of multimedia applications • VRML slower and limited • Revolution 3D, not for the Web
Proposed System…Hand Modeler • Shockwave Control • Sprites • Scripts • State flags
Proposed System…Hand Modeler • Major Functions: • Select Hand Model • Rotation • Camera View • Adjust Lighting • Reset All
Hand Modeler Functions • Select Hand Model • Right, Left or Both Hands • Right Hand values negated for Left Hand • Hierarchy of Joints
Hand Modeler Functions • Select Hand Model • Problem # 1: Speed Issues • Solution: Trade between realism and speed
Hand Modeler Functions • Select Hand Model • Problem # 2: Default Orientation of Hand • Solution: Adjustment of Camera
Hand Modeler Functions • Rotation • Rotate about X, Y, Z axis • Root Joint Only • 10 degrees • Rotation of Model Itself, not Camera
Hand Modeler Functions • Rotation • Problem # 1: Inconsistent hand orientations due to a combination of rotations • Solution: A list instead of state flags
Hand Modeler Functions • Camera View • Translation and Scaling • State flags • Camera instead of model
Hand Modeler Functions • Adjust Lighting • Brightness and Contrast • Contrast disabled
Hand Modeler Functions • Adjust Lighting • Problem: Default Director light • Solution: Lighting from scripts
Hand Modeler Functions • Adjust Delay • Delay between frames • State flag
Hand Modeler Animation • Frame-by-frame • Data for specific frame applied • Adjust from user controls
Proposed System…Sign Language Modeler • Sign Language Modeler – retrieve data and pass to Hand Modeler • Three languages: • Java • JavaScript • Shockwave Lingo • Data on client side – minimum network delay
Proposed System…Sign Language Modeler • Sign Language Modeler • Problem: Slow retrieval of data • Solution: Streaming of data
Sign Language Modeler Functions • Process Text • Read user input • JavaScript for speed and Evalscript • Validation in JavaScript, not Java
Sign Language Modeler Functions • Process Motion Data • Token = parsing of motion data in BVH file • Digitally signed Java applet • Dependent on structure of BVH file • String tokenizer: only offset values (OFFSET part), number of frames, motion data (FRAMES part) • Comma delimiter (,)
Sign Language Modeler Functions • Process Motion Data • variable data: faster tokenizing, shorter strings to append • Data sent back as string of form: <character translated> <offsets> <number of frames> <motion data>
Sign Language Modeler Functions • Process Motion Data • Problem # 1: Different natures of security certificates in Netscape and IE • Solution: Two implementations • Problem # 1: No permission • Solution: Netscape asks again, IE need to restart
Sign Language Modeler Functions • Map Joint Angle Data • Shockwave Lingo • Offset and motion data to joints • Data from Process Motion Data (Java) through JavaScript function Evalscript • Maps specific data in every time frame
Sign Language Modeler Functions • Map Joint Angle Data • variable MocapData: data from Evalscript • Values retrieved from MocapData based on form of string • variable PassData: currentmotiondata to be mapped
Sign Language Modeler Functions • Map Joint Angle Data • Problem: Shared data conflict due to concurrency • Choices: • Binary Semaphores • Semaphores • Wait-Signal Mechanisms • Solution: Binary Semaphores – simple, yet adequate and effective
Proposed System…Sign Language Messenger • Instant messaging • Separation of major functions instead of centralization – no bottlenecks
Sign Language Messenger…The Server Program • All the users, all the messages • instant_messenging class: client >> web server >> server program
Sign Language Messenger…The Server Program • Keeping Track of Users… • client_thread group: dynamic storage, no waste of time in file access • Different functions for interactivity (threads) client_threadgroup.write_message_to(x, sender + "¿" + recipient + "¿" + msg);
Sign Language Messenger…The Server Program • Keeping Track of Users… • Problem: Information transfer between threads • Choices: • Arrays • Vector Lists • Solution: Vectors: for distributed computing
Sign Language Messenger…The Server Program • Enabling User Connection • server_temp_thread: thread-embedded socket to a client • Server alert from client • Necessary functions first int x = client_threadgroup.search_for_username(s); CLIENT: socket = new Socket("server ip", port); SERVER: server_temp_thread t = new server_temp_thread(new_socket); t.start();
Sign Language Messenger…The Server Program • Enabling User Connection • Problem: Checking if logged before acceptance as user • Solution: client_thread_group static, so accessible
Sign Language Messenger Functions • Create New Account • add_db: add to db • check_existing • Prepared statements instead of one-by one insertion: faster
Sign Language Messenger Functions • Login • Database instead of files, faster • Problem: Status of login • Solution: • lok: login ok • lusok: password incorrect • lnotok: login not ok
Sign Language Messenger Functions • Details • Change or retrieve password • Database instead of files, faster
Sign Language Messenger Functions • Contacts List • contacts class • File instead of database, too much space
Sign Language Messenger Functions • Message: Read Messages • Thread • Format of message: <sender> <recipient> <message> • “¿”: delimiter
Sign Language Messenger Functions • Message: InTray • Files instead of database: avoid redundancy
Sign Language Messenger Functions • Message: Send Messages • User message restructured into defined format
Sign Language Messenger Functions • Log-out • Close session
Sign Language Messenger Misc. Design Issues • Java Server Pages vs. Java Servlets • JSP over ASP • JSP same functionality as servlet • Presentation from logic • Mixed HTML and code? Java Beans.