1 / 26

Gaze Controlled Robotic Camera System

Gaze Controlled Robotic Camera System. Anuj Awasthi Anand Sivadasan Veeral Patel. Outline . Background Significance Problem Statement Concept Methodology Specific aims Budget Project Participation Time Frame. Background.

sebastien
Télécharger la présentation

Gaze Controlled Robotic Camera System

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Gaze Controlled Robotic Camera System Anuj Awasthi Anand Sivadasan Veeral Patel

  2. Outline • Background • Significance • Problem Statement • Concept • Methodology • Specific aims • Budget • Project Participation • Time Frame

  3. Background • Laparoscopic robotic surgery • Eye tracker application • Visual mouse • Human factors • Computer vision based control • Face mouse • Voice control

  4. Requirements in Laparoscopic Surgery • Maintain the surgical point of interest in the centre of the image. • Provide the required magnification of the area. • Produce and maintain a horizontal image of the point of interest. • Perform the preceding actions automatically, although they can be modulated by the surgeon.

  5. Visual Mouse Application • Obtaining the horizontal and vertical coordinates with the eye tracker • The technique of live streaming of the horizontal and vertical coordinates • Interfacing of eye tracker and computer

  6. Eye Tracker System

  7. Human Factors Computer vision based control of robotic camera • Camera control based on computer vision tracking of the surgical tools. • Image processing used to differentiate surgical tool of interest from surroundings. • No input required from the surgeon. Disadvantages • Surgeon’s area of interest not taken into consideration • Assumes surgical area to be surgeon’s area of interest always • Surgeon ends up looking at corners of the screen often

  8. Human Factors Face Mouse control for robotic camera • Image based system • Tracks facial features of surgeon real time • Controls camera based on pitch, yaw and roll of surgeon’s face Disadvantages • Constant face movements causes strain • Difficult to keep pace with movement of tools

  9. Human Factors Voice control of Robotic camera • Uses voice and pedal controls • Uses voice recognition techniques • Set of voice commands the camera Disadvantages • Considerable burden on surgeon • Difficult to perform dual inputs

  10. Significance • Reduction in work load on surgeon • Accuracy of surgical tasks • Impact on surgical time • Hands Free Control

  11. Problem Statement “To develop a camera control system which reduces the work load on the surgeon without compromising on the quality of surgeon’s video display ”

  12. Concept Gaze based robotic camera • Acquire gaze of the surgeon with eye tracker. • Camera manipulation using eye tracker data interfaced with robot controls

  13. Robotic Hardware • A small wireless 320 X 240 resolution camera with an inbuilt transmitter • A Receiver Set • Two Servomotors (HS 422) • Links • Usbor Servo Controller • Pivot Post • Gripper • Washer, Set of Clamps, Bolts, Nuts • Eye tracker System

  14. Methodology Operation site Surgeon Site

  15. Surgical Site • Server System (HOST Computer) • Usbor Servo Controller • Visual C++ 6.0 Coding • Servo Motors • Robot Arm • End Effecter • Inverse Kinematics to be followed • Wireless Camera • AAA Battery supplied • Receiver

  16. Surgeon’s Site • Dedicated system (Client ) • Image Acquisition through Internet • Streaming Video • Live Motion JPEG System • Image Processing • Intel’s Open CV Library • Improve Brightness and Contrast • Eye tracker System

  17. Fuzzy Based Control Cluster 2 Cluster 3 Pupil Cluster 1 Cluster 4 Cluster 6 Cluster 5

  18. Fuzzy C-Means Algorithm • Point of Gaze keeps fluctuating. • Entire Eye tracker screen supposed to be divided into clusters. • Fuzzy C-Means Algorithm used. • Degree of belongingness of the point of gaze to a cluster is supposed to be the Degree of Membership of the fuzzy function • Point of Gaze co-ordinates assumed to be same as co-ordinates of cluster centers.

  19. Specific Aims • To cover the surgical area with camera. • To obtain the point of gaze of the surgeon with eye tracker. • To control the robotic camera based on the point of gaze coordinates. • To facilitate Surgeon’s view.

  20. Budget

  21. Project Participation • Robot Assembling : Veeral, Anuj & Anand • Inverse kinematics : Anuj & Anand • Software for Kinematics Control: Anuj & Veeral • Interfacing Eye tracker and Robot : Anuj, Veeral & Anand • Eye Tracker Output : Anand & Veeral

  22. Time Frame

  23. References • M. Farid,F. Murtagh,J.L. Starck.” Computer Display Control and Interaction using Eye-Gaze". School of Computer Science ,Belfast,UK. • Atsushi Nishikawa “Face Mouse : A Novel Human-Machine Interface for Controlling the Position of a Laparoscope” IEEE Transactions on Robotics and Automation,Vol. 19,No. 5 ,October 2003. • Murtagh F. ”Eye Gaze Tracking System-Visual Mouse Application Development”,3rd Year Training Report ,E.N.P.S. Engineering Degree, March-August 2001.

  24. Reference (Contd..) • M.E. Allaf. “Laparoscopic Visual Field – Voice vs. foot Pedal interfaces for control of AESOP Robot "Surgical Endoscopy.Feb 1998. • A. Casals,J. Amat,E. Laporte. ”Automatic Guidance of an Assistant Robot in Laparoscopic Surgery” International Conference on Robotics and Automation, IEEE 1996. • R. Hurteau,S. DeSantis “Laparoscopic Surgery Assisted by a Robotic Cameraman:Concept and Experimental Results”IEEE 1994.

  25. References (Contd….) • George P. Mylonas,Danail Satyanov. ”Gaze Contingent Soft tissue Deformation Tracking for Minimally Invasive Robotic Surgery” MICCAI 2005, LNCS 3749, pp. 843 – 850, 2005. • Shamsi T. Iqbal ,Brian P. Bailey. “Using Eye-Gaze Patterns to Identify User tasks”GHC04,2004

  26. THANK YOU!!!!!! QUESTIONS???????

More Related