180 likes | 333 Vues
This presentation will probably involve audience discussion, which will create action items. Use PowerPoint to keep track of these action items during your presentation In Slide Show, click on the right mouse button Select “Meeting Minder” Select the “Action Items” tab
E N D
This presentation will probably involve audience discussion, which will create action items. Use PowerPoint to keep track of these action items during your presentation • In Slide Show, click on the right mouse button • Select “Meeting Minder” • Select the “Action Items” tab • Type in action items as they come up • Click OK to dismiss this box • This will automatically create an Action Item slide at the end of your presentation with your points entered. Course Projects Topics and Plans Martin Jagersand
Today • Some tips on carrying out your projects • Literature search and readings • Quick prototyping (e.g. Matlab) , then final implememntation (e.g. c, c++), or combining Matlab with c, c++ mex files. • Balance between reading and doing • Labs and resources available • Win free trip? • Short presentation and discussion of your topic and plans
Preliminary project topics • Individual and group aspect: • Every person has some individual focus and all individual pieces combine to a whole. • Main topics: Vision for 3D modeling and robotics • Real time tracking • Integrating tracking with 3D modeling • Integration of a-priori knowledge in 3D modeling • Predictive display and visualization • Visual servoing for robots (manipulator or mobile) • Visual specification and planning of robot tasks
Resources • SW: I/we will install and try to help support: • Real time video input (under linux, video pipeline done) • Basic tracking, XVision • Geometry, Hand-eye, Robotics code • HW: Access to machines in Robotics/Vision lab • Cameras: Linux IEEE 1394 in both grad labs and course labs csc235. • Web cams, 200Hz high speed cams, 1600x1200 Hi-Res cam available • VZ motion tracker (3000Hz special device) • Could also use digital still camera, camcorder. • Vision for motion control: Robot arms, hands, mobile robot • WAM, Barrett hand, Segway (one dedicated more coming), old Pumas • Vision and haptics: 3 Phantom omnis. • Visualization: Ok in lab (HW acc graphics), In research lab (new ATI and nvidia, SGI HW, projectors and CAVE) • Anything else? Some resources for buying available
Martin’s tips • Plan incremental progress and checkpoints. • Makes it easier to identify promising directions as well as difficulties and redefine plans as needed. • Find balance between reading and doing • It is difficult to fully grasp methods by only reading • Some experiments are incomplete, results wrong • Practical trying out can add a lot of insight. • Learn how to quickly prototype in e.g. matlab
Literature search • Goal: Find the 10-15 most relevant and recent papers in a subarea. • Method: • Seed with a few relevant papers. • Do internet search. e.g. “research index” or google scholar • Do citation search backwards and forwards. • Find common “buzz words”. Do title and abstract text search. • Check most recent proceedings manually. (They won’t be indexed yet)
Literature search 2 • Expect to: • Read the titles of hundreds of papers (and web pages) • Read the abstract of 20-40 of papers • Skim through dozens of papers • In order to find the 10-15 or so relevant papers. Read these in detail to understand the topic. • Of these select a handful of the most closely related to benchmark your project to.
Report: • Review • Summarize the main contributions and comparing the results in the papers. • Your contribution and experiments. • Methods • Results • Discussion • Where does it fit into the bigger picture • Future work
Schedule • Now good time to think about and refine project plans • Late Oct Written project proposal. • Include reference list and a start at literature review, ie. Read some papers and write a few pages summary • Throughout course in class: • Keep up to date on your project progress. • In class presentation of project readings and analysis • End of semester: Project reports.
Preliminary project topics • Objective: Vision for 3D modeling and robotics • Individual parts • Real time tracking • Integrating tracking with 3D modeling • Integration of a-priori knowledge in 3D modeling • Predictive display and visualization • Mapping and navigation for mobile robots • Visual servoing for manipulators • Visual specification and planning of robot tasks • Singularity and obstacle avoidance • Subgroups: Vision, Robotics
GPU accelerated visual tracking • Tracking readings: Color, Feature, SSD, SIFT. • Investigate what maps naturally to GPU, CPU • Make incremental plan, e.g. • Video pipeline: Cam->Video RAM or CPU RAM? • Basic image processing on GPU: Lin alg, conv, filt, im deriv • Implement and test various tracking: Color, SSD, SIFT • Integrate with other system parts • Design and carry out experiments: Test sequences robustness, accuracy etc.
Tracking and 3D modeling • Make tracking more robust by restricting 2D image tracker movement to those consistent with a 3D interpretation • Rigid constraints: • Loose constraints • Language for partial constraints (Collab with visual spec) • Convergence tradeoff: • Restricted tracker may not reach elliptical point • Unrestricted tracker may track wrong points • Experiments: What are good test sequences? How accurate is tracking? Captured 3D? How robust?
Use of a-priori knowledge in modeling • Can make 3D capture easier: • Tells if recovered scene is “probable” • Orth, plan etc: and gives Euclidean structure w/o cumbersome “self calibration” • Types of a-priori knowledge: • Generic: orthogonality, parallelism, planes. • Specific: architectural: houses&features(doors windows) Indoors: Furniture on floor, lamps on wall, scales: room, furn, items… • How to mathematically incorporate: • Hard constraints • Probabilistic • How to practically add: • Image editor collaborate with Visual spec project • Experiments: • Scenes from photos, • Indoor scenes from video
Predictive DisplayVisual User Interface • Systems oriented project • How to modify 3D capture to incrementally detect changes in 3D remote scene, send and incorporate in model • How to display 3D model in HMD • Minimize latencies • How to track and interpret human motions. Control robot motion based on these. • Could also be set up in CAVE
Visual servoing of manipulators • Manipulators: Arm’s hands, high DOF devices • Investigate properties of measurement: • E-functions, properties of robot. • How to estimate visual-motor: • kinematics? Dynamics? • Local, global? • Design controllers • Combine joint and visual feedback • Integrate with tracking and visual space specifications • Experiements.
Visual specification • What are natural visual task primitives? • What tasks do the solve? Completeness under particular geometry? • How “smooth” E-functions do they give? (Collaboration with visual servoing). • How to have human enter: • Pointing 2D image editor • and gesturing in 3D • Experiments and test in vision-based manip
Singularity and Obstacle avoidance • Investigate what aspects of calibrated Euclidean approach carry over • How to find and characterize “difficult regions” • Find: Computer vision, contact, other sensing • Region representation: Points, Regions, Potential function • How to combine with visual servoing controller • Primary or secondary controller objective • Effects on convergence • Experiments with synthetic data, real sequences
Next steps • Firm up project ideas, specification • Identify readings • Discuss plans • Write project proposal • Investigate hardware needs and plan use • Plan interaction with other people/groups • Iterate: Devise method, implement, test • Final integration with other parts • Write final report