1 / 19

Multimodal Perception in Traffic Scene Understanding Experiment, Beijing 2011

An intersection experiment conducted in Beijing merging laser, camera, and GPS technologies to study object detection and scene understanding in traffic scenarios. Data fusion and research tasks explored for navigation and reasoning.

dorjan
Télécharger la présentation

Multimodal Perception in Traffic Scene Understanding Experiment, Beijing 2011

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. MPR Intersection ExperimentBeijing, 2011 SpringJointly by PKU and HEUDIASYC

  2. MPR Intersection Experiment • Motivation/Purpose

  3. Scenario Design Laser Camera Host V S N

  4. Host V: POSS GPS/IMU Two Flea2 Compose stereo Cameras Horizontal laser scanner Obliquely laser scanner More?

  5. Scenario Design Laser Camera Host V Haidian Gymnasium Laser1: SW, NPC4 S Peking Univ. N Laser2: NW, NPC1 Server NPC7 & AP Laser3: NN, NPC5 An overhead bridge ChangChun Dorm. of Peking Univ.

  6. Scenario Design Host Vehicle Trajectories Haidian Gymnasium S Peking Univ. N An overhead bridge ChangChun Dorm. of Peking Univ.

  7. Intersection Experiment • Time and Place: • 05/23/2011, 13:00-17:00 • T word intersection, in the west of PKU • Experiment Facilities: Sensors • Intersection • 3 ground lasers • 1 ground mono camera • Vehicle • 2 lasers: horizontal and obliquely downward at the front of car • Stereo camera: 2 mono cameras • GPS

  8. Cont. • Experiment Facilities: Computers • Intersection • 3 laptops: laser data collection client • 1 laptop: laser data collection server • Vehicle • 1 pc: laser and GPS data collection • 1 pc: video data collection • Other Facilities: • Batteries, Cables, Wireless AP, Patch Boards, Tripod, etc…

  9. Experimental Procedure: Preparation • Vehicle • Sensors: Stereo Camera, Horizontal Laser and Obliquely Laser • Calibration: Online • Mono Camera + Horizontal Laser <(u, v)、(x, y, z=1.0)> (x, y, ?)

  10. Cont. • Intersection • Sensors: Ground Lasers and Video Camera • Lasers must be horizontal • Time Synchronization • All computers should in one network(PC on POSS and Laptops at intersection) • Do time synchronization • Calibration • Ground Lasers: Nature object or land markers • Ground Lasers + Video Camera: Land markers

  11. Ground Laser + Video Calibration 4 5 6 3 2 1

  12. Ground Laser + Video Calibration

  13. ExperimentProcedure:Measurements • Measurements • Start the measurements of ground lasers and video camera. • Start the measurements of on-vehicle lasers and cameras. • Let the host vehicle run across the intersection

  14. Experiment Data • Intersection • Ground Lasers Data • *.lms1, *.lms2, *.lms3 • Ground Video • exp20110523_westgate_1.avi • exp20110523_westgate_1.log : start time of video

  15. Experiment Data • Vehicle • Lasers • *.lms1: Horizontal Laser Data • *.lms2: Obliquely Laser Data • Stereo Camera • *.avi: stereo video • *.txt: start time of corresponding video • GPS data • *.pos

  16. Experiment Data • Primary Data Processing • Time Synchronization • Laser data, GPS data • Correct the start time of video • Offline Calibration • Intersection ground laser + video • Laser and Video data fusion • Using calibration results to project laser points on video images. • Intersection • Vehicle

  17. Experiment Data • On Vehicle Data Fusion • Incorrect calibration result • Didn’t notice that the ground is not horizontal while doing calibration

  18. Experiment Data • Intersection Data Fusion • Not so exactly

  19. Research Tasks • What perceptual algorithm do we study? TASK 1: Multimodal perception • Multimodal sensor fusion-based object detection, recognition and tracking • Navigable space detection (road geometry, boundaries, lanes) • Static object detection: signs, trees, facades • Moving/movable object detection and tracking: cars, cycles, pedestrians • Multimodal data constrained SLAM (GPS, GIS) • Data representation and vicinity dynamic maps • What knowledge do we need to reasoning? TASK 2: Reasoning and scene understanding • An open comparative dataset for testing for cross-cultural robustness in traffic scene understanding • Learning for scene semantics and moving object behaviors • Traffic situation awareness with uncertainty, scene semantics and information fusion

More Related