Ramika Hegde ramikahe@usc

1. GOOGLE STREET VIEW: Capturing The World At Street Level-Dragomir Anguelov, Carole Dulong, Daniel Filip, Christian Frueh, Stéphane Lafon, Richard Lyon, Abhijit Ogale, Luc Vincent, and Josh Weaver, Google 2010 Ramika Hegde ramikahe@usc.edu

2. Introduction • Summary Street View serves millions of Google users daily with panoramic imagery captured in hundreds of cities in 20 countries across four continents. A team of Google researchers describes the technical challenges involved in capturing, processing, and serving street-level imagery on a global scale. • Inspiration ? • Larry Page drove around the San Francisco Bay Area and recorded several hours of video footage using a camcorder pointed at building facades. • Larry’s idea-this kind of imagery useful at a larger scale • Research collaboration with Stanford University called CityBlock that soon thereafter became Google Street View . • Concern The World has 50 million miles of roads across 219 countries. Driving through these roads is equivalent to circumnavigating the globe 1250 times!

3. Generation of Vehicles • 1st Generation • Garage Phase • Cameras, Lasers, and a GPS on the roof & several computers in its trunk. • Disadvantage- Drive through multiple times to capture images • 2nd Generation • Chevy Van • Laser scanners • 2 high-speed video cameras, 8 high-resolution cameras (rosette (R) configuration) • Computers recording data (20 HD at 500 Mbps.) • Disadvantage- could not be built and operated at scale. • 3rd Generation • Lite Cars • Focus-Reliability . Recorded wheel encoded messages from the antilock brake system. • Advantage-recorded a vast amount of Imagery enabling expansion to Austrailia , New Zealand, Japan. • Disadvantage-Low Image Resolution.

4. Generation of Vehicles – contd… • Fourth Generation • ‘R5’ : Street View’s Panoramic camera system. • Custom hinged mast that allows the camera to be retracted when the vehicle passed under low bridges. • 3 Laser scanners on the mast enabling the capture of 3D data alongside the imagery. • Captured the majority of imagery live in Street View today. • Fifth Generation vehicle design in works. • Links: The White House Tour.

5. Data Collection Platforms • In Parallel with the road vehicles, several other data collection platforms were developed-Trike , Trolley , Snowmobile ,Trekker. Snowmobile Trike Trolley Trekker

6. Confrontations • Hard drives are sensitive to shock, vibration, and temperature extremes, both while the vehicle is in operation and, to a lesser degree, while being shipped. • Techniquesto minimize data loss • Shock-mounted disk enclosures • Custom-shipping packaging with extra-thick foam • Solid-state disk drives

7. Street View Cameras R2 System • a ring of eight 11-megapixel, interline-transfer, charge-coupled device (CCD) sensors with commercial photographic wide-angle lenses. R5 System • ring of eight cameras plus a fish-eye lens on top to capture upper levels of buildings. R7 System • 15 of these same sensors and lenses, but no fish-eye, to get high-resolution images over an increased field of view.360 degree view R7 Street View camera system.

8. Pose Optimization • Accurate position estimates of Street View vehicles are essential for associating our high-resolution panoramas with a street map and for enabling an intuitive navigation experience. • A batch algorithm open sourced by Google to achieve a smoother and locally accurate solution for the pose trajectory. This trajectory is computed at a resolution of 100 Hz. • An online Kalman-filter-based algorithm is deployed on the vehicles to provide real-time navigation information to the drivers. • A probabilistic graphical model of the network is constructed . The model includes detailed knowledge about one-way streets and turn restrictions, display approximate street address information, and draw blue overlays on the map

9. Navigating Street View Imagery • 360 degree panorama is the most popular among the street view surfaces. • The experience is made richer by combining Street View Imagery with data sources. • Building on the 3D data that we collect as well as Google Maps data, we can place markers and overlays in the scene, resulting in 3D-annotated Street View images • Google Street view is open to user contribution- Users can correct the exact locations of points of interest by directly dragging markers in Street View and automatically snapping it to facades-Figure(1) • Street View also surfaces user-contributed photos from Flickr, Panoramio , and Picasa in Street View-Figure(2) Figure(2) Figure(1)

10. Leveraging 3D data for Smart Navigation • Click-to-go:3D Navigation Mode • Lets users click their mouse on a point in the scene and be transported to the image nearest to that point’s 3D location. • Enabling such a feature requires creation of a DEPTH MAP.

11. Depth Map • It stores the distance and orientation of every point in the scene .It encodes the scene’s dominant surfaces, such as building facades and roads , while ignoring smaller entities such as cars and people. • Laser range scans or image motion (optical flow) when laser data isn’t available. Laser Range Scan-which accurately measure the depth of a vertical fan of points on the two sides and the front of the vehicle.(Figure 4) • In the absence of laser range data, we recover the depth by computing optical flow between successive images of the street facade on both sides of the vehicle. The optical flow at a given point depends on the vehicle’s motion and that point’s depth.

12. Depth Map-contd… • Panoramic Depth Map: • Tracing rays from each panorama position. • Each pixel in the depth map represents a lookup into a table of 3D plane equations, which enables the client code to reconstruct the real depth values at runtime. • Lossless Compression: The encoded depth map is just a few KB in size. Turning photos into 360 degree Panorama view.

13. Computing 3D Models • Street View data to create photorealistic 3D models for Google Earth. • Google Earth created 3D city models using airborne imagery, resulting in low-resolution facades with little detail are Suitable for fly-through an Unsuitable for walk-through experience. • 3D facade models reconstructed from Street View’s laser scans and imagery are high resolution. • Final 3D Facade Models : existing airborne models are fused into a single model that has high-resolution facades as well as rooftops and back sides from an airborne view. Original 3D models of a New York (airborne data) only. (b) Fused 3D model with high-resolution facades.

14. Pros & Cons of the Paper • Pros Gives us a surface view of how exactly Google Street View works. Enlightens us with the evolution of Vehicles used in Google Street View. Depicts how camera equipment has changed from Street View’s Initial Stage to the Current Stage. Describes ‘Click to go’ Feature well. • Cons Doesn’t give us a clear picture of Depth Map. Doesn’t describe how User’s can contribute to Street View. Hasn’t mentioned additional information provided-trafficand weather updates.

15. Additional • Wi-Spy- User Privacy. • Germany fines Street View. • Apple’s Street View. • Apple’s Map failure. • Street View Expansion. • Google Street View Hyperlase.