1 / 9

Data Deluge

010011100100100110101000101010100010101 1001010011011000111011010101101010101010101110110110 0101110101111 1001101101010. Data Deluge. Rama Chellappa Richard Baraniuk. Accelerating Data Deluge. 1250 billion gigabytes generated in 2010 # digital bits > # stars in the universe

sondra
Télécharger la présentation

Data Deluge

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 010011100100100110101000101010100010101100101001101100011101101010110101010101010111011011001011101011111001101101010010011100100100110101000101010100010101100101001101100011101101010110101010101010111011011001011101011111001101101010 Data Deluge Rama ChellappaRichard Baraniuk

  2. Accelerating Data Deluge • 1250 billion gigabytes generated in 2010 • # digital bits > # stars in the universe • growing by a factor of 10 every 5 years • Total data generated > total storage • Increases in generation rate >>increases in transmission rate Available transmission bandwidth

  3. In 2009 alone, the U.S. Air Force shot 24 years' worth of video over Iraq and Afghanistan using spy drones. The trouble is, there aren't enough human eyes to watch it all. The deluge of video data from these unmanned aerial vehicles, or UAVs, is likely to get worse. By next year, a single new Reaper drone will record 10 video feeds at once, and the Air Force plans to eventually upgrade that number to 65. John Rush, chief of the Intelligence, Surveillance and Reconnaissance Division of the U.S. National Geospatial-Intelligence Agency, projects that it would take an untenable 16 000 analysts to study the video footage from UAVs and other airborne surveillance systems. The best—and perhaps only—way forward is to have a computer watch it all. But programming a system to automatically search video and pick out noteworthy information is not an easy problem. And so far, no one has developed software that can keep up with the military's high-tech hardware.

  4. Case in Point: DARPA ARGUS-IS • 1.8 Gpixel image sensor • video rate output: 770 Gbits/s • data rate input: 274 Mbits/sfactor of 2800x way out of reach ofexisting compressiontechnology • Reconnaissancewithout conscience • too much data to transmit to a ground station • too much data to make effective real-time decisions

  5. Past DARPA Efforts • Investment in the $10s of millions has led to promising, yet limited progress • AVS – Airborne visual surveillance • deluge handled by processing only video frames that contain the activity of interest • few minutes of video clips of three vignettes • very limited in scope in terms of activities • only processed clips containing activities • VIVID • videos of longer length processed for detecting and tracking moving vehicles and humans • three minutes of training video + 1 minute of testing video for confirmatory ID problem • ARGUS, … • Sense/process/transmit brick wall looms!

  6. Data Deluge Brick Wall • No general framework for dealing with deluge • trading off sensor resolution vs. processor complexity • trading off computation at sensor vs. computation at collection point vs. computation in the cloud • optimizing SWAP • universal data representations • applicable to wide range of applications • Meanwhile, promising recent work in • computational photography (marry sensor w/ computer) • compressive sensing (randomized dimensionality reduction) • sparsity (new compression techniques) • machine learning (processing using massive data archive) • computational platforms (low SWAP) …

  7. Study/Workshop Goal • Explore and synthesize recent progress on new • mathematical models for sensor data • sensing systems • computational data processing platforms that enable radically new families of sensors that deal head-on with the data deluge • Understand fundamental limits on degree we can mitigate data deluge (function of sensor/task) data information

  8. Potential Study Group Members • Rama Chellappa, Richard Baraniuk (co-chairs) • Takeo Kanade, Marc LeVoy, Ruzena Bajcsy • Bob Bolles, Shree Nayar, Berthold Horn • Terry Tao, David Donoho, Guillermo Sapiro, Ronald Coifman, … • Folks from MERL, Sandip Tiwari (related DSRC study) • Mathematicians, statisticians, vision experts, optics, materials, computer engineers, software folks, interface folks,… • Interplay with on-going DARPA/ISAT initiatives • “Black Clouds” study (2009-2010) • BLADE (I2O) • Analog-to-Information Receiver (MTO) • KECoM (MTO) …

More Related