1 / 31

Real-time Non-photorealistic Viewfinder

This blog post discusses the innovative approach to Non-Photorealistic Rendering (NPR), emphasizing its ability to stylize images while sacrificing photorealism. By utilizing a GPU-based backend for efficient image processing, we explore techniques like color simplification, edge enhancement, and quantization. The NPR viewfinder effectively transforms ordinary scenes into artistic interpretations, showcasing its applications in visual communication and augmented reality. Join us as we dive into the details of our NPR algorithm and demonstrate how it can change our perception of the world around us.

nessa
Télécharger la présentation

Real-time Non-photorealistic Viewfinder

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Real-time Non-photorealistic Viewfinder

  2. Real-time Non-photorealistic Viewfinder * Tony Hyun Kim and Irving Lin CS478 Computational Photography Dev. blog: cs478.blogspot.com *Raskar, et. al. (2004)

  3. INTRODUCTION: NPR * • Non-photorealistic rendering (NPR), or “stylization,” allows artistic interpretation of an image at the expense of photorealism. • Promotes simplicity and reduction of detail, better emphasizing the semantic content of the image. • Applications: visual communication, augmented virtual reality, image compression. Also, just looks interesting. *Raskar, et. al. (2004)

  4. OUR WORK: TOPICS TO BE DISCUSSED • Demonstration of NPR viewfinder • How it works: • Details of the NPR algorithm • GPU-based “backend” for image processing • Integration with tablet hardware (flash)

  5. Our typical view of the world. Boring…

  6. Let’s stylize our worldview!

  7. Another perspective…

  8. Let’s take it outside…

  9. Going towards the university

  10. Cool car

  11. Going to the Quad…

  12. Cartoon pillars

  13. More cartoon pillars!

  14. View of the Hoover tower

  15. Now, in the Hoover tower

  16. The tower’s shadow

  17. HOW IT WORKS NPR algorithm Blah BlahBlah • The world looks “cartoony” with: • Color simplification • Edge enhancement • Color quantization (optional)

  18. HOW IT WORKS NPR algorithm Blah BlahBlah • The world looks “cartoony” with: • Color simplification • Edge enhancement • Color quantization (optional) • Use a bilateral filter: • “Clustering” of nearby pixel colors • Number of approximations available • Separable approx. (Pham, et. al. 2005) • No spatial weight. (Fischer 2006)

  19. HOW IT WORKS NPR algorithm Blah BlahBlah • The world looks “cartoony” with: • Color simplification • Edge enhancement • Color quantization (optional) • Use neighbor gradients: • Just like the sharpness calculation in autofocus routine (“Hello Camera”) • Up-down-left-right gradients

  20. HOW IT WORKS NPR algorithm Blah BlahBlah • The world looks “cartoony” with: • Color simplification • Edge enhancement • Color quantization (optional) • Bucket the pixel value: • Quantization of Y  cell-shaded look • Quantization of U,V  funny colors

  21. HOW IT WORKS GPU-based backend Blah BlahBlah • NPR algorithm is a sequence of image filters • For example: • Bilateral filtering (multiple rounds) • Edge detection • For performance reasons, we perform all image processing on the GPU • GPU-backend implemented in FCamAppThread

  22. HOW IT WORKS GPU-based backend Blah BlahBlah Inside of FCamAppThread…

  23. HOW IT WORKS GPU-based backend Blah BlahBlah Inside of FCamAppThread… • Get a single 640 x 480 viewfinder frame

  24. HOW IT WORKS GPU-based backend Blah BlahBlah Inside of FCamAppThread… • Copy frame data to “SharedBuffer” • Data is visible to both CPU and GPU

  25. HOW IT WORKS GPU-based backend Blah BlahBlah Inside of FCamAppThread… • Apply sequence of shaders in GPU

  26. HOW IT WORKS GPU-based backend Blah BlahBlah Inside of FCamAppThread… • Copy result back to FCam renderer

  27. HOW IT WORKS GPU-based backend Blah BlahBlah • Typical perf: ~100 ms per frame (10 fps)

  28. Switching gears…

  29. FLASH-BASED SELECTIVE NPR No-flash • Request a stream of alternating flash/no-flash video. Steer it to the right destination. • Use flash/no-flash pair to segment foreground and background (very naïve!) Flash

  30. FLASH-BASED SELECTIVE NPR

  31. CONCLUSIONS • Demonstration of NPR viewfinder • Stylizes 640 x 480 video at 10 fps • Bilateral filtering, edge detection, quantization • See: cs478.blogspot.com for more results • Backend for GPU-based image processing: • Generic, can be used with other filters • Proof-of-principle integration with hardware • Flash-based, background-selective NPR

More Related