1 / 37

Image and Video Upscaling from Local Self Examples

Image and Video Upscaling from Local Self Examples. Gilad Freedman Raanan Fattal Hebrew University of Jerusalem. Background and o verview Algorithm description L ocal self similarity Non-dyadic filter bank Filter design Results. Single i mage upscaling. 1. l arge 2. realistic

drew
Télécharger la présentation

Image and Video Upscaling from Local Self Examples

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Image and Video Upscaling from Local Self Examples Gilad Freedman Raanan Fattal Hebrew University of Jerusalem

  2. Background and overview Algorithm description Local self similarity Non-dyadic filter bank Filter design Results

  3. Single image upscaling • 1. large • 2. realistic • 3. faithful • 4. fast

  4. Previous work parametric image model example based noisy result Freeman et al. 2002 Sun et al. 2008 Shan et al 2008 Fattal 2007 Glasner et al. 2009 generic looking edges

  5. New approach: locale example based local self similarity small upscaling ratios corner step edge 4/5 Increase exemplar quality and size maintain search locality novel components: local self similarity non-dyadic filter bank line step edge 1/2 non smooth shading new non-dyadic filter bank

  6. Background and overview Algorithm description Local self similarity Non-dyadic filter bank Filter design Results

  7. Local self-examples upscaling interpolated image frequency content original image low pass high pass

  8. Local self-examples upscaling For each patch: Add to interpolated image Take corresponding patch Search a local area for best example interpolated image frequency content low pass high pass

  9. Local self-examples upscaling Repeat for all patches, to fill the high frequencies interpolated image frequency content low pass high pass

  10. Overview Algorithm description Local self similarity Non-dyadic filter bank Filter design Results

  11. Local self similarity croppeddownscaled

  12. Local self similarity Patches in original image can matched locallywith ones in downscaled version

  13. Local examples are enough query dbimage local 4.0 2.9 3.55 1.6 1.05 1.05 2.7 2.05 2.05 image database full image best matches 3.3 2.96 3.06 6.5 5.61 5.61

  14. external database local search Visual assessment – external, exact NN, local Large external example database global search Searching the entire image Searching local regions in image

  15. Comparison of example search methods

  16. Background and overview Algorithm description Local self similarity Non-dyadic filter bank Filter design Results

  17. Need for non-dyadic scalings large ratios mixed ratios small ratios

  18. Dyadic filters higher half full frequency content 1:2 lower half dyadic filter bank

  19. Non-dyadic filter bank higher part 4:5 full frequency content 1:2 lower part non-dyadic filter bank

  20. Non-dyadic filters: downscaling example for the 2:3 ratio: dyadic case: 1. convolve with 2 filters 1. convolve with one filter 2. subsample by 2 2. subsample each by 3

  21. Non-dyadic filters: upscaling example for the 2:3 ratio: dyadic case: 1. zero upsample by 1 2. convolve with 1 filter 1. zero upsample by 2 2. convolve with 2 filters 3. sum

  22. Use of the filters in upscaling Upscaling using inverse scaling filters Smoothing by downscaling and upscaling

  23. Background and overview Algorithm description Local self similarity Non-dyadic filter bank Filter design Results

  24. 1. Uniform stretch • When interpolating, smooth areas come from inputUniformly spaced grids should remain uniform 255 brightness 0 grid coordinates

  25. 2. Consistency The interpolated image, if downscaled should be equal to the input. Formally, upsample downsample Previous methods achieve consistency by solving large linear systems to achieve this property

  26. 3. PSF modeling Difference between point spread functions Large image - small camera point spread function Small image - large camera point spread function

  27. 4. Low frequency span frequency When upsampling don’t add new frequencies Upsamplingfilter should be low-pass original interpolated

  28. 5. Singularities preservation similar amount of blur blurred Image interpolated image

  29. Real time video upsampling on GPU main GPU memory Search and filter-banks are both local operations GPU cores NTSC to full HD @ 24 fps

  30. Background and overview Algorithm description Local self similarity Non-dyadic filter bank Filter design Results

  31. Ours X3 (zoomed in) Bicubic x3 (zoomed in)

  32. Ours X3 Bicubic x3

  33. Genuine Fractals™ x4 Ours X4

  34. Ours X4 Glasner et al. 2009 x4

  35. Thank you! Paper & additional results can be found at: www.cs.huji.ac.il/~giladfreedmn

More Related