1 / 41

Video Object Tracking and Replacement for Post TV Production

Video Object Tracking and Replacement for Post TV Production. LYU0303 Final Year Project Spring 2004. Outline. Brief project overview and targets achieved New components added to the proposed system Working principles of individual parts Future work Q & A. Overview.

goldsteinr
Télécharger la présentation

Video Object Tracking and Replacement for Post TV Production

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Video Object Tracking and Replacement for Post TV Production LYU0303 Final Year Project Spring 2004

  2. Outline • Brief project overview and targets achieved • New components added to the proposed system • Working principles of individual parts • Future work • Q & A

  3. Overview • Post-TV productionprocesses and changes the content of a video clip • Difficult for computers to process video in a global view • Step-by-step programming enables the computer to “think” more thoroughly

  4. Overview • We have introduced the following parts in the last semester: • Bitmap I/O • RGB <-> HSV converter • Edge detector • Edge equation finder • Translation detector • Texture mapper • New parts have been added in order to increase thefunctionality of the system

  5. Additional Basic Components • The following parts have been added • Graphical User Interface (GUI) • Corner point finder • Video file reader and writer • Video player processor (frame-based) • Improved texture mapper • The following part has been modified • Bitmap I/O • Edge finder • Texture mapper • Removed Equation processor

  6. Graphical User Interface • Although UI is not a main project part, a good GUI can significantly decrease the time needed for processing and program maintenance • Since C++ language is being used, the Microsoft Foundation Classes are adopted to develop the user interface • Provide most of the basic functions needed for information input and output

  7. Graphical User Interface

  8. Corner Point Finder • As we know that processing dots are much faster than processing lines, we may wish to try to find the corner points directly before finding the edge equations • Stick some brightly coloured labels at the corners of the rectangular surface • Can apply on cylindrical objects as well to indicated the curve control points • A simple K-Means method is used to group the points together.

  9. Corner Point Finder • In order to know the orientation of the mapped texture, we need to know which is the “first” point • For simplicity we just make the first point larger so that it contains more corner points than the others First point

  10. Corner Point Finder Group No. of Member • We repeat the process until all the corner points has been processed • Points which are too far away from the main groups are discarded • Find out the centroids of the groups and move them out to give an estimation of corner positions 1 1 1 2 1 5 2 1 2 5 3 1 3 5 4 1 4 5

  11. Corner Point Finder

  12. Video File I/O • As we are talking about video processing, it is inevitable to handle video reading and writing • DirectShow interfaces are being used in the project • Able to process any type of AVI video as long as the Window Media Player can play them

  13. Video Player • We need to choose a video renderer that is suitable for our application • To reserve the function of real-time processing and video texture in the future, the Video Mixing Renderer 9 (VMR9) is used • Allows the rendering of more than one video or bitmap on the screen at the same time

  14. Video Processor • Video are sequence of bitmap frames • We capture each video frame and process them in the same way as processing the bitmaps • After a video frame is being processed, we add them to an AVI file • Default video compressor is DivX Mpeg4 encoder (FourCC code “divx”)

  15. Video Renderer Video System Structure

  16. Texture Mapper • A graphics design technique used to wrap a surface of a 3-D object with a texture map • The 3-D object acquires a surface texture similar to the texture map. • Colors, brightnessvalues or altitudes

  17. Texture Mapper Image coordinates Texture coordinates (r,c) (u,v) Mapping function

  18. Scan-line conversion Scanline yk scanning order Scanline yk+1 • line by line • process each pixel on every line • last semester, we mapped cubes. How about this semester? for every line

  19. Cylindrical mapping How to map this cylinder with different textures?

  20. Cylindrical mapping • Mapping cylinders 3 parts: • Surface detection • Line scanning • Texture lookup + Orientation determination

  21. Surface detection • Hough algorithm • to detect the two straight edges • four points are detected

  22. Surface detection • Scanning from top to bottom • the scanning slope is orthogonal • yellow points are detected • by color transition

  23. Surface detection • Model the curved edges by ellipses • x2 / a2 + y2 / b2= 1 • 2 ellipses are needed • rotational factor y x z

  24. Scan-line mapping • Scan-line from left to right • Problem: How to determine the corresponding texture for each pixel, say p=(x,y) ? (x,y) y x z

  25. Texture lookup • Rotate the cylinder to the upright position along z-axis • Translate the cylinder to (0,0) and determine the z coordinate p=(x’,y’,z’) • Rotate the along x-axis  upright from viewpoint (0,0) y h x z

  26. Texture lookup • Detect the marker which indicates where the texture starts • Look up the texture coordinate (u,v) by (y’+h/2)/h = v tan-1(x’/z’) / 2 = u y x z (u,v)

  27. Cylindrical mapping After mapping the top we have : MAPPING MAPPED CYLINDER TEXTURES

  28. Spherical mapping How to map this sphere with different textures?

  29. Spherical mapping • Mapping spheres Also 3 parts: • Surface detection • Line scanning • Texture lookup + Orientation determination

  30. Surface detection • Scan from 4 directions and get the boundary points • Determine center

  31. Line scanning • Scan from left to right. • For each pixel p=(x,y), look up the texture by transforming it to the upright position (0,0) y Upright x z

  32. Texture lookup • Translate to origin • Look up z-coordinate • Rotate the ball along z-axis • Rotate the ball along x-axis  (x’,y’,z’) (5) (6) (0,0) y x z

  33. Texture lookup • Lookup the texture by v = cos-1(y’/R) / u = (cos-1(x’/(R sin(v))) ) / (2)

  34. Spherical mapping • After mapping: Mapped Sphere Texture

  35. Shadow mapping • Mapping of surface brightness • Method: • Covert the image from RGB  HSV • Modify the V (intensity) value • 5 types of shadow mapping

  36. Shadow mapping • Mapping with texture brightness  often unrealistic • Mapping with image brightness

  37. Shadow mapping • Mapping with average brightness of the image  more realistic • Mapping with other shadow maps (e.g. wood)

  38. Shadow mapping • Transparency mapping (background information has to be obtained)

  39. Problem with sphere mapping The texture is “squeezed” at the pole

  40. Solution Modify the texture to this: Mapped sphere becomes: No distortion at pole

  41. Q & A

More Related