Create Presentation
Download Presentation

Download Presentation

Foundations of Computer Vision Lecture 13

Download Presentation
## Foundations of Computer Vision Lecture 13

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -

**Foundations of Computer VisionLecture 13**Roger S. Gaborski 1**Color Segmentation**Roger S. Gaborski**RGB Color Image**M x N x 3 array of color pixels The three planes correspond to red, green and blue values. Each pixel location is actually a triplet, (r, g, b) Each plane drives the respective red, green and blue inputs of a color monitor Data classes: Double [0,1] Uint8 [ 0 255] Color Image Processing Roger S. Gaborski Roger S. Gaborski 3**ZR**ZG ZB Blue component Green component Red component Roger S. Gaborski**Row of Data (red, green and blue)**Roger S. Gaborski 6**Extract individual color planes:**fRed = rgb_image(:,:,1); fGreen = rgb_image(:,:,2); fBlue = rgb_image(:,:,3); Combine individual frames rgb_image = cat(3, fRed, fGreen, fBlue); Roger S. Gaborski Roger S. Gaborski 8**RGB color space**rgbcube(vx, vy, vz) Roger S. Gaborski Roger S. Gaborski 9**HSV color space**hsv_image = rgb2hsv(rgb_image); rgb_image = hsv2rgb(hsv_image); Roger S. Gaborski Roger S. Gaborski 10**SegmentationNot Well Defined**Definition – divide an image in to non-overlapping regions based on properties of image, such as, Gray level Color Texture Motion Depth perception Etc. Roger S. Gaborski Roger S. Gaborski 11**Image segmentation in RGB**Partition image into regions Color-based segmentation Obtain a sample of color pixels that match region we wish to segment Estimate the mean of the three colors Store color values in column vector m Classify each pixel in image as having a color specified by m Roger S. Gaborski Roger S. Gaborski 12**Classify pixels- Euclidean distance**We need a measure of similarity to classify pixels – how ‘close’ is a given pixel to m? Let z denote arbitrary pixel in the image z is similar to m if the distance between them is less than some threshold, T Euclidean distance: D(z,m) = ||z-m|| where ||x|| is the norm of x Roger S. Gaborski Roger S. Gaborski 13**Euclidean Distance**D(m,z) = [ (z-m)T (z-m) ]1/2 = [(zR-mR)2 + (zG-mG)2 + (zB-mB)2]1/2 D(m,z) T is a solid sphere of radius T. Points within, or on the surface are defined as being part of the segmented region But, how can you include the information that the red pixels vary over a wide range, but green and blue pixels don’t? Roger S. Gaborski Roger S. Gaborski 14**RECALL: Standard deviation**A measure of the spread of the data Calculation: average distance from the mean of the data set to a point Denominator of n-1: samples n: entire population Roger S. Gaborski Roger S. Gaborski 15**Variance**Another measure of the spread of the data in a data set Calculation: Variance: original statistical measure of spread of data. unit: square term. e.g. cm2 SD: square root of variance, the same unit with original data Roger S. Gaborski Roger S. Gaborski 16**Covariance**Variance – measure of the deviation from the mean for points in one dimension e.g. heights Covariance as a measure of how much each of the dimensions vary from the mean with respect to each other. Covariance is measured between 2 dimensions to see if there is a relationship between the 2 dimensions e.g. number of hours studied & grades obtained. The covariance between one dimension and itself is the variance Roger S. Gaborski Roger S. Gaborski 17**Covariance**variance (x) = covariance (x, y) = For 3-dimensional data set (x,y, z): covariance (x, y), covariance (x, z), covariance (y, z) covariance (x, x) = variance (x) , covariance (y, y) = variance (y) covariance (z, z) = variance (z) Roger S. Gaborski Roger S. Gaborski 18**Covariance matrix**Representing Covariance between dimensions as a matrix. e.g. for 3 dimensions: cov(x,x) cov(x,y) cov(x,z) C = cov(y,x) cov(y,y) cov(y,z) cov(z,x) cov(z,y) cov(z,z) Diagonal is the variances of x, y and z cov(x,y) = cov(y,x) hence C matrix is symmetrical along the diagonal N-dimensional data will result in nxn covariance matrix Roger S. Gaborski Roger S. Gaborski 19**Interpretation**Exact value is not as important as it’s sign. A positive value of covariance indicates both dimensions increase or decrease together. e.g. number of hours studied vs. the grade for that subject A negative value indicates while one increases the other decreases, or vice-versa. e.g. active social life at RIT vs. performance in CS dept. Covariance is zero: the two dimensions are independent of each other. e.g. heights of students vs. the grades for a subject Roger S. Gaborski Roger S. Gaborski 20**Covariance**MATLAB: cov Diagonal: variances Matrix is symmetric Roger S. Gaborski Roger S. Gaborski 21**MahalanobisDistance**• Generalization D(z,m) = [ (z-m)T C-1 (z-m) ]1/2 where C is the covariance matrix of the sample data • D(z,m) T is a solid 3-D elliptical body • The principal axes are orientation in direction of maximum data spread • If C = I, the Mahalanobis reduces to the Euclidean distance Roger S. Gaborski Roger S. Gaborski 22**Covariance**For 3-dimensional data set (x,y,z): cov(x,y), cov(x,z), cov(y,z) cov(x,x) = var(x) , cov(y,y) = var(y), cov(z,z) = var(z) Roger S. Gaborski Roger S. Gaborski 23**CALCULATING COVARIANCE MATRIX**Data = [ 1 2 3; 5 7 9; 4 5 6] Mean = 3.3 4.6 6.0 (red, green, blue ) Subtract mean from each color Data value dataM = -2.3333 -2.6667 -3.0000 1.6667 2.3333 3.0000 0.6667 0.3333 0**C11 = sum(dataM(:,1) .* dataM(:,1))/2**(divideby n-1 = 3-1 = 2) C11 = 4.3333 >> C12 = sum(dataM(:,1) .*dataM(:,2))/2 C12 = 5.1667 Etc.**NUMERICAL EXAMPLES**Roger S. Gaborski Roger S. Gaborski 26**NUMERICAL EXAMPLE: Euclidean Distance**D(z,m) T is a solid sphere of radius T. Points within, or on the surface are defined as being part of the segmented region Roger S. Gaborski Roger S. Gaborski 27**Euclidean Distance**dEuc(P,M) =SQRT { (P-M)’ (P-M) } M = [5; 7; 6], P = [7; 6; 1] M are the means of the samples, P is a specific pixel M and P are 3x1 vectors P-M is a 3x1 vector Need to transpose first vector**(P-M)’ = [ 2 -1 -5]**(P-M)’ (P-M) = 30 dEuc(P,M) =SQRT { (P-M)’ (P-M) } = SQRT(30) = 5.47**Mahalanobis distance**• Generalization where C is the covariance matrix of the sample data • D(z,m) T is a solid 3-D elliptical body • The principal axes are orientation in direction of maximum data spread • If C=I, the Mahalanobis reduces to the Euclidean distance Roger S. Gaborski Roger S. Gaborski 30**Assume inverse of covariance matrix:**1 0 0 0 1 0 0 0 .5 Roger S. Gaborski**Mahalanobis distance**dMah(P,M) =SQRT { (P-M)’ C-1 (P-M) } = [ 2 -1 -5]* C-1 = [2 -1 -5] 1 0 0 0 1 0 0 0 .5 = [2 -1 -2.5 ] (P-M)**[2 -1 -2.5 ] (P-M) =**• [2 -1 -2.5 ] 2 • -1 = 17.5 • -5 • But we still need to take square root; • Sqrt(17.5) = 4.18**Examples of segmentation**using color information Roger S. Gaborski**Segment an orange based on color samples – No covariance**• What steps do we • take? • 1. • 2. • 3. Roger S. Gaborski Roger S. Gaborski 35**Segment an orange based on color samples**• What steps do we take? • Sample orange pixels • calculate means of r, g, b • 2. ? • 3. ? Roger S. Gaborski Roger S. Gaborski 36**Segment an orange based on color samples**What steps do we take? 1. Sample orange pixels; Calculate means of r,g,b 2. Calculate distance measure For every pixel in image –LOOPS? 3. ? Roger S. Gaborski Roger S. Gaborski 37**Segment an orange based on color samples**What steps do we take? 1. Sample orange pixels, calculate means of r,g,b 2. Calculate distance measure For every pixel in image – NO LOOP 3. Threshold distance image Roger S. Gaborski Roger S. Gaborski 38**Pixel sampling**A small 5x5 patch sampled samplePatch = image(200:204,300:304,:) Roger S. Gaborski**%Distance Measurement**image = imread('Orange1.jpg'); image = im2double(image); figure, imagesc(image),grid %Select samples samplePatch=image(200:204,300:304,:) %Mean values mr = mean(mean(samplePatch(:,:,1))) mg = mean(mean(samplePatch(:,:,2))) mb = mean(mean(samplePatch(:,:,3))) Roger S. Gaborski Roger S. Gaborski 40**distMeasure = sqrt((image(:,:,1)-mr).^2 +**(image(:,:,2)-mg).^2 + (image(:,:,3)-mb).^2); figure, imagesc(distMeasure),colormap(gray), title('Distance Measure') figure, imagesc(distMeasure<.15), colormap(gray), title('distMeasure<.15') figure, imagesc(distMeasure<.25), colormap(gray), title('distMeasure<.25') Roger S. Gaborski**Euclidean Distance Map**Background have larger distance with sampled patch, so by intensity image it’s brighter Orange has small distances with sampled patch (darker) T=0.25 (more area included) T=0.15 Roger S. Gaborski Roger S. Gaborski 42**Orange Flower Example**• 10 pixel sample from orange flower • >> mr = mean(data(:,1)) • mr = 1 • >> mg = mean(data(:,2)) • mg = 0.5325 • >> mb = mean(data(:,3)) • mb= 0 Roger S. Gaborski**distMeasure = sqrt((I(:,:,1)-mr).^2 + (I(:,:,2)-mg).^2 +**(I(:,:,3)-mb).^2); >> figure, imshow(distMeasure, []) Roger S. Gaborski**Flower should be pixel locations with small distance measure**Threshold distance measure What threshold value??? figure, hist(distMeasure(:),100) Roger S. Gaborski**>> Iflower = distMeasure<.45;**>> figure, imshow(Iflower) Roger S. Gaborski**>> Fred = I(:,:,1).*Iflower;**>> Fgrn = I(:,:,2) .*Iflower; >> Fblu = I(:,:,3) .*Iflower; >> F(:,:,1) = Fred; >> F(:,:,2) = Fgrn; >> F(:,:,3) = Fblu; >> figure, imshow(F) Roger S. Gaborski**Extract and Analyze Brandy**im=imread('IMGP1715.JPG'); >> imSm = imresize(im, .25); >> figure,imshow(imSm) Roger S. Gaborski 48**Approaches**Gray scale thresholding Roger S. Gaborski 49**1.Gray scale thresholding**• Approach – First convert to gray scale (losing color • information), then threshold • >> imSmGray = rgb2gray(imSm); • >> imSmGray = im2double(imSmGray); • >>figure, imshow(imSmGray) • >>figure, imshow(im2bw(imSmGray,graythresh(imSmGray))) Roger S. Gaborski 50