1 / 37

370 likes | 444 Vues

. NEW COURSE: SKETCH RECOGNITION

Télécharger la présentation
## Learn how to make your drawings come alive…

**An Image/Link below is provided (as is) to download presentation**
Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.
Content is provided to you AS IS for your information and personal use only.
Download presentation by click this link.
While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.
During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

****NEW COURSE: SKETCH RECOGNITION Analysis, implementation, and comparison of sketch recognition algorithms, including feature-based, vision-based, geometry-based, and timing-based recognition algorithms; examination of methods to combine results from various algorithms to improve recognition using AI techniques, such as graphical models. Learn how to make your drawings come alive…**Sezgin**• Finds corners in a polygon or in a complex shape. • First: Polygon**Direction**Direction of each stroke segment = arctan2(dy,dx) Add check to make sure graph continuous (e.g., add 2pi)**Curvature**Change in direction for each segment**Speed**Speed of each segment (already computed in Rubine)**Threshold Curvature**• Threshold = mean curvature**Threshold Speed**• Threshold = 90% of mean**Select Vertices for Each**• Max of all sections above threshold • Fd = curvature points**Select Vertices for Each**• Max of all sections above threshold • Fs = speed points**Initial Fit**• Intersection of Fd and Fs (and of course the endpoints)**Local neighborhood: i-k, i+k**• Si is a stroke point • k is number of stroke point on either side to search • Le = Euclidian distance from Si-k to Si+k • K not defined • Lets try k = 3 … AND • Variable k where min(Le > 6 pixels)**Curvature Certainty Metric for Fd**• di = curvature at point i • l = stroke segment curve length from Si-k to Sj+k • Lets call it CCM(vi)**Speed Certainty Metric for Fs**• Lets call it SCM(vi)**Vertex Possibilities:**• Fd: Possibilities based on curvature • Fs: Possibilities based on speed • CCM(Fd) : Curvature Certainty Metric • SCM(Fd) : Speed Certainty Metric • H0 = Initial Hybrid Fit = intersection of Fd and Fs (and endpoints)**Selecting new points to add**• Hi’ = H0 + max(SCM(Fs – H0) ) • Hi’’ = H0 + max(CCM(Fd – H0)) • Pick highest scoring from each list • Try to add it to the H0**Make Line Segments for Hi’ and Hi’’**• For each vertex vi in Hi – create a list of line segments between vi and vi+1 • If there are n points in the vertex selection, there will be n-1 line segments**Calculate Errorfor Both Hi’ and Hi’’**• S = total stroke length (not Euclidean distance) • ODSQ = orthogonal distance squared • ODSQ(s,Hi) = distance of stroke point s to appropriate line segment (previous slide)**Distance from point to line segment**• if isPointOnLine(point, line) • theDistance = 0; • else • array1 = getLineAxByC(line); • array2 = getAxByCPerpendicularLine(line, point); • A = [array1(1), array1(2); array2(1), array2(2)]; • b = [array1(3); array2(3)]; • intersectsPoint = linsolve(A, b)'; • if isPointOnLine(intersectsPoint, line) • theDistance = getLineLength([intersectsPoint; point]); • else • dist(1) = getLineLength([line(1,:); point]); • dist(2) = getLineLength([line(2,:); point]); • theDistance = min(dist); • end • end**Select either Hi’ or Hi’’**• Pick the one with the lower error • (in the paper this is also the one with the `higher score’ – score in this sense is just an internal ranking metric) • H1 = Hi’ or Hi’’ (the one with the lower error) • We have added one vertex point.**Repeat**• Create new Hi’ and Hi’’ • Hi’ = H1 + max(SCM(Fs – H1) ) • Hi’’ = H1 + max(CCM(Fd – H1)) • (Note one of them will be the same as the previous time.) • Recompute error and continue**When to Stop**• Stop when error is below the threshold. • … what is threshold? • Not in paper • Lets • Compute the error for H0 = e0 • Compute the error for Hall (all the chosen v) = eall • We want something in the middle: close to eall • .1*(e0-eall) + eall • You will try other thresholds in your implementation**Implications**• Anything that you can describe geometrically you can build sketch system for**Curves**• Stroke between corners can be curve or line • Is line if l2/l1 is close to 1. • Lets use .9 < l2/l1 *1.1 < 1 • Else is curve**Sezgin Bezier Curve Fitting**• Want to replace with a Bezier curve • http://www.math.ubc.ca/people/faculty/cass/gfx/bezier.html • Bezier Demo • 2 endpoints and 2 control points**Bezier curve equation**• http://www.cl.cam.ac.uk/Teaching/2000/AGraphHCI/SMEG/node3.html • http://www.moshplant.com/direct-or/bezier/math.html • P0 (x0,y0) = p1, P1 = c1, P2 = c2, P3 = p2 • x(t) = axt3 + bxt2 + cxt + x0 • y(t) = ayt3 + byt2 + cyt + y0 • cx = 3 (x1 - x0)bx = 3 (x2 - x1) - cxax = x3 - x0 - cx - bx • cy = 3 (y1 - y0)by = 3 (y2 - y1) - cyay = y3 - y0 - cy - by**Sezgin Bezier Curve Fitting**• Control points • Endpoints: u = p1, v = p2 • T1= tangent vector – initial direction of stroke at point p1 • T2 = tangent vector of p2 – initial direction of stroke at point p2 • K = stroke length / 3 • 3 is common approximation • c1=k*t1 + v • c2 = k*t2 + u**Want to Test our Approximation**• Perhaps this is really a very complex curve which can’t be fit with a simple Bezier curve • E.g., the treble clef of a musical staff • Discretize the curve. (It doesn’t say into how many parts – I leave that up to you.) • Linear approximation for each part**If error too high**• Break our curve down the middle into two curves, and try again.**Matlab Curve Fitting**• function [estimates, model] = fitcurvedemo(xdata, ydata) • % Call fminsearch with a random starting point. • start_point = rand(1, 2); • model = @expfun; • estimates = fminsearch(model, start_point); • % expfun accepts curve parameters as inputs, and outputs sse, • % the sum of squares error for A * exp(-lambda * xdata) - ydata, • % and the FittedCurve. FMINSEARCH only needs sse, but we want to • % plot the FittedCurve at the end. • function [sse, FittedCurve] = expfun(params) • A = params(1); • lambda = params(2); • FittedCurve = A .* exp(-lambda * xdata); • ErrorVector = FittedCurve - ydata; • sse = sum(ErrorVector .^ 2); • end • end**Circles and Ellipses**• Least squares fit • Make a bounding box of stroke and form • Oval is formed from that bounding box. • Circle is easy: • Find out how far the circle is from the center = d, return (d-r)^2 • Ellipse more complicated**Project Suggestions**• Build a finite state machine recognizer for the computability class to easily draw and hand in their diagrams. • Build a physics drawing program that attaches to a design simulator (we have interactive physics 2005) • Build a fashion drawing program. You draw clothes on a person, and it puts them one the person.**Project Ideas**• Build a robot drawing and simulation program. You draw the robot and have a number of gestures to have it do different things • Gesture Tetris**Project Suggestions**• Use both rubine and geometrical methods in recognition • Develop new ways for editing. • Build new low level recognizers**Projects!**• Proposal due: Sept 22 • Visualization Contest • Ivc.tamu.edu/competition • Smart Boards coming • IAP (Industrial Affiliates Program) Demo**Projects!**• 2 types: • Cool application • Sketch front end to your own research system • Fun application to go on smart board/vis contest • Gesture Tetris • TAMU gesture-based map/directory info • Computability/Physics/EE/MechEng simulator • New recognition algorithm • Significant change to old techniques to make a new application**Final Project Handin**• Implementation… Build it… • In class Demonstration (5-10 minutes) • Previous work • Find at least 3 relevant papers (not read inc class) • Assign one for class to read, you lead short discussion • Test • Run your recognition system on data. • Find out what data you need (e.g., UML class diagrams) • Each student in class will supply others data • + Find 6 more people outside (to give 15 different people) • Paper • Introduction (why important) • Previous Work • Implementation • Results • Conclusion**Syllabus**• http://www.cs.tamu.edu/faculty/hammond/courses/SR/2006

More Related