1. Using Computer Vision to Test Web Display Xu Liu liuxu@cs.umd.edu

2. Why test web display? • Display Bugs Bug “Opera 7.54” Normal IE6

3. More bugs BUG - Opera Normal - Firefox Normal - Mozilla Normal - IE

4. Where comes these bugs? • Different parsers on DHTML, CSS • Web designer doesn’t follow W3C standard • IE tolerant buggy HTML • Java Script, ActiveX, Flash

5. Which kinds of bugs do we have? • Text/Image Overlap • Incorrect blank area • Missing Text/Image Generally they are all layout problems

6. How do we detect these bugs • Is HTML source helpful? • Yes, but we need a correct parser which is being tested • HTML does not have straight forward relationship with display • Eyes always tell the truth – directly use the image of snapshot

7. Find the outlier • Let them vote: • IE, Firefox, Mozilla, Netscapte, Opera, MyIE… • Anyone seems distinct from others is probably an outlier • Assumption : Major are correct, Minor are incorrect

8. Basic Question • How do we compare 2 images? Are they same? Missing Front and end These 2 look different but they should be considered the same

9. Simpler Question • How do we compare 2 sequences? S1: 1 2 3 4 5 6 8 7 S2: 1 2 4 5 5 5 6 7 Direct Compare |S1-S2|=0+0+1+1+0+1+2+0=5 Dynamic Time Warping !! In fact ||S1-S2||=0, they should be considered the same

10. Dynamic Time Warping (DTW) • A widely used technique in signal processing • Speech recognition, Image matching Compare S1,S2

11. Can we directly apply DTW to compare web pages? • No! If we directly compare 2 pages: Most of the error will be omitted Local VS Global Segmentation First!!

12. A Segmented Page

13. How do we derive segmentations Edge Detection First Over Segment Merge

14. Next • For every page • We have all its segments • For every segment • If it CANNOT be found in other pages, possibly it is an error

15. Result http://www.microsoft.com/smallbusiness/default.mspx by Opera 7.54

16. Result http://www.microsoft.com/learning/default.asp by Opera 7.54

17. Shortage and Future Work • Make segmentation more accurate • Make the system completely automatic