Proxy-Based Image Verification

1. Proxy-Based Image Verification Thomas Kotwal tkotwal@mit.edu MIT Laboratory for Computer Science January 9, 2002

2. System Layout • Bob has access to an untrustworthy computer. • He wishes to communicate with his proxy. • He can trust what he sees on the screen, because his camera equipped K21 is monitoring the screen content.

3. Proxy-Based Approach • Add an IR link between the K21 and untrustworthy computer • This allows the K21 to send data (image) to the proxy • Proxy can do large amounts of computation • Proxy knows what the screen should display

4. U.C. displays an image and the K21 takes a picture • K21 sends picture + nonce + MAC to proxy via U.C. using IR link • Proxy verifies the picture and sends nonce + MAC to K21 via U.C. using IR link Security Protocol

5. User enters a request • U.C. sends the request to the proxy • Proxy sends an image summarizing the request to the U.C. • U.C. displays the image and the K21 takes a picture of it • K21 sends picture + nonce + MAC to the proxy using the IR link • Proxy verifies the picture and sends nonce + MAC to the K21 using the IR link Performing a Secure Request

6. Image Verification Problem Is this… …the same as this? • Steps to solution • Undo distortions • Compare content • Obstacles: • Linear and non-linear distortions • Decreased resolution • Noise

7. Step 1: Undo Image Distortion • Undo lens distortion • Model as radially symmetric distortion • Non-linear process • Undo linear distortions • Corrects for affine (scaling, rotation, translation) and projection distortion • Requires four known points in distorted image • Undo other non-linear distortions • Corrects for curvature of screen, etc. • Least-squares with orthogonal polynomials • May not be necessary

8. a) Radially Symmetric Distortion (x,y) (u,v) r k : constant

9. b) Linear Distortion (u,v) (x,y)

10. Step 2: Compare Content • Assume content is text only • Perform OCR on processed image • Advantage: proxy knows what the text should say • To save computation time compare each character with what it should be, not every possible character • Constrain font to facilitate OCR routine

11. Next Step: Compare Images • Generalize content to be anything, not just text • Find areas of large difference between original and processed image • Requires very accurate undistortion

12. Summary of Approach • K21 takes a picture and uses its IR link to send the picture to the proxy • Proxy corrects for image distortion and performs OCR SELL: IBM 300 SHARES TOTAL: $300 • Proxy sends a confirmation back to the K21

13. Why Proxy-based Approach? • Advantages • Proxy knows what the screen should display • No image processing on the K21 • No calibration necessary • Camera does not have to be immobile during session • Translates easily to color • Disadvantages • Requires IR link between K21 and untrustworthy computer • Possibly long verification time • Image verification allows for possibility of error

14. Summary / Status Q: How do you use a computer that you don’t trust? Our Answer: Use visual authentication • Pixel-Matching solution is running • Proxy-based solution is under development

15. Questions?