Lecture 3 Input and interaction

1. Lecture 3Input and interaction • Interaction • Input devices • Event-driven input • Display lists • Menus • Picking • Buffering • A simple paint program

2. 1963 Sketchpad • The first system to explore pen-based user-interfaces • by Ivan Southerland (of Evans and …) • broached field of HCI • not graphics but important • OpenGL doesn’t support interaction directly • limits portability • not specifically graphics • we’ll use GLUT for device interactions

3. Input devices • Physical vs. logical • Physical • mouse, keyboard, etc. • perspective of how they interact with system • Logical • function • perspective of what they send application • higher-level

4. Input devices • Logical examples • Output: when we use printf in C, the actual string may be output to a monitor or printer • Input: we get the position of a user’s cursor whether we use a mouse or a data tablet • Our application doesn’t care at high level • Two categories, pointer and keyboard • pointers may be mice, pens, trackballs, etc. • pointers do not return character codes • can return relative or absolute positions

5. Input devices • Absolute-positioning (w.r.t. some fixed position) • pen returns position w.r.t. tablet (settable) • data glove always returns 3D position coords • Relative-positioning (w.r.t. current position) • mouse pos. always begins where cursor is • Trigger & measure • trigger: physical input on device (button, key) • measure: what device returns to application (button, state, x position, y position)

6. Input modes • Request & sample modes • request_locator (device_id, &measure); • after trigger, wait until input received • ex. get x,y pos. after right mouse button depressed • sample_locator (device_id, &measure); • assumes trigger, get input from a queue • ex. get current x,y position of mouse • limits user control of program • which device to sample from? • modal, user must specify context (not natural)

7. Input modes • Event mode - the most flexible input mode • triggers generate events (events store data) • measures are sent to 1)an event queue program checks queue events are examined and acted upon 2)a special-purpose function program associates function with trigger up front function is called with the event trigger generates the function is called a callback

8. Event-driven input • Callback for display • glutDisplayFunc(display) • Callback for mouse • glutMouseFunc(mouse) • void mouse (int btn, int state, int x, int y) { if (btn == GLUT_LEFT_BUTTON && state == GLUT_DOWN) drawSquare(x,y); if (btn == GLUT_MIDDLE_BUTTON && state == GLUT_DOWN) exit (); }

9. Account for different origins (90, 100) (90, 400) 500 500 Event-driven input • drawSquare() function • use new mouse position to draw new square drawSquare(int x, int y) { y=wh-y; glColor3ub(r, g, b); glBegin(GL_POLYGON) glVertex(x+size, y+size); glVertex(x-size, y+size); glVertex(x-size, y-size); glVertex(x+size, y-size); glEnd(); glFlush(); } What are we doing here?

10. Event-driven input • Callback for keyboard • glutKeyboardFunc(keyboard); • Callback for display • glutDisplayFunc(display); • call in indirectly with glutPostRedisplay(); • after changing parameters, attributes, etc. • if nothing has changed, system will avoid drawing • Callback for idle • glutIdleFunc(idle) • perform tasks when nothing else is happening

11. Event-driven input • Callback for window resize event • glutReshapeFunc(reshape) • What do we do in reshape(width, height)? • What if new window is bigger or smaller? Do we grow, shrink, or resize the image? • What if aspect ratio has changed? Do we similarly resize the image, distort it, or clip it out? • Use the new width and height to • adjust clip rectangle and viewport sizes gluOrtho2D (0.0, width, 0.0, height); glViewport (0, 0, width, height);

12. Clients and servers • In general • Servers perform tasks for clients • In OpenGL • assumes the OpenGL program is the client • assumes workstations are graphics servers that provide display and input services to the OpenGL program • for example, the client can display its output on any networked server

13. Analytically-computed reflectionArvo (1995)

14. Display lists • History • refreshing the screen required special processing via a display processor that output a display list that was stored display memory • the display list was rendered fast enough to avoid flicker, leaving the host computer free • there is no longer any distinction between host and display processor (unless we count special graphics processing hardware)

15. Display lists • In immediate-mode, • we compute, rasterize, and display (gasket.c) • we re-compute to redisplay • In retained-mode, • we store the results of our computation in a display-list • we send an instruction to redisplay • disadvantages • require memory on server • require overhead to create

16. Display list example • #define Box 1 • glNewList(Box, GL_COMPILE) • glBegin(GL_POLYGON) • glColor3f(1.0, 0.0, 0.0); • glVertex2f(-1.0, -1.0); • glVertex2f(1.0, -1.0); • glVertex2f(1.0, 1.0); • glVertex2f(-1.0, 1.0); • glEnd(GL_POLYGON); • glEndList();

17. Display list example • glMatrixMode(GL_PROJECTION); • for (i=1; i<5; i++) { • glLoadIdentity(); • gluOrtho2D(-2.0*i, 2.0*i, -2.0*i, 2.0*i); • glCallList(Box); • } • Clip rectangle results (left, right, bottom, top) • i=1: (-2.0, 2.0, -2.0, 2.0) • i=2: (-4.0, 4.0, -4.0, 4.0) • i=3: (-6.0, 6.0, -6.0, 6.0) What if we want to change the color each time through? What happens as clip rect grows? What if it was huge?

18. Display lists • Create multiple lists • glGenLists(number) generates multiple ids • glCallLists displays multiple lists

19. Menus • Glut provides pop-up windows • define entries • link menu to mouse button • define callback function for each menu entry

20. Menu example • Menu creation is order-dependent • /*Create a menu named sub_menu with callback named size_menu, with two entries*/ sub_menu = glutCreatMenu(size_menu); glutAddMenuEntry (“incr. square size”, 2); glutAddMenuEntry(“decr. square size”, 3); • /*Create a menu with callback named top_menu, with two entries*/ glutCreateMenu(top_menu); glutAddMenuEntry(“quit”, 1); glutAddSubMenu(“resize”, sub_menu); • /*Make right mouse button trigger menu event*/ glutAttachMenu(GLUT_RIGHT_BUTTON); incr. square size decr. square size quit resize incr. square size decr. square size

21. Menu example • You write the callback functions • /*top_menu callback checks for quit only*/ void top_menu (int id) { if (id == 1) exit(); } • /*sub_menu processes resize options */ void size_menu (int id) { if (id == 2) size = 2.0*size; else if (id == 3 && size > 1) size = size /2.0; else if (id == 3) printf(“At minimum square size”); glutPostRedisplay(); }

22. Picking • User selection of object on screen • In immediate mode, we compute, rasterize and display. How does this present a problem for identifying objects on screen? • we pick a pixel • how to work backward from pixel to object? • Several solutions • render ea. object to its own clip rect and viewport • for every position in frame buffer we add an obj id to a lookup table that is examined at each pick • for every object we compute a bounding rectangle or box (or extents)*

23. Picking • What if objects overlap? • return list of objects, not just one • What if cursor is near but not on object? • bounding box will accommodate some • also, you can leave slack in accuracy when you check position against object extents • What if pick is in a virtual 3D environment? • we will cast a ray into the scene and check each object for an intersection, returning list

24. Buffering • Single buffering • objects are always rendered into the same framebuffer, which is always being displayed • delay of clearing of and re-drawing into frame-buffer will cause flicker if re-draw takes longer than refresh or refresh and animation not synced • Double buffering • keep two buffers (front and back) • always display front, always render into back* • swap front and back when rendering complete

25. A simple paint program • Review the paint program in chapter 3 • (A.6, p 541) as a prelude to our project 2 • Code text is here. • Go over project 2.