Interactive Graphics and Sound in MATLAB Using PTB: Fundamentals and Tasks

1. Matlab Class 6Xiaotao Su, Ph.D. Visual Psychophysicist & IT Group Leader &Hristiyan Kourtev Rutgers Center for Cognitive Science (RuCCS)

2. Displaying images on screen w/ PTB • img = imread(‘winter.jpg’, ‘jpg’); • You draw an image just like you would a rectangle: • Screen(window, ‘PutImage’, img, [0, 0, 200, 200]);

3. Image info and structures • img_info = imfinfo(‘winter.jpg’) • img_info.Width will be the width of the image • screen(window, ‘PutImage’, img, [0, 0, img_info.Width, img_info.Height]);

4. Making/Playing sounds w/ PTB • A sound is a 2 x N matrix where N is the number of bits that make up the sound file and the two channels are for left and right • [sound, samplerate, samplesize] = wavread(‘feedback.wav’); sound = sound’; Snd(‘Play’, sound, samplerate, samplesize); Tip: To make your own wav files I recommend using an application called audacity http://audacity.sourceforge.net/

5. Getting Input from the Mouse • [mouse_x, mouse_y, buttons] = GetMouse(window_ptr); • Hide and show the windows cursor during a simulation:HideCursor, ShowCursor • If sum(buttons)>0, a button has been clicked

6. Task 1 – Mousing Around • Draw a green circle of radius 30 pixels, whose location on the screen is controlled by the mouse • Change the color of the circle to red when a button is clicked and back to green when released Hints: You will need to use the following functions • Draw a circle:Screen(window, 'FillOval', dot_color, circle_rect); • The position of the cursor marks the center of the circle and the rectangle it is inscribed in.

7. while(toc<5) • [mouse_x, mouse_y, buttons] = GetMouse; • if(sum(buttons)>0) • dot_color = red; • else • dot_color = green; • end • cursor = [ mouse_x-dot_r, mouse_y-dot_r, ... • mouse_x+dot_r, mouse_y+dot_r]; • Screen(window_ptr, 'FillOval', dot_color, cursor); • Screen('Flip', window_ptr); • end • clear Screen; • ShowCursor; • catch • clear Screen; • ShowCursor; • end % mousingAround.m clear all; try which_screen=0; bg_color = [0, 0, 0]; [window_ptr, screen_dimensions]= … Screen(which_screen, … 'OpenWindow', bg_color); dot_r = 20; %radius of cursor HideCursor; green = [0, 255, 0]; red = [255,0, 0]; tic;

8. Direction/Velocity Vectors • A vector has both magnitude and direction • Direction = [x,y] • Magnitude = v = √(x2+y2) = sqrt(x^2 + y^2) % Pythagorean x v y x

9. Detecting Collisions With Borders rect1 = [10, 10, 50, 50]; % [x1, y1, x2, y2] screen_dimensions = [0, 0, 1280, 1024]; • Collision w/ top border if: rect1(2) <= screen_dimensions(2) • Collision w/ left border if: rect1(1) <= screen_dimensions(1) • Collision w/ bottom border if: rect1(4) >= screen_dimensions(4) • Collision w/ right border if: rect1(3) >= screen_dimensions(2)

10. Task 2 – Bouncing off the walls • Modify your existing program to add 2 blue squares that start off in a random direction at speed 5 and 7 pixels respectively and bounce off the walls Hints: • Will need to use the Pythagorean theorem to calculate the direction vectors making sure the magnitude is as specified • If a square bumps into the left or right walls, invert (multiply by -1) the x component of its velocity vector • If a square bumps into the top or bottom walls, invert (multiply by -1) the y component of its velocity vector

11. Task 3 – Click, Click Modify your program to: • Add a circular cursor controlled by the mouse • Make the squares clickable. They should change color to purple when clicked. Hint: Purple = [255,0,255]; Hint: The cursor is inside the square if: if((mouse_x>shape_rect(1))&... (mouse_x<shape_rect(3))&... (mouse_y>shape_rect(2))&... (mouse_y<shape_rect(4))) shape_color = purple; end