Integrating Alice and Media Computation: Creative Approaches to Image and Sound Manipulation

1. Alice and Media Computation:Goals for this session • Show interesting things you can do combining Alice and Media Computation. • Provides a story-telling motivation to explore image and sound manipulation • To see how one integrates Media Computation images and sounds into Alice. • In slide set: • A brief Java example of Chromakey • A set of Python utilities to make simple image and sound manipulation easier. • To focus more on Alice and less on textual programming, to start.

2. “Can’t I do everything in Alice?” • Why do programming outside of Alice? • Sometimes you want special images or sounds in your Alice story. • You can make them by programming those images and sounds, then bringing them into Alice. • Sometimes you want to add in special movie effects to your Alice movie. • Overall, programming outside of Alice is the special effects studio for Alice.

3. Sound Frequency Project • Create an Alice world with 3 frogs • Resize one to be bigger and one to be smaller • Using croak.wav • Create a higher sound • Take every other sample • Create a lower sound • Take every sample twice • Import the sound files and have the frogs play their sounds and tap their foot Alice and Media Computation Intro

4. Using Chromakey • Create objects in Alice and position them as desired • Delete the ground • Set the color of the background to green • in the world properties • Use a conditional to replace the green color • You may also need to scale one of the pictures Alice and Media Computation Intro

5. Drawing Projects • Draw something on an Alice picture • Like a hat, glasses, or a scarf • Fireworks • Create images in Media Computation that you use as textures in Alice • like a gradient for the ground • Clip an Alice picture • To look like a porthole • Create a poster advertisement for your Alice movie. • With your name on it • Create a comic panel with Alice pictures Alice and Media Computation Intro

6. Comic Strip Project • Create several scenes in Alice and take pictures of the scenes • Use Media Computation to create a Comic Panel and/or Comic Strips • Draw different types of text balloons • Speech balloon, thought balloon, scream balloon Alice and Media Computation Intro

7. Add Yourself to an Alice Movie Alice and Media Computation Intro

8. Add Alice Characters to a Real Movie • Split out frames from a movie of people • Capture frames from an Alice movie using MovieCapturePanel • Use chromakey to copy the Alice items to the real movie Alice and Media Computation Intro

9. How to use image manipulation with Alice • A. Create an image to appear as a Billboard in an Alice world. • B. Create images in Alice that you can manipulate as snapshots. • Like with chromakey • C. Make movies in Alice that you can then manipulate as video.

10. A. Loading images as billboards in Alice • Step 1: Make a cool image that you want to appear as an object in your Alice world. • Step 2: Import the image as a Billboard.

11. Step 3: Move it around and use it like an object

12. B. Manipulating an Alice world snapshot • Step 1: Set up an Alice world the way you want it. • If you want to chromakey, • drag the “Ground” into the trash • make the “Atmosphere Color” of the world blue (default) or green.

13. Creating the snapshot • Step 2: Program your objects to move where you want them to go. • Step 3: Play your world, and at the right time, hit “Pause” and “Take Picture”

14. Snapshot from Tutorial,Moved to the Moon >>> capturedbirds = makePicture( getMediaPath("capture00.jpeg")) >>> capturedbirds Picture, filename W:\mediasources\capture00.jpeg height 360 width 666 >>> birds = scale(capturedbirds, 0.75) >>> birds Picture, filename None height 271 width 500 >>> chromakeyBlue(birds,moon)

15. C. Manipulating an Alice videofrom Python • Step 1. Tell Alice not to remove frames after making a video! (Change Preferences.)Only have to dothis once.

16. Record your video • Step 2: From File menu, choose “Export Video.” • Click “Record” then “Play” • When done, “Stop Recording” and “Export Video.”

17. Your frames are now saved

18. Doing Chromakey to a Set of Frames import os def chromakeySet(frames,background): newdir = "C:/temp/" for file in os.listdir(frames): if file.endswith(".jpg"): printNow(file) frame = makePicture(frames+file) chromakeyBlue(frame,background) writePictureTo(frame,newdir+file)

19. Using Chromakey to put Penguin in the jungle >>> jungle = makePicture( getMediaPath("jungle.jpg")) >>> chromakeySet( "W:/mediasources/frames/", jungle)

20. Using newly created sounds in Alice

21. Making your characters play your sound

22. Choosing to import a sound

23. All set!

24. A Java-based example • Doing Chromakey with an Alice-generated image, from Java, to demonstrate the analagous methods.

25. Get a picture from Alice for Chromakey from Java • Click on Edit and then on Preferences and then on the Screen Grab tab • Change image format to png (has loseless compression) • Set up the scene in Alice • Add the objects and position them as desired ChromakeyAlice

26. Get a picture from Alice for Chromakey • Right click on ground in the object tree • And then select delete • If you need to change the sky (atmosphere) color • Click on world in the object tree • Then click on properties in the detail window • Change the atmosphereColor to green ChromakeyAlice

27. Save the picture to a file • Click on the Play button in Alice • Then click on the Pause button to stop the action when you want to take a picture • Click on the Take Picture button to save a picture • Usually on the desktop as captureXX.png ChromakeyAlice

28. Chromakey Method /** * Method to do chromakey using the passed background * color and the distance to the color * @param newBg the new background image to use to replace * @param color the background color to compare to * @param dist the distance that limits the chromakey * it will happen if the distance is less than or equal * to this value */ public void chromakey(Picture newBg, Color color, double dist) { Pixel currPixel = null; Pixel newPixel = null; ChromakeyAlice

29. Chromakey Method - cont // loop through the columns for (int x=0; x<getWidth(); x++) { // loop through the rows for (int y=0; y<getHeight(); y++) { // get the current pixel currPixel = this.getPixel(x,y); /* if the color at the current pixel is within the passed distance of the * chromakey color then change the background */ double currDist = currPixel.colorDistance(color); ChromakeyAlice

30. Chromkey Method - cont if (currDist <= dist) { newPixel = newBg.getPixel(x,y); currPixel.setColor(newPixel.getColor()); } } } } ChromakeyAlice

31. Chromakey onto current picture /** * Method to do chromakey by copying the pixels that have a larger distance * to the passed source color than the passed distance. * @param sourcePict the picture with the background color * @param sourceColor the background color to compare to * @param dist the distance from the color to use * @param xSourceStart the starting X for the source picture * @param ySourceStart the starting Y for the source picture * @param xSourceEnd the ending X for the source picture * @param ySourceEnd the ending Y for the source picture * @param xTarget where to copy to in target (this) * @param yTarget where to copy to in target (this) */ ChromakeyAlice

32. Chromakey Method public void chromakey(Picture sourcePict, Color sourceColor, double dist, int xSourceStart, int ySourceStart, int xSourceEnd, int ySourceEnd, int xTarget, int yTarget) { Pixel currPixel = null; Pixel newPixel = null; // loop through the columns and rows for (int x=xSourceStart, tX = xTarget; x<xSourceEnd; x++, tX++) { for (int y=ySourceStart, tY = yTarget; y<ySourceEnd; y++, tY++) { ChromakeyAlice

33. Chromakey - cont // get the current pixel currPixel = sourcePict.getPixel(x,y); /* if the color at the current pixel is greater than the * passed distance from the source color then copy it */ if (currPixel.colorDistance(sourceColor) > dist && tX < this.getWidth() && tY < this.getHeight()) { newPixel = this.getPixel(tX,tY); newPixel.setColor(currPixel.getColor()); } } } } ChromakeyAlice

34. Prepare for chromakey • Get the color that the background is • Blue = (73,124,204) • Figure out the boundary of what you want to copy • 0,0, width-5,height • Figure out where to copy to • 155,170 ChromakeyAlice

35. Chromakey Result Picture dino = new Picture(FileChooser.pickAFile()) Picture beach = new Picture("beach.jpg"); import java.awt.Color; beach.chromakey(dino,new Color(73, 124, 204),10,0,0, dino.getWidth()-5,dino.getHeight(),155,170) beach.explore(); ChromakeyAlice

36. Challenge • Try to use the chromakey method on an Alice picture • You can use one you have created • Or use one from the AlicePicts folder ChromakeyAlice

37. Introducing Image Manipulation for Alice • We’ll start with Python, introducing some very simple code for making it easy to do the image and sound manipulations for Alice. • Available in the python-book folder. • We’ll do one example in Java.

38. What color shirt? Where is it? >>> shirt = makePicture(pickAFile()) >>> explore(shirt) >>> print shirt Picture, filename C:\mediasources\IMG_0808.JPG height 480 width 640 Y=210 X=400 X=220

39. distance() function tells how close colors are >>> print white color r=255 g=255 b=255 >>> print black color r=0 g=0 b=0 >>> print distance(white,black) 441.6729559300637 >>> print red color r=255 g=0 b=0 >>> print pink color r=255 g=175 b=175 >>> print distance(red,pink) 247.48737341529164

40. Changing the whole shirt def changeAllShirt(picture): myGreen = makeColor(132,240,204) for pixel in getPixels(picture): color = getColor(pixel) if distance(color,myGreen) < 200: setColor(pixel,pink)

41. Results

42. Just doing part of the picture, and changing fewer pixels def changeShirt(picture): myGreen = makeColor(132,240,204) for x in range(220,400): for y in range(210,480): pixel = getPixel(picture,x,y) color = getColor(pixel) if distance(color,myGreen) < 50: setColor(pixel,pink)

43. Results

44. Coloring the shirt from a different picture >>> beach = makePicture(pickAFile()) >>> shirt = makePicture(pickAFile())

45. Swap a new background for a color def newBackgroundShirt(picture,background): myGreen = makeColor(132,240,204) for x in range(220,400): for y in range(210,480): pixel = getPixel(picture,x,y) color = getColor(pixel) if distance(color,myGreen) < 100: backgroundPixel = getPixel(background,x,y) backgroundColor = getColor(backgroundPixel) setColor(pixel,backgroundColor)

46. Results

47. How do we change the size of a picture? • To copy a picture, get the color values from pixels in one picture, and set those color values in the pixels in the other picture. def copyPicture(picture): returnPicture = makeEmptyPicture(getWidth(picture),getHeight(picture)) for pixel in getPixels(picture): color = getColor(pixel) setColor(getPixel(returnPicture,getX(pixel),getY(pixel)),returnPicture) return returnPicture

48. To scale the picture • smaller = scale(picture,0.5) • How does it work? • To get it to be only ½ the size (in both horizontal and vertical) directions, we need to lose some pixels. • Easiest way: Skip a few. Every other pixel gets copied. • Can generalize this for any size scaling.

49. Code to scale def scale(picture,factor): newHeight = int(factor*getHeight(picture))+1 newWidth = int(factor*getWidth(picture))+1 returnPic = makeEmptyPicture(int(newWidth),int(newHeight)) sx = 0 for tx in range(0,newWidth): sy = 0 for ty in range(0,newHeight): if (int(sx) < getWidth(picture)) and (int(sy) < getHeight(picture)): sp = getPixel(picture,int(sx),int(sy)) tp = getPixel(returnPic,tx,ty) setColor(tp,getColor(sp)) sy = sy + (1/factor) sx = sx + (1/factor) return returnPic

50. How do we grab part of a picture? guys = makePicture(getMediaPath("IMG_0805.JPG")) james = copyPartPicture(guys,352,217,618,475) • copyPartPicture(picture,startX,startY,endX,endY) • Gives you a new picture inside the box defined by upper-left corner and lower-right corner.