Introduction to Processing and Serial Communication with Arduino

1. Danger Shield, Arduino and Processing Materials by Lindsay Craig, Dave Stillman and Ben Leduc-Mills MediaLive 2012

2. ProtoSnap and Processing Class Overview • What is Processing? Installation and Intro • Serial Comm from Arduino to Processing • Drawing a dot & controlling position • Drawing a circle & controlling grayscale • Drawing a square • Switching control using buttons • Additional Danger Shield material

3. Processing? • Processing is a free, open source, cross-platform programming language and environment for people who want to create images, animations, and interactions. • Created in 2001 by Casey Reas and Ben Fry at the MIT Media Lab. • Downloads, updates, reference, forums, etc. at: http://processing.org

4. Installation • For Linux: • Download the .tar.gz file to your home directory, then open a terminal window and type: • Tar xvfz processing-xxxx.tgz • (replace xxxx with the rest of the file’s name, which is the version number) • This will create a folder named processing-1.5 or something similar. Then change to that directory: • cd processing-xxxx • and run processing: • ./processing • For Mac: • Double-click the .dmg file and drag the Processing icon from inside this file to your applications folder, or any other location on your computer. Double click the Processing icon to start Processing. • For Windows: • Double-click the .zip file and drag the folder inside labeled Processing to a location on your hard drive. Double click the Processing icon to start Processing. • If you are stuck go to http://wiki.processing.org/index.php/Troubleshooting for help.

5. Anatomy of a sketch A sketch is a file or project you create in Processing. When you first open up a new sketch it will be completely blank.

7. Setup() This function runs once, at the very beginning of your sketch. You will use setup to set up certain aspects of your sketch, makes sense right? This is a good place to put your sensor calibration code. (More on sensor calibration later.) Most importantly for this class you will begin Serial communication in the setup function. The setup function without anything in it looks like this:

8. Draw() This function is where everything happens in your sketch. The draw loop is the portion of code that keeps repeating while the Processing sketch is open. The draw loop looks like this:

9. Basic Graphics we will be using point (x, y); background(color); ellipse (x, y, width, height); fill (red, green, blue, transparency); rect(x, y, width, height)

10. Reference • One very convenient way to access Processing’s Help Reference is to highlight a function or a word used in your code, right click and select Find in Reference (which brings you to the processing.org reference page):

11. Hooking into Processing We can send values from the Danger Shield to Processing through an Arduino, and use these values to control things in Processing First step: Open MediaLive.ino Like Processing, we have a setup() function In that function, we need to open Serial communication with this line of code: Serial.begin(9600);

12. Hooking into Processing Next we have to set up our pin definitions from the Danger Shield. There is a byte array to control our 7 segment display below this code (but it’s commented out, don’t worry about it).

13. Hooking into Processing We now have to set up all pins to the right pinMode, in our setup function as well as pulling any internal pull up resistors HIGH. The establishContact() function keeps the Arduino sketch from exiting the setup function until it hears from Processing and establishes contact.

14. Hooking into Processing Here’s the establishContact() function. It continually sends “Hello” out on the transmit serial pin. This travels to the computer. While the computer has not responded there is no data in the Serial buffer so Arduino just keeps sending “Hello”. Once the Arduino hears anything from the computer it exits establishContact() as well as the setup function and enters the Arduino draw function.

15. Hooking into Processing In our loop() function, we need to read from all the sensors and send the values out the serial port by calling Serial.write and Serial.print We’ll talk about the difference between these two commands as well as the start byte, delimiter and the end byte next.

16. Hooking into Processing The difference between Serial.write and Serial.print is that write is used to transmit byte type variables and print is used to transmit all other variable types.

17. Hooking into Processing Here are the rest of the loop() statements. Basically we are just sending more data about the other sensors on the Danger Shield. Then we send an end byte with write. Finally we send an empty println() command so Processing recieves a ‘\n’ and knows we have sent all the sensors values and will be starting over from the beginning.

18. Receiving Data in Processing • import processing.serial.*; • Serial usbPort; • int [ ] sensors = null; • boolean firstContact = false; • void setup() { • usbPort = new Serial (this,Serial.list( ) [0], 9600); • usbPort.bufferUntil (‘\n’); • } We’re sending data from the Danger Shield but we need a way to receive it in Processing. First, import the serial library. We’ll also need a Serial object to define which serial port we’ll be using, as well as an integer array for our sensor data. We also have a boolean variable to keep track of whether we’ve heard from Arduino or not. In your setup function, you need to initialize your Serial object, passing it the parent object (don’t worry about this) which port you want to use, and the baud rate. Make sure you pick the same baud rate that you defined in the Arduino sketch. The bufferUntil function just stores our incoming data in a buffer until we’re ready to do something with it.

19. Receiving Data in Processing Our next step is to define a SerialEvent function – this function automatically gets called every time a character in our bufferUntil() statement is read from the serial port. We then read a chunk of data into a String, trim whitespace, and split it using our delimiter character into our sensors[ ] integer array. This puts each sensor value into its own addressable place in the array. There is a println that should be printing out the sensors values it sees – try running the sketch to make sure you’re getting values. If you’re getting errors trying unplugging and plugging your Arduino back in. The code is on the next slide.

20. Receiving Data in Processing

21. Receiving Data in Processing There’s a lot going on here, so don’t worry if it doesn’t make sense at first. Basically, we check for Serial Communication from Arduino. Once it reads a carriage return it checks to make sure the data string is not empty. Then we print out the string to the console in Processing so we can see what data we are receiving.

22. Receiving Data in Processing Next there is some code to deal with the eventuality that we have not made contact with the Arduino yet. If we haven’t made contact it listens for a “Hello”. When it receives this it clears the Serial Buffer with usbPort.clear(), sets the variable firstContact true and sends an ‘A’ to the Arduino. This causes the Arduino to exit the establishContact() function and start sending data. After all that we print “contact” to the Processing console so that we can follow what is going on and we know we’ve made contact with the Arduino.

23. Receiving Data in Processing If firstContact is true we do the following code which splits the string up by our delimiter character into an array. This lets us put each sensor value into its own variable. Then there is a for loop that prints out the sensor values so we can see them. Then we assign the various values from the sensors array to individual variables. If you don’t put the array values into their own variables you will get a null pointer exception when you try to use them in your code.

24. Using Received Data in Processing So we’ve got all of the data we want placed in variables. Now we need to use them. In the draw loop we are calling the dotDrawer function and sending it the values of the slider 1 and 2 sensors.

25. Using Received Data in Processing Inside the dotDrawer function we have to use map to calibrate the slider values (which go up to 1023) so they correspond to X and Y values in the window created by Processing. Then we can use those values to draw a point in the window.

26. Using Received Data in Processing But wait! Why does the point draw itself over and over again? How can we fix this?

27. Using Received Data in Processing In order to fix this we will need to re-draw the background of the window over and over again before we draw the point. Here’s what that looks like:

28. Using Received Data in Processing Ok, that’s fixed but one pixel is kind of hard to see. Let’s put an ellipse in place of the pixel and let’s use the third slider to control the size of the ellipse.

29. Using Received Data in Processing Next we’ll use the light sensor and the button below the third slider in combination to control the color of the circle. First we need to add a “while” statement that checks to see if the button has been pushed.

30. Using Received Data in Processing Inside of the “while” statement’s curly brackets we need to use the photocell value to change the value of the circle by using the fill function. Unfortunately we still have to use the map function to make the sensor data fit the parameters wanted by fill.

31. Using Received Data in Processing It would be nice to see how the color is changing while you are setting it though, right? Add this to your “while” loop to see that: (You can just copy and paste it.) But you will need to move some other code around to actually make this work.

32. Using Received Data in Processing Here’s one way you could move the code around so it works: What is another?

33. What else? We’ve still got two buttons we haven’t used. Next we will use them to switch the circle between a circle and a square.

34. Making a new tab and function First thing we have to do is create a function that creates a sphere. To do this click on the arrow in the upper right corner of the Processing environment and select “New Tab”. Name the tab “squareDrawer”.

35. Making a new tab and function Here’s what we need to type to make our empty function named squareDrawer.

36. Making a new tab and function Let’s pass it the same variables that we sent to dotDrawer. (Copy and Paste is your friend.)

37. Making a sphere like the circle Now copy the following code from dotDrawer and paste it into your new function.

38. Making a sphere like the circle Now add the following code to actually display the square when the function is called.

39. Switching between the sphere and the circle Next we have to go back to our draw function and add code so the two buttons switch back and forth between the two functions. To do this first make a variable to keep track of which of the two left most buttons were pushed last. I named mine buttonPressed. It’s important to put it above the draw function.

40. Switching between the sphere and the circle At the beginning of your draw function add two “if” statements that check the first and second button and change the variable you just created to keep track of the last button that was pushed. (Make sure you don’t mix up the == and the =.)

41. Switching between the sphere and the circle Now simply add two more if statements that check the buttonPressed variable to decide which of your two functions should be called.

42. Switching between the sphere and the circle But what if we want both images to be displayed and we want to be able to switch back and forth between which one we are controlling? Well…. Then things get a little more complicated.

43. Switching between the sphere and the circle We need to make four different variables for each shape. We need to keep track of the x and y positions of the shape, the size and the grayscale. Create these variables above the draw loop.

44. Switching between the sphere and the circle Inside of each function assign the pertinent values to the variables. (dotDrawer function shown.)

45. Switching between the sphere and the circle These variables will only be reassigned when the code in the function executes.

46. Switching between the sphere and the circle Here is the squareDrawer function with the variable assignments. Feel free to add the code that controls the grayscale of the square.

47. Switching between the sphere and the circle Finally let’s add some code at the end of each of the shape’s functions that draws the other shape with the saved variables.

48. Switching between the sphere and the circle This will create an issue with your square’s color. Why and how would you fix this?

49. Switching between the sphere and the circle Finally, while we may not have time for it let’s talk about how we could make sure that the circle or square stays in the same place when we switch between their functions.

50. For more info on Processing and the Danger Shield go to: http://learn.sparkfun.com/curriculum/16 and http://learn.sparkfun.com/curriculum/1