Using Lego Mindstorms NXT in the Classroom

1. Using Lego Mindstorms NXT in the Classroom Gabriel J. Ferrer Hendrix College ferrer@hendrix.edu http://ozark.hendrix.edu/~ferrer/

2. Outline • NXT capabilities • Software development options • Introductory programming projects • Advanced programming projects

3. Purchasing NXT Kits • Two options (same price; $250/kit) • Standard commercial kit • Lego Education kit • http://www.lego.com/eng/education/mindstorms/ • Advantages of education kit • Includes rechargeable battery ($50 value) • Plastic box superior to cardboard • Extra touch sensor (2 total) • Standard commercial kit • Includes NXT-G visual language

4. NXT Brick Features • 64K RAM, 256K Flash • 32-bit ARM7 microcontroller • 100 x 64 pixel LCD graphical display • Sound channel with 8-bit resolution • Bluetooth radio • Stores multiple programs • Programs selectable using buttons

5. Sensors and Motors • Four sensor ports • Sonar • Sound • Light • Touch • Three motor ports • Each motor includes rotation counter

6. Touch Sensors • Education kit includes two sensors • Much more robust than old RCX touch sensors

7. Light Sensor • Reports light intensity as percentage • Two modes • Active • Passive • Practical uses • Identify intensity on paper • Identify lit objects in dark room • Detect shadows

8. Sound Sensor • Analogous to light sensor • Reports intensity • Reputed to identify tones • I haven’t experimented with this • Practical uses • “Clap” to signal robot

9. Ultrasonic (Sonar) Sensor • Reports distances • Range: about 5 cm to 250 cm • In practice: • Longer distances result in more missed “pings” • Mostly reliable • Occasionally gets “stuck” • Moving to a new location helps in receiving a sonar “ping”

10. Motors • Configured in terms of percentage of available power • Built-in rotation sensors • 360 counts/rotation

11. Software development options • Onboard programs • RobotC • leJOS • NXC/NBC • Remote control • iCommand • NXT_Python

12. RobotC • Commercially supported • http://www.robotc.net/ • Not entirely free of bugs • Poor static type checking • Nice IDE • Custom firmware • Costly • $50 single license • $250/12 classroom computers

13. Example RobotC Program void forward() { motor[motorA] = 100; motor[motorB] = 100; } void spin() { motor[motorA] = 100; motor[motorB] = -100; }

14. Example RobotC Program task main() { SensorType[S4] = sensorSONAR; forward(); while(true) { if (SensorValue[S4] < 25) spin(); else forward(); } }

15. leJOS • Implementation of JVM for NXT • Reasonably functional • Threads • Some data structures • Garbage collection added (January 2008) • Eclipse plug-in just released (March 2008) • Custom firmware • Freely available • http://lejos.sourceforge.net/

16. Example leJOS Program sonar = newUltrasonicSensor(SensorPort.S4); Motor.A.forward(); Motor.B.forward(); while (true) { if (sonar.getDistance() < 25) { Motor.A.forward(); Motor.B.backward(); } else { Motor.A.forward(); Motor.B.forward(); } }

17. Event-driven Control in leJOS • The Behavior interface • boolean takeControl() • void action() • void suppress() • Arbitrator class • Constructor gets an array of Behavior objects • takeControl() checked for highest index first • start() method begins event loop

18. Event-driven example class Go implements Behavior { private Ultrasonic sonar = new Ultrasonic(SensorPort.S4); public boolean takeControl() { return sonar.getDistance() > 25; }

19. Event-driven example public void action() { Motor.A.forward(); Motor.B.forward(); } public void suppress() { Motor.A.stop(); Motor.B.stop(); } }

20. Event-driven example class Spin implements Behavior { private Ultrasonic sonar = new Ultrasonic(SensorPort.S4); public boolean takeControl() { return sonar.getDistance() <= 25; }

21. Event-driven example public void action() { Motor.A.forward(); Motor.B.backward(); } public void suppress() { Motor.A.stop(); Motor.B.stop(); } }

22. Event-driven example public class FindFreespace { public static void main(String[] a) { Behavior[] b = new Behavior[] {new Go(), new Stop()}; Arbitrator arb = new Arbitrator(b); arb.start(); } }

23. NXC/NBC • NBC (NXT Byte Codes) • Assembly-like language with libraries • http://bricxcc.sourceforge.net/nbc/ • NXC (Not eXactly C) • Built upon NBC • Successor to NQC project for RCX • Compatible with standard firmware • http://mindstorms.lego.com/Support/Updates/

24. iCommand • Java program runs on host computer • Controls NXT via Bluetooth • Same API as leJOS • Originally developed as an interim project while leJOS NXT was under development • http://lejos.sourceforge.net/ • Big problems with latency • Each Bluetooth transmission: 30 ms • Sonar alone requires three transmissions • Decent program: 1-2 Hz

25. NXT_Python • Remote control via Python • http://home.comcast.net/~dplau/nxt_python/ • Similar pros/cons with iCommand

26. Developing a Remote Control API • Bluetooth library for Java • http://code.google.com/p/bluecove/ • Opening a Bluetooth connection • Typical address: 00:16:53:02:e5:75 • Bluetooth URL • btspp://00165302e575:1; authenticate=false;encrypt=false

27. Opening the Connection import javax.microedition.io.*; import java.io.*; StreamConnection con = (StreamConnection) Connector.open(“btspp:…”); InputStream is = con.openInputStream(); OutputStream os = con.openOutputStream();

28. NXT Protocol • Key files to read from iCommand: • NXTCommand.java • NXTProtocol.java

29. An Interesting Possibility • Programmable cell phones with cameras are available • Camera-equipped cell phone could provide computer vision for the NXT

30. Introductory programming projects • Developed for a zero-prerequisite course • Most students are not CS majors • 4 hours per week • 2 meeting times • 2 hours each • Not much work outside of class • Lab reports • Essays

31. First Project (1) • Introduce motors • Drive with both motors forward for a fixed time • Drive with one motor to turn • Drive with opposing motors to spin • Introduce subroutines • Low-level motor commands get tiresome • Simple tasks • Program a path (using time delays) to drive through the doorway

32. First Project (2) • Introduce the touch sensor • if statements • Must touch the sensor at exactly the right time • while loops • Sensor is constantly monitored • Interesting problem • Students try to put code in the loop body • e.g. set the motor power on each iteration • Causes confusion rather than harm

33. First Project (3) • Combine infinite loops with conditionals • Enables programming of alternating behaviors • Front touch sensor hit => go backward • Back touch sensor hit => go forward

34. Second Project (1) • Physics of rotational motion • Introduction of the rotation sensors • Built into the motors • Balance wheel power • If left counts < right counts • Increase left wheel power • Race through obstacle course

35. Second Project (2) if (/* Write a condition to put here */) { nxtDisplayTextLine(2, "Drifting left"); } else if (/* Write a condition to put here */) { nxtDisplayTextLine(2, "Drifting right"); } else { nxtDisplayTextLine(2, "Not drifting"); }

36. Third Project • Pen-drawer • First project with an effector • Builds upon lessons from previous projects • Limitations of rotation sensors • Slippage problematic • Most helpful with a limit switch • Shapes (Square, Circle) • Word (“LEGO”) • Arguably excessive

37. Pen-Drawer Robot

38. Pen-Drawer Robot

39. Fourth Project (1) • Finding objects • Light sensor • Find a line • Sonar sensor • Find an object • Find freespace

40. Fourth Project (2) • Begin with following a line edge • Robot follows a circular track • Always turns right when track lost • Traversal is one-way • Alternative strategy • Robot scans both directions when track lost • Each pair of scans increases in size

41. Fourth Project (3) • Once scanning works, replace light sensor reading with sonar reading • Scan when distance is short • Finds freespace • Scan when distance is long • Follow a moving object

42. Light Sensor/Sonar Robot

43. Other Projects • “Theseus” • Store path (from line following) in an array • Backtrack when array fills • Robotic forklift • Finds, retrieves, delivers an object • Perimeter security robot • Implemented using RCX • 2 light sensors, 2 touch sensors • Wall-following robot • Build a rotating mount for the sonar

44. Robot Forklift

45. Gearing the motors

46. Advanced programming projects • From a 300-level AI course • Fuzzy logic • Reinforcement learning

47. Fuzzy Logic • Implement a fuzzy expert system for the robot to perform a task • Students given code for using fuzzy logic to balance wheel encoder counts • Students write fuzzy experts that: • Avoid an obstacle while wandering • Maintain a fixed distance from an object

48. Fuzzy Rules for Balancing Rotation Counts • Inference rules: • biasRight => leftSlow • biasLeft => rightSlow • biasNone => leftFast • biasNone => rightFast • Inference is trivial for this case • Fuzzy membership/defuzzification is more interesting

49. Fuzzy Membership Functions • Disparity = leftCount - rightCount • biasLeft is • 1.0 up to -100 • Decreases linearly down to 0.0 at 0 • biasRight is the reverse • biasNone is • 0.0 up to -50 • 1.0 at 0 • falls to 0.0 at 50

50. Defuzzification • Use representative values: • Slow = 0 • Fast = 100 • Left wheel: • (leftSlow * repSlow + leftFast * repFast) / (leftSlow + leftFast) • Right wheel is symmetric • Defuzzified values are motor power levels