U niversal R eal-time B ehavior I nterface

1. UniversalReal-timeBehaviorInterface URBI Tutorial Jean-Christophe Baillie ENSTA / Gostai Aldebaran Robotics

2. WHAT IS URBI?

3. URBI Key Features URBI is a complete solution to control robots. The base of the system is a new Interface Language. Simplicity Easy to understand, but with extremely advanced capabilities for demanding applications. URBI is used as well by Research Labs and by 12 years old kids as a hobby. Flexibility Independent of the robot, OS, platform, interfaced with many languages (C++, Java, Matlab…), Client/Server architecture. Modularity Software devices can be transparently plugged in the language to extend it, as internal new functions or external modules running on different computers (DOM: Distributed Object Management). Parallelism Parallel processing of commands, concurrent variable access policies, event based programming, task scheduling, … many new powerful concepts oriented towards parallel programming and Artificial Intelligence

4. URBI Engine URBI Module (plugin) Mac OSX, Intel or standard URBI modules (remote) URBI Engine Windows, Linux, … commands messages super calculator or, simply onboard

5. TECHNICAL PART Inside URBI

6. Objects to control hardware micro headPan legRF3 Every sensor, motor, camera or physical hardware in the robot is an object. micro gripL legR3 An URBI object is similar to a C++ object: it has methods and properties. The val property is related to the device value (telnet session, port 54000, with Aibo ERS7) headPan = 15; // or headPan.val = 15 headPan; [136901543:notag] 15.1030265089 accelX; [136901543:notag] 0.002938829104 camera; [145879854:notag] BIN 5347 jpeg 208 160 ############# 5347 bytes ########### speaker = bin 54112 wav 2 16000 16; ####### 54112 bytes #####

7. Messages and tags All messages from the server are composed with the same standard structure: headTilt; [136901543:notag] 15.1030265089 the command to the server: the message from the server: Time stamp (ms) Command tag Message content mytag: headTilt; [136901543:mytag] 15.1030265089 custom tag: useful to know who is sending what and to control running commands. Any command or group of commands can be prefixed by a tag. This is one of the most powerful features in URBI, crucial to handle parallelism properly

8. Advanced Tagging stop mytag; block mytag; unblock mytag; freeze mytag; unfreeze mytag; Stops the commands kills any new command with tag "mytag" freezes any running or new command stop / block / freeze mytag: { command1; if (distance < 50) sometag: command2; while (index < 10) { ping; index++; }; ... }; From another client or from other commands running in parallel

9. Parallelism Operators , and ; are also available and have a semantics identical to & and | except that they have looser constraints: gap A | B B.Start == A.end A ; B B.Start >= A.end A B A B A A A & B B.Start == A.start A , B B.Start >= A.start B B gap NB: Brackets can be used to group commands, like in C: { headPan = 15 | headTilt = 23 time:1000 } & neck = 10; Commands can be executed in serial or parallel mode: headPan = 15 & neck = 30; Set headPan to 15 and neck to 30 at the same time headPan = 15 | neck = 30; Set headPan to 15 and after, set neck to 30.

10. Complex Assignments 15 headPan = 15 time:5s; headPan = 15 speed:0.34; -2 5000ms 15 -2 speed = 0.34 unit/s 15 -2 accel = 0.02 unit/s² headPan = 15 accel:0.02; headPan = -2 sin:1s ampli:3, 6 units -2 Any function can be assigned as time parameterized trajectory with the function modifier (v2.0): 1000ms headPan = function(t):sqr(t)+sin(3*t+pi); Simple assignment: headPan = -2; Numerical assignments can be specified via “modifiers” This command never terminates put the command in background

11. Blending modes Blending modes: variable->blend = … variable "property" Conflicting assignments can occur from several clients or inside the same program. How to handle it? NB: this is also true for sound devices => simple multiplexer Each assignment occurs at the same time and is added to the others => Used to superimpose sinuses in Fourier decomposition neck.val->blend = add; neck.val->blend = mix; Like add, but they are averaged instead of added neck.val->blend = queue; Each assignement occurs only when the others are finished neck.val->blend = discard; Each conflicting assignment is ignored neck.val->blend = cancel; Each new assignment terminate any other pending assignment neck.val->blend = normal; The latest assignment has the focus, be the others run in background

12. Objects: OOP and broadcasting Usual OOP features are available: Usual OOP features are available (1.0 only) motor class motor { var val; function switchon(); event overheat; }; legRF1 = new motor("xx24"); legRF2 = new motor("xx27"); legRF3 = new motor("xx789"); // broadcast grouping group legRF { legRF1, legRF2, legRF3 }; legRF subclassing and multiple inheritance possible head legs tail legRF1 legRF2 legRF3 URBI multiple parallel launch going downward: broadcasting. function motor.switchon() { echo “on “+val; }; legRF.switchon(); broadcast switchon()  gets motor.switchon()  legRF1.switchon() & legRF2.switchon() & legRF3.switchon(); Usual virtual method search going upward function motor.switchon() { echo “on “+val; }; legRF2.switchon();gets motor.switchon

13. C-like Features Function definition Functions can be defined on devices or virtual devices: Control structures: Standard control structures are available and some more specific to URBI: function robot.walk (x,y) { … /* walk code*/ }; function add (x,y) { return x+y }; function fibo(n) { if (n<2) return 1 else { a = fibo(n-1); b = fibo(n-2); return a+b } }; // the classical for loop for (i=0;i<10;i++) echo i; // soft tests : must be true for 3ms while (headsensor > 0) { instructions… } // loop 10 times loopn (10) legLF1 = legRF1; // Funny average calculation with for& avg = 0; avg->blend = mix;for& (i=0;i<10;i++) avg = tab[i]; no semicolon usage: robot.walk (14,255); myresult = fibo(10); robot.process_mystring("bonjour");

14. Event catching Instruction A test = true; at (test) { at (test) { instructionsA; instructionsA; }; } onleave { instructionsB; }; whenever (test) { whenever (test) { instructionsA; instructionsA; }; } else { instructionsB; }; Instruction B test = false; Instruction A test = true; Instruction B test = false; Several event catching mechanisms are available: waituntil (test); Terminates only when test becomes true. => If given a number, the wait command pauses for this nb of ms. usage: waituntil (test) | instructions…

15. Event catching (2) timeout (time) command; //example timeout(10s) robot.walk(); command will be executed until time is over. This command runs in the background. stopif (test) command; //example stopif(headSensor) loop legRF1 = legLF1; command will be executed until the test becomes true. This command runs in the background. freezeif (test) command; //example freezeif(!ball.visible) balltracking(); command will be executed until the test becomes true, then it is freezed. When test becomes false again, it is unfreezed. Also runs in the background. You can control the lifespan of a command:

16. Event catching(3) emit myevent; emit myevent (1,"hello"); This creates an spiking event with or without parameters. at (myevent) ... whenever (myevent (x,y)) echo x+y; at (myevent (1,s)) echo s; ... You can catch events with a simple test. If there are parameters, you get them together with the event and you can filter on the base of those parameters value. emit(2s) myevent; emit() myevent (1,"hello"); every(2s) commands; every(2s) emit myevent; This will add a duration to then event. Possibly, no time limit. The every command starts the command at given time intervals. It can be used to create « pulsing events » You can emit your own events with the emit function:

17. Multicore Integration Hardware OS Automatic load balancing of parallel commands on a variable number of cores and threads, with real-time scheduling capabilities (currently in development, v.2 only) Physical Thread Core 1 Physical Thread URBI Kernel URBI Scheduler URBI Code Client1 Physical Thread Core 2 URBI Code Client2 Physical Thread Network Layer Physical Thread URBI Code Client3 … URBI Commands = Micro-threads (logical thread) Physical Thread Core 3 Synchronisation

18. Advanced Features Important files: URBI.INI CLIENT.INI URBI.INI is executed at start and usualy contains useful device groupings, aliases and an anim sequence. CLIENT.INI is executed when a new client connects to the robot and usualy contains a welcome message or a battery monitor. Many other advanced features: Interconnection commands, variables by name, aliases, load/save files, safe checking, debugging options,… None of these are required to move a joint! joint = 30; URBI can be used by non experts as well

19. Examples camera ball detection module headPan legR3 // Ball tracking program: whenever (ball.visible) { headPan = headPan + camera.xfov * ball.x & headTilt = headTilt + camera.yfov * ball.y }; // Get up on the Aibo getup: { { leg2 = 90 time:2s & leg3 = 0 time:2s } | leg1 = 90 time:1s | leg2 = 10 time:1s | { leg1 = -10 time:2s & leg3 = 90 time:2s } }; // Event detection at (headSensor ~ 2s) speaker.play("hello.wav"); at (distance < 40) emit collision; ...

20. Walk Sequence Simple walk: Using an original walk from the robocup, we extracted the two main Fourier coefficients for each joints. With blend = add mode, these two components are added on each joint to reproduce the original periodic oscillation: function robot.walk(duration) { echo "go for " + string(duration) + " secs"; direction = 1; if (duration <0) { duration = - duration, direction = -1 }; walk: timeout(duration) { for &(x=1;x<=2;x++) for &(y=1;y<=2;y++) for &(j=1;j<=3;j++) for &(d=1;d<=2;d++) robot.leg[x][y][j] = walk.mean[x][j] sin:walk.speed*walk.coef[d] ampli:walk.amp[x][j][d]*4 phase:direction*walk.phase[x][y][j][d]+pi*(direction-1)/2 } };

21. Behavior example speaker = lost; ball.visible == false Track ball Search ball ball.visible == true speaker = found; This example shows how to write behavior graphs with URBI: // Tracking state function tracking() { whenever (ball.visible) { headPan = headPan + camera.xfov * ball.x & headTilt = headTilt + camera.yfov * ball.y } }; // Searching state function searching() { period = 10s; { headPan’n = 0.5 smooth:1s & headTilt’n = 1 smooth:1s } | { headPan’n = 0.5 sin:period ampli:0.5 & headTilt’n = 0.5 cos:period ampli:0.5 } }; // Transitions track_transition: at (ball.visible ~ 400ms) { stop search; speaker = found; track: tracking(); }; search_transition: at (!ball.visible ~ 400ms) { stop track; speaker = lost; search: searching(); };

22. Finite State Machines (another way to do behavior graphs) freeze state2.tag2; function state2.init() { emit() in_state2; state2.main: loop { ... } }; state2.tag1: at (in_state2 && cond1) { action1 | state4: state4.init() ; stop state2 }; state2.tag2: at (in_state2 && cond2) { action2 | state3: state3.init() ; stop state2 }; Separated function state2.init() { state2.tag1: at (cond1) { action1|emit go_state4; stop state2}; state2.tag2: at (cond2) { action2|emit go_state3; stop state2}; state2.main: loop { ... }; }; at (go_state2) state2: state2.init(); at (go_state4) state4: state4.init(); Integrated state 2 cond3 Action3 cond1 state 1 Action1 cond2 Action2 state 3 state 4 cond4 Action4 state 2 Local event gate cond1 Action1 cond2 Action2

23. How to use URBI? telnet or urbilab client headPan.val = 15; headPan.val; [136901543:notag] 15.1030265089 ... C++ client // C++ code with liburbi C++ main() { UClient * client = new UClient("myrobot.ensta.fr"); int pos; pos = complex_calculation(x,y); client->send(“headPan.val = %d;”,pos); } Java client // Java code with liburbi Java import liburbi.UClient; robotC = new UClient(robotname); robotC.send("motor on;"); robotC.setCallback(image, "cam"); URBI.INI onboard scripts URBI Server • simple commands • functions definition • complex scripts Remote/Plugged C++ Module other integrated clients (matlab, python, . . .) // C++ object inherit fro UObject UStart(ball); class ball : UObject { ball(string); UVar x,y; . . . };

24. UObject Architecture ball.cpp ball.h #include "ball.h" UStart(ball); ball::ball(string s) : UObject(s) { UAttachVar (ball,x); UAttachVar (ball,y); UAttachFunction (ball,init); UAttachFunction (ball,getSize); UNotifyChange("camera.val", &ball::newImage); UNotifyChange(x); UNotifyChange(y, &newY); x = 0; y = 0; int tmp = (int)x; } // example of callback int ball::newImage(UVar& img) { char* data = ((UBinary)img).image.data); ... } #include "uobject.h" using namespace urbi; class ball : public UObject { public: ball(string); // exported field UVar x; UVar y; // exported methods void init(string color); float getSize(); // callbacks int newImage(UVar&); }; // external callbacks int newY(UVar&); ball detection module On the URBI side: ball_red = new ball("red");

25. Contact : Jean-Christophe Baillie, contact@urbiforge.com at (talk.finished == true) echo « Thank you for » + « your attention »;