If you have any questions please feel free to interrupt me

1. If you have any questions please feel free to interrupt me

2. The vision system for Marie Curie

3. Main Tasks in our system Image recognition Machine Learning Control of robot’s behavior Environment

4. Marie Curie will be communicating with Schroedinger’s Cat robot

5. Interaction of Marie Curie and Cat • Marie Curie does not change its main coordinates, she can only move her hands, head and legs, but she remains attached to the desk. • Cat can move freely in the area of the stage. • Cat should not bump into Marie or furniture of the lab. • Marie should know where the cat is located and look to him • Cat should know where Marie is located and talk to her.

6. Ideal view of the ceiling camera Black curtain Cat shelf equipment equipment Marie Curie equipment

7. There will be a Kinect camera looking from the ceiling to the stage

8. The role of the ceiling camera • The camera will be attached to the ceiling or will be in some position very high, as high as we can. • We have done something similar but the robots were small. • The camera should know x,y coordinates of every robot and its orientation (pose) • Marie Curie does not change its main coordinates, so it is easy • Cat is fast so we have to track the triplet (x, y, )

9. There is nothing like that in Disneyland • All behaviors of robots in commercial theatres are strictly scripted, robots move on rails, they cannot make an error. • In our case we have interaction, improvisation, and robots are subject to noisy behaviors. • This task is more similar to robot soccer than to existing robot theatres in the world.

10. There will be another camera looking to faces of the audience We will call it the human-control camera or a front camera

11. The role of the front camera • The camera is attached to the wall near the glass window of the theatre, looking towards humans located in the corridor. • This camera will look at the audience • There are several goals of having this camera • Recognizing (x, y, ) of every person that looks at the performance (perhaps not more than 5). • Recognizing the emotion on the faces of these people. • Recognizing their gestures with hands and legs, full bodies and faces. • Use these data to control the behavior of the robots, songs selected, slides selected, lights and other effects.

12. There is nothing like that in Disneyland • All behaviors of robots in Disneyland are strictly scripted. • Rarely humans can change robots’ behaviors.

13. This is a new task for our team • Marek Perkowski has never done anything like this before • Perhaps nobody in the world has done something like this. • This is good as we are doing something new. • Hopefully we have done something similar and have a good experience from the past. • We were doing ROBOT SOCCER. • We will try now to use our past experience and theory for a new task.

14. Ideal view of the ceiling camera Black curtain Cat shelf equipment equipment Marie Curie equipment

15. This is ideal, in reality the image will be much distorted with noise and lightning and geometry Y axis Black curtain c Yc Cat Xc shelf equipment equipment Marie Curie equipment X axis

16. The idea of Robot Soccer

17. 3. Robot Soccer and Similar Tasks • Robot Soccer Competition – RoboCup – FIRA – Remote controlled systems – Autonomous robots • Clustering

18. 3.1 Robot Soccer “RoboCup is an international joint project to promote AI, robotics, and related fields. It is an attempt to foster AI and intelligent robotics research by providing a standard problem where a wide range of technologies can be integrated and examined. RoboCup chose to use the soccer game as a central topic of research, aiming at innovations to be applied for socially significant problems and industries. The ultimate goal of the RoboCup project is: By 2050, develop a team of fully autonomous humanoid robots that can win against the human world champion team in soccer.” [RoboCup 1998]

20. As you see, it is difficult to approximate every robot with a rectangle. It will be even more difficult in our case.

22. Overhead Vision • Our goal is to start with Overhead Vision (Ceiling Camera) and check how it will work. • We may move to more cameras if necessary.

23. Local Vision

24. Design Criteria for robot soccer • Controller Hardware: Enable on-board image processing – Interface to digital camera – Incorporate graphics LCD – Incorporate user buttons – Wireless communication between robots • Sensors:Allow variety of additional sensors: – Shaft encoders – Infra-red distance measurement sensors – Compass module • Software: Flexibility to accommodate for different robot equipment – Operating system RoBIOS – Hardware description table HDT

25. What is AI? Research in Artificial Intelligence (AI) includes: • design of intelligent machines • formalization of the notions of intelligence and rational behavior • understanding mechanisms of intelligence • interaction of humans and intelligent machines.

26. Objectives of AI • Engineering : costruct intelligent machines • Scientific :understand what is intelligence.

27. Can a robot do these? • Understand? • Simulate its environment? • Act rationally? • Collaborate and compete? • Display emotions? A bold claim: A team of Robots will beat the FIFA World Cup champions by 2050!

28. RoboCup - Aim • ”pushing the state-of-the-art” • ”By mid-21st century, a team of fully autonomous humanoid robot soccer players shall win the soccer game, comply with the official rule of the FIFA, against the winner of the most recent World Cup. • TO BOLDLY GO WHERE MAN HAS GONE BEFORE (cf. Star Trek) • Formalised Testbed

29. Do you really believe that a team of Robots could beat the FIFA World Cup champions by 2050? • By all accounts this may sound overly ambitious. • In fact, if you compare this goal to other ground breaking achievements it is not ambitious at all. • The Wright brothers' first airplane was launched and 50 years later man landed on the moon. • Even more recently Deep Blue the computer programmed to play chess, played chess grand master Garry Kasparov and won -- roughly 50 years after the deployment of the first computer. • It's a long time. • Think what has happened since 1950.

30. Power of AI In 1997 a computer, Deep Blue, won a chess match with world champion Kasparov. • Accident? • IBM paid Kasparov to loose? • Brute force with no intelligence? • So, what is intelligence? Is the following AI?

31. Simulation Turing test (1950)

32. Chess versus soccer robot Chess RoboCup Environment Static Dynamic State Change Turn taking Real time Info. accessibility Complete Incomplete Sensor Readings Symbolic Non-symbolic Control Central Distributed Difference of domain characteristics between computer chess and soccer robots

33. Intelligent Agents • Agents are situated • Perception of environment • Execution of actions • Agents can communicate and collaborate • they can differ • than can compete and be more or less egoistic/altruistic • The agents have: • objectives, • communications, • intentions.

34. Professor Kim from KAIST A New Approach The founder of Robot Soccer and FIRA president Two organizations: 1. FIRA (earlier) 2. RoboCup (larger)

35. Four Blocks in two PCBs (Printed Circuit Boards) • Micro-controller (upper PCB) • Communication module (upper PCB) • Motor and driving circuits (lower PCB) • Power (lower PCB) side view front view top view

36. Importance of Robot Soccer • Communication • Cooperation • Coordination • Learning • Competence • Real Time Robot Soccer Evolution • Computer simulations • Wheeled brainless robots • Wheeled autonomous robots • Legged autonomous robots

37. Robot Soccer Purpose • “The number one goal of [robot soccer] is not winning or losing, but accumulating diverse technology.” • - Mr. Dao (Senior VP of Sony Corporation).

38. MiroSot FIRAcategory • 3 robots on 1 team • Size : 7.5cm * 7.5cm * 7.5cm • Ball : orange golf ball • Playground : black wooden rectangular playground • (150cm * 130cm * 5cm) • Vision : global vision system • (more than 2m above playground)

39. Experimental Setupof the Vision System • Control panel

40. NaroSot FIRAcategory • 5 robots on 1 team • Size : 4cm * 4cm * 5.5cm • Ball : orange table-tennis ball • Playground , Vision : same as Mirosot

41. KheperaSot FIRAcategory • 3 robots on 1 team • Ball : yellow tennis ball • Playground : green playground (105cm * 68cm * 20cm) • Robot : Khepera Robot • Vision : K213 Vision Turret

42. RoboSot FIRAcategory • 3 robots on 1 team • Size : 15cm * 15cm * 30cm • Ball : red roller-hockey ball • Playground : black wooden rectangular playground (220cm * 150cm * 30cm) • Vision :on the robot • Under preparation

43. Small-Size League

44. Small-Size League (F-180) • Field: 2.7 m x 1.5 m Size Area : 18cm rule (fit inside in 18cm diameter cylinder) Height : 15cm (global vision), 22.5cm (otherwise) • teams of autonomous small size robot play soccer game on a field equivalent to a ping-pong table. • Each team consists of 5 robots.

45. Small size league The field is the size and color of a Ping Pong table

46. orange golf ball • Robots move at speeds as high as 2 meters/second • Global vision is allowed

47. Robot Soccer Initiative Vision system Basic Architecture for Robot Soccer Systems Host computer Communication System Host computer Communication System Robots on the playing field “Brainless” System

49. Vision System • Vision : global vision system(more than 3m above ground) Each team has its own camera and PC

50. Small-Size League • 20 minutes, 2 breaks