Overview of Unmanned Aerial Vehicle Researches in UC, Berkeley

1. Overview of Unmanned Aerial Vehicle Researches in UC, Berkeley David H. Shim, Shankar Sastry Project Manager, Berkeley Aerobot Team University of California, Berkeley ACCLIMATE Review November 2, 2005 http://robotics.eecs.berkeley.edu/bear ACCLIMATE

2. Acknowledgements • Principal Investigator: • Professor Shankar Sastry, Dept of EECS, UC Berkeley • Project Manager: • David Hyunchul Shim, Ph.D. • Postdoctoral Researchers: • Jonathan Sprinkle, Ph. D. (embedded software) • Mike Eklund, Ph.D. (underwater vehicle, Boeing OCP) • Graduate Researchers: • Hoam Chung (model predictive control) • Todd Templeton (vision-based geographic reconstruction) • Jongho Lee (time-triggered RTOS) • Technical Support • Travis Pynn ACCLIMATE

3. Mission Statement • Develop a comprehensive set of theories and technologies for the joint operation of heterogeneous autonomous agents and human participants. • Implement scalable, effective, and reliable platforms for autonomous systems research • Validate the proposed ideas on the developed ACCLIMATE

4. Research Fields • Flight Control • system identification, feedback law design • Guidance • static/dynamic path planning • formation, collision avoidance, pursuit-evasion • Sensing Enhancement • vision-based landing, terrain map building, • Navigation: INS/GPS integration • System integration • Software development: OCP, Giotto • Communications ACCLIMATE

5. Exemplary Scenarios • Perch and Move (DARPA Seedling project, 2004) • Develop a set of methodologies to locate a radio signal source in a complex and changing environment using readily deployable unmanned aerial vehicles equipped with radio sensors Fields of research - Sensing and Navigation Collision avoidance Map building - Flight control automatic take-off landing site selection - Vehicle Development electric helicopter with payload sensors Operation in complex urban environment ACCLIMATE

6. Exemplary Scenarios • Convoy Protection (Fort Hunter Liggett, 2005) • Develop a set of UAVs that fly with the convoy, providing realtime surveillance video feed with various angles. The UAVs typically take off from a small pod and land on one of the convoy vehicle with high accuracy Fields of research - Sensing and Navigation Target detection/following Vision-based navigation Collision avoidance Map building - Flight control precision automatic take-off and landing ACCLIMATE

7. Berkeley UAV Platforms ACCLIMATE

8. Ursa Electra (July 2003~present) Fully autonomous electric helicopter First-hand test vehicle for advanced concepts Ursa Major 1 (Nov. 2002 ~) Low-cost, high-payload platform Aggressive Maneuver, Vision-based landing Multi-agent scenarios, Model-predictive control Ursa Magna 1,2 (June 1999~present) Advanced navigation&control algorithm development platform Multi-agent scenarios, formation flight, Vision-based landing Ursa Maxima (July 2000~present) High-payload platform for Multi-agent scenarios, formation flight, obstacle avoidance BEAR Fleet: rotorcrafts ACCLIMATE

9. BEAR Fleet: Ursa Magna1,2 now equipped with laser scanner in place of camera for OA experiment GPS Antenna Wavelan Antenna Integrated Nav/Comm Module Ultrasonic Height meter Length: 3.5m Body Width:0.7m Height: 1.08m Dry Weight: 44 kg Payload: 20kg Engine Output: 12 hp Rotor Diameter: 3.070m Flight time: 30 min Avionics operation time: 180 min Laser Rangefinder ACCLIMATE

10. BEAR Fleet: Ursa Maxima 1,2 Equipped with onboard alternator to power avionics and sensors GPS Antenna Laser Scanner on pan/tilt mount Wavelan Antenna Laser scanner control computer Flight Control System Length: 3.63m Body Width:0.72m Height: 1.08m Dry Weight: 58 kg Payload: 30kg Engine Output: 21 hp Rotor Diameter: 3.115m Flight time: 30 min Avionics operation time: 180 min ACCLIMATE

11. BEAR Fleet: Ursa Major1 Based on Bergen Industrial Twin Helicopter Twin gas engine Avionics Enclosure Length: 1.5m Body Width:0.7m Height: 0.7m Dry Weight: 7 kg Payload: 10kg Rotor Diameter: 2.0m Flight time: 30 min System operation time: 60 min onboard computer ACCLIMATE

12. BEAR Fleet: Ursa Electra 1 GPS Antenna Avionics Payload Device Flight Battery Ultrasonic sensors Length: 2.2 m Width: 0.26m Height: 0.41m Weight: 8.5 kg Rotor Diameter: 1.8m Flight time: 20 min (40V 8000mAh) System operation time: 90 min ACCLIMATE

13. BEAR Future UAV Testbeds Length: 1.2 m Width: 0.35m Height: 0.7m Weight: 8.2 kg Rotor Diameter: 1.5m Flight time: 20 min System operation time: 90 min GPS Antenna Avionics Nav-computer GPS receiver Wireless comm • IMU on vibration isolation mount ACCLIMATE

14. BEAR Future UAV Testbeds Small-size Coaxial Helicopter for indoor navigation Payload: up to 1KG ACCLIMATE

15. Berkeley UAV System Architecture • Scalable, dynamically configurable system for multiple heterogeneous agent scenario is achieved by • Standard sensor/actuator configuration • Common code architecture • Standardized communication protocol and interfacing method • Scalable ad-hoc wireless network ACCLIMATE

16. Wireless Communication Inertial Navigation Sensor CMIGITS-II Short Range: Wireless LAN Long Range(up to 30mi): Wireless Modems IMU+GPS 100Hz Update Berkeley BEAR Fleet: Ursa Maxima 1 Flight Controller: ADL P3 Laser Range Finder Modified light weight scanner Maximum Detection Range: 50m Pentium III 700MHz QNX RTOS Motorized scanner tilt mount GPS: Novatel OEM4 2cm StDAccuracy 20Hz Update Ground Control Station Laptop/PDA Windows XP/Windows CE ACCLIMATE

17. Avionics Software Architecture • Multi-process architecture with IPC • Processes dedicated for major navigation sensors and actuators running at nonuniform rates • Interprocess communication: POSIX-compliant shared memory with QNX-proprietary proxy for thread synchronization • Telemetry via wireless modem or IEEE802.11b ad-hoc mode ACCLIMATE

18. Hardware Micro controller: motion control Onboard computer: communication, video processing, camera control Sensors Sonars: obstacle avoidance, map building GPS & compass: positioning Video camera: map building, navigation, tracking Communication Serial Wave-LAN: communication between robots and base station Radio modem: GPS communication Pioneer Ground Robots Lucent WaveLAN GPS Antenna PTZ Color Camera GPS Ultrasonic array 4WD PT ACCLIMATE

19. BEAR Facility: Hydraulic Landing Deck • Hydraulically actuated Stewart Platform • Controlled by NI 6-axis motion control package • Capable of reproducing ship motions under various types of waves Designed by Tullio Cellano III ACCLIMATE

20. Mobile Ground Station • Agent monitoring/commanding • Full duplex voice communication • DGPS broadcasting • Auto-tracking video camera w/ recording • Wireless video reception • Weather station • Hoist ACCLIMATE

21. Mobile Access and Command Extension (MACE) “fly-by-fingertip” operation using touch screen minimal obscurity monocle display Real-time vehicle location & status monitoring Waypoint programming Hand-held UAV Interfaces ACCLIMATE

22. UAV Component Technologies ACCLIMATE

23. Berkeley UAV System Capability • Waypoint navigation - preloaded/interactive waypoint navigation - high speed tracking mode - automatic take-off/landing • MPC-based dynamic path planning - model-based trajectory generation - collision avoidance (TCAS) - aerial pursuit-evasion game • Vision-based navigation and landing • Formation flight ACCLIMATE

24. Flight Mode-based Way-point navigation • A helicopter flight can be composed with a number of flight primitives such as take-off, hover, turn, vertical flight, forward flight, etc. • A script language (VCL)provides an abstraction layer from high-level waypoint request to low-level vehicle control GO {AUTO,MANUAL} : Change Flight mode to either automatic or manual TakeoffTo <coord>{abs,rel} : perform autonomous take-off Hover <coord>{abs,rel} {heading=<heading>{deg,rad}} <duration>{sec,min} : hover with given heading angle for given time FlyTo <coord>{abs,rel} {vel <velocity>{mps,kmps,fps,knots,mph}} {passby,stopover} {autoheading} : Cruise to certain way point stopping over or passing by MoveTo <coord>{abs,rel} {vel <velocity>{mps,kmps,fps,knots,mph}} {autoheading} : Visit a sequence of way points with fixed heading Land : Perform automatic landing ** Helicopter flight is often more convoluted than a sequence of flight primitives MPC-based waypoint navigator ACCLIMATE

25. VCL-based waypoint navigation ACCLIMATE Perch & Move Flight Sequence (Aug 2004)

26. MPC-based Path Generation ACCLIMATE

27. Vision-based landing ACCLIMATE

28. Model-predictive approach • A promising control methodology for a system with input saturation, system nonlinearity, and state constraints in a dynamic environment • Stabilization and tracking problem is solved as an optimization problem of a cost function • Tracking error and control energy are penalized over a finite time interval from present to future • Control input saturation can be easily handled by simply forcing the saturation during optimization • State constraints can be included by introducing appropriate cost functions ACCLIMATE

29. Model-predictive approach • A sequence of control {u(t)}, t=tk,…tk+h that minimizes the weighting function over a finite horizon is computed every sampling time • Only u(t) at t=tk is used for control output. The rest may be recycled as the initial value to speed up the optimization at next sample hit • Generally very expensive in terms of computing power (>100 MFLOPs) • but within reach of most latest CPUs (Pentium 4 class) • Very efficient optimization algorithm is desired • Gradient-search method (Bitmead et al) is adopted here. ACCLIMATE

30. Model-predictive approach • Partially nonlinear system model is discretized (Euler) • Cost function is defined as the sum of quadratic tracking error and control input over finite horizon: t=tk … tk+N where Cost function for tracking error ACCLIMATE

31. MPC: input and state constraints • Input saturation is enforced by bounding the input with maximum control value during the optimization process • State saturation is enforced by including the following state penalty – ACCLIMATE

32. Distanceto the wall R0 MPC: application to urban navigation • Obstacle avoidance is addressed by including the proximity penalty in the cost function J • At any given time, the nearest point to • adjacent buildings is found and used. • Although it sounds simple, it assumes • the full information of the surrounding • environment • A partial map would suffice for MPC ACCLIMATE

33. MPC: gradient-based approach • The cost function is rearranged with Lagrange multiplier: • We want to choose that minimizes J By choosing such that We have ACCLIMATE

34. Obstacle Avoidance: Motivation • UAVs have been successfully deployed for numerous missions, mostly restricted to mid to high-altitude • Near-future UAVs are expected to interact with the surroundings: active sensing and realtime trajectory replanning is necessary • Many UAV programs require obstacle avoidance as a standard capability of UAV flight control system • (DARPA UCAR, OAV programs) ACCLIMATE

35. Obstacle Avoidance System Key Elements • Dynamic path planning: real-time path generation using model predictive control • Sensing: onboard 3D laser scanner or preprogrammed obstacle maps • Experiment system: Berkeley UAV architecture implemented on Yamaha industrial helicopter platform with 3D laser scanner ACCLIMATE

36. MPC-based Path Generation ACCLIMATE

37. Collision Avoidance using NMPTC • Five helicopters are given a destination point. • The shortest (optimal) trajectory will lead to a collision. • Each vehicle can detect other vehicles position only within the sensing/communication region. • Can each vehicle fly safely and optimally? Unsafe Desired Trajectory Resolved by NMPTC with Collision Avoidance ACCLIMATE

38. Collision Avoidance using MPC • Two UAVs are intentionally set on a head-on crash course • Model-predictive control-based trajectory planner computes safe trajectories with sufficient clearance in real time • Each vehicle’s current coordinate is used for MPC at each computation May 2003 ACCLIMATE

39. Collision Avoidance using NMPTC ACCLIMATE

40. 1. Initial position 2. Popup wall 3. Ceiling lowers 4. column NMPTC for Dynamic Environment ACCLIMATE

41. NMPC in 3-D complex environment NMPC • The nearest point of the surrounding buildings found • The obstacle weighting function is applied. • Receding horizon approach is advantageous • than potential function approach when the • helicopter stuck in deadlock by including the finite • future behavior in the cost function ACCLIMATE

42. NMPC in a Complex 3-D Environment Stuck in a local minimum Potential function NMPC Break away from a local minimum point ACCLIMATE

43. NMPC in 3-D complex environment ACCLIMATE

44. Obstacle Sensing: map-based • Map-based approach • - Obstacle map is measured and stored in the computer • - Upon request, the nearest obstacle coordinate is passed to the MPC unit - Sensing is always perfect, thus eliminating any risks due to sensing failure or any unpredicted control behavior ACCLIMATE

45. Obstacle Sensing: SICK laser scanner - An industrial two-dimensional laser scanner is used (www.sick.de) - It could be slightly heavy for most R/C helicopter-based UAV platforms Custom weight reduction is available from a local machineshop service ACCLIMATE

46. Object Detection with Laser Scanner Powerpole Electric powerlines Simulated Urban Objects Trees - Laser scanner in general can detect natural and artificial objects very well with great accuracy (powerlines, buildings, trees… but not glass) - active sensing: no lighting is required UAV in the air ACCLIMATE

47. Nearest-point Method • Cost Function: where At each k over finite horizon, the nearest point on the obstacle from the hypothetic vehicle location is found ACCLIMATE

48. Local partial map building for MPC Data is held in FIFO ACCLIMATE

49. Experiment Setup • Testbed • Ursa Magna 2 will be used for map-based OA • Ursa Maxima 1 will be used for laser scanner based OA • System Setup • Vehicle onboard navigation system • Ground Monitoring and Command Station • Ground-based MPC engine running on MATLAB in realtime • Ground-based 3D visualization station of laser scanner in realtime • Wireless Ethernet communication among systems • Crew • Experiment Coordinator: in charge of experiment sequencing • Safety pilot: in charge of routine flight and emergency take-over • Ground Operator: assists Experiment Coordinator to monitor Ground station • Camera Crew: takes video record of all experiment activities ACCLIMATE

50. Scanner Control Computer Tilt Mount Position Command • - Encoder • Servo • Micro controller Encoder reading - PIII 700MHz PC104 module Measured range data Ground Station Measured range data • 361meas/scan Minimum range data • Real-time 3D Visualization Vehicle state Light weight 2D Laser Scanner Flight Computer • Real-time optimization Reference trajectory • GPS+INS • PIII 700MHz • PC104 module Navigation data Minimum range data MPC Engine Obstacle Sensing using Laser Scanner ACCLIMATE