Company Profile

1. INTERNAL Centro Italiano Ricerche Aerospaziali Planning and Replanning of UAV flight in Surveillance Missions THE ROLE OF HUMAN-MACHINE INTERFACE IN AIRSPACE APPLICATIONS AND THE ENABLING TECHNOLOGIES Forlì, September 19TH 2012 Dr. Paolo Leoncini – CIRA Ing. Ilaria Sale – Alenia Aermacchi Company Profile

2. Briefing Objectives Briefing Objectives • Mission Planning/Replanning Concept for UAV flight in surveillance missions • External UAS Mission Planning • GCS Embedded Mission Planning/Replanning • Autonomous Mission Replanning

3. Mission PlannING/REPLANNING Concept for UAS • In recent years, the use of unmanned aerial systems UAS has progressively increased: applications range from military operations to border security and surveillance missions. • Tasks such as fighting forest fires, monitoring wildlife, search-and-rescue, traffic monitoring, aerial mapping are ideal applications for UAS. • In order to effectively use UAS capabilities and reduce the operational risks, a detailed mission planning is required. Planning process for a UAS is very complex since there are more paradigms to consider than for a tradition manned aircraft. • To reduce the operator’s workload and increase the system’s level of autonomy – especially in case of a replan – advanced route creation/validation algorithms are needed. In this work, we present the results obtained by an activity jointly done by AleniaAermacchi and Centro ItalianoRicercheAerospaziali about the development of mission planning/replanning systems for UAS introduction

4. Mission PlannING/REPLANNING Concept for UAS Mission Planning/Replanning Concept • UAS surveillance mission includes several and complex tasks. • In the near future, large, complex, time-critical missions will likely require multiple unmanned vehicles (UV) and multiple operators, able to combine their efforts as a team. • Joint operations require efficient mission planning and mission monitoring capabilities. • Major goal of a single UAS duting a surveillance mission is to provide scheduled information (i.e. sensor imagery) to a requesting unit (i.e. C4I) • Mission Planning Systems represent the key for successfully interoperability and cooperation between different actors.

5. Mission PlannING/REPLANNING Concept for UAS Mission Planning/Replanning Concept Mission: a sort of “folder” containing all information needed to perform the assigned goals in a given environment (constraints).

6. Mission PlannING/REPLANNING Concept for UAS Autonomous mission replanning functions integrated in the On Board Mission Computer • Advanced algorithms for autonomous replan • Severe constraints on computational time. General Design Principle On-board Segment Ground Segment Autonomous Replanner Embedded GCS Mission Planner External Mission Planner • External dedicated devices for full stand-alone mission planning management maintaining the commonality with GCS standard interfaces in terms of functionalities, algorithms and HMI. • Full planning editing and management capability. • Dedicated Human Machine Interface and more information with respect to a GCS planner. • Less constraints on computational time. • Mission Planner basic functionalities are integrated into Ground Control Station for mission importing, creation and exporting. • Capability to execute near real-time replan with the UAV in flight. • Reduced functions compared to the external planners for simplified used in operative environment. • More constraints on computational time.

7. Mission PlannING/REPLANNING Concept for UAS Mission Planning & Level of Automation Ground Based Planners LOA = 1 ÷ 5 A system, generally can work with several levels according to the proper characteristics and to the operator’s selection. On-Board Replanners LOA = 6 LOA Scale of Parasuraman, Sheridan et al.

8. External Pre-Flight Mission Planning

9. EXTERNAL PRE-FLIGHT MISSION PLANNING EXTERNAL PRE-FLIGHT MISSION PLANNING Purposes of a UAV surveillance mission is to acquire/collect imaging sensor data of ground targets images/video and geo-localization data Geo-referencing images Still/fixed or moving targets (target tracking) Typical UAV sensor payload for a surveillance mission: daylight camera (visible spectrum), wide to telescope FoV IR camera (SWIR a.k.a. near visible, MWIR, LWIR a.k.a. thermal), variable FoV Gyro-stabilized PTZ gimbal turret w/ GPS and INS along camera LoS Airborne Synthetic Aperture Radar (SAR)

10. EXTERNAL PRE-FLIGHT MISSION PLANNING EXTERNAL PRE-FLIGHT MISSION PLANNING Planning a UAV flight ensuring surveillance mission objectives, by keeping into account: Payload sensors capabilities Requested target visibility (NIIRS index (*), duration) Terrain area/topography of the land Aircraft performances Data-link communication coverage (comm stations) Fuel consumption No-fly zones/forbidden areas Others constraints Flight plan for a surveillance mission: an ordinary per-waypoint path, plus the payload sensor automation plan… Camera selection, control (orientation, zoom, …) all over the flight path Data-link/comm’s station switch (*) NIIRS: National Imagery Interpretability Rating Scale

11. EXTERNAL PRE-FLIGHT MISSION PLANNING EXTERNAL PRE-FLIGHT MISSION PLANNING Sensor coverage

12. EXTERNAL PRE-FLIGHT MISSION PLANNING PRE-FLIGHT MISSION PLANNING: HMI ISSUES Pre-flight surveillance mission plan: an off-line task Not human performance-critical, nor heavy workload Essentially a GUI affair, rather than HMI Mission simulation (preview, rehearsal) is more HMI-relevant The way the machine conveys the resulting plan to the mission designer He/she can interactively adjust locally the sensor automation plan… … or getting back to the planner to redistribute waypoints so to meet mission requirements Based on visual simulation / EO-IR sensor simulator Opportunity for experimenting advancedNatural User Interface concepts/technologies EO VIDEO IR

13. GCS Embedded Mission Planning

14. GCS EMBEDDED Mission Planning • Planning and Replanning basic functionalities are integrated into the GCS using the same interfaces normally used to manage aircraft and related payloads • Reduced functions compared to the external planners for simplified use in operative environment. GCS EMBEDDED Mission Planner/REPLANNER Mission Planner 10’’ touch screen display

15. The GCS embedded mission planner provides the following main features: • Creation of mission folder with all the information needed to perform a specific mission: • geo-referenced maps (vector maps, raster maps and GIS) • geo-referenced images • aviation data (airports, airspaces, airways, etc.) • operational area data • Mission plan management: • creating, saving and deleting plans, • importing and exporting plans • Advanced Route validation algorithms • check of imported route, • check of manually created route, • check of automatically created route. GCS EMBEDDED Mission Planning GCS EMBEDDED Mission Planner: basic functionalities

16. GCS embedded mission planner has an innovative Human-Machine interface featured by a touchscreen. GCS EMBEDDED Mission Planning Touchscreen Interface • Advantages: • Screen layout based on maximising support for information-intensive applications, such as mapping and imagery, • More instinctive interactions, • New types of interaction (e.g. scroll slider), • High flexibility and reconfiguration capability since all controls are generated via software, • Top-level control functions are managed principally by button controls and pop-up menus, • Greater useful available area with respect to traditional Multi Functional Displays, since pushbutton and alphanumeric keyboards are displayed only when needed.

17. GCS EMBEDDED Mission Planning Touchscreen Interface - HMI Issues pushbutton size • Pushbuttons size and separation have been defined according to MIL-STD-1472G. • For controls like virtual slider or scroll bar, no indication are provided by the standard (2012). • In most cases the minimum size (16 x 16 mm) has been considered sufficient. • For comparison, a traditional fingertip pushbutton has a minimum size of 10 mm and separation of 6 mm (sequential access).

18. GCS EMBEDDED Mission Planning Touchscreen Interface - HMI Issues Actuation feedback • The operator does not have a physical feedback that a control has been pushed. • The technology for the “tactile” feedback is still immature (especially in term of size). • Using a proper color coding the GUI provides a clear feedback to the operator about the automation state. This solution has been considered satisfactory by test pilots. • Virtually guarded pushbuttons for critical functions • GUI guides the operator using visual cues (e.g. labels, pop-up, dialog box, etc.)

19. GCS EMBEDDED Mission Planning Basic Mission Validation checks are integrated into the Embedded GCS in order validate the mission before the export to the aircraft. Mission Planner Algorithms • The following paradigm are considered for Route validation: • terrain avoidance • respect of mission zone (area of operation, corridors, no fly zones), • altitude constraints • target visualization by an EO/IR sensor • fuel consumption and performances • datalink coverage

20. Autonomous Mission Replanning Automatic Night Operations

21. autonomous Mission rePlanning autonomous Mission REPlanning • Automation is widely embedded in the UAS, allowing the operators to be focused on the mission tasks during very long endurance and repetitive tasks. • More complex and advanced replanning operations could be performed directly by autonomous UAV according to external stimulus (e.g. a target, a threat, a failure, etc…). • These operations rely completely on sophisticated algorithms. • Initial prototyping autonoumus planning function is currently integrated in the AleniaAermacchi’s MALE Ground Control Station and considered as a sort of “supervisor” for the pilots. • AleniaAermacchi and CIRA are currently jointly working in testing and exploitation of the Autonomous Mission Replanning with the goal to transfer this functionality on board.

22. autonomous Mission rePlanning Autonomous replanning raises also issues about the role of the human, and in particular its capability to put a veto about automation decisions or performing override/modification of the system proposals. Some automation interaction issues are possible such as: • mode awareness/confusion • increasing cognitive and monitoring demand; • unbalanced workload; • mis-calibration of trust in the automation. The challenge shall be to provide the systems and related interfaces with the necessary flexibility to manage a/c and missions also taking into account the UAS insertion in non segregated airspace. autonomous Mission REPlanning

23. autonomous Mission rePlanning autonomous Mission REPlanning – HMI • When the replanning module has computed a new valid route, a dedicated Replan pop-up is displayed • The operator can assess the new proposed route displayed on the Navigation Format (Touchscreen Display) • The GUI provides a clear indication of the new route by a proper color coding • The newproposedplancouldbeaccepted (ActivateReplan) or rejected (RejectReplan) by the operator

24. Overall uas planning/replanning picture Alenia Aermacchi Ground Control Station External Planner CIRA

25. Evolution of autonomous operation System autonomous operations on board based (Human Veto) System autonomous operations ground based (Human Veto) System assisted operations – Ground based Manned - Non Autonomous Operations Post 2014 Planned 2013-2014 Currently under qualification Early Sky-Y Flights 2007-2010

26. Q&A Questions?