Pablo and Autopilot: Performance Tuning in Distributed Computing Environments

1. Pablo Research Group - Department of Computer Science - UIUC Pablo and Autopilot: Performance Tuning in Distributed Computing Environments Shannon Whitmore/Ruth Aydt Pablo Research Group Department of Computer Science University of Illinois at Urbana-Champaign http://www-pablo.cs.uiuc.edu

2. Pablo Research Group - Department of Computer Science - UIUC PresentationOutline • Autopilot • Basic concepts • Component interactions • Pablo-provided monitor/control programs • Autodriver • Virtue • Integration of Autopilot with Cactus

3. Autopilot Toolkit Pablo Research Group - Department of Computer Science - UIUC • Provides a framework for the capture and analysis of real-time application and infrastructure data in a multi-threaded distributed environment • Offers the ability to control volume of performance data through • selective registration and property matching • analysis and data reduction at point of collection • constant, periodic, or on-demand transmission of data • ability to dynamically enable/disable data collection • Includes a control interface to allow steering of infrastructure policies and applications, either interactively or via automated decision procedures

4. Pablo Research Group - Department of Computer Science - UIUC Basic Autopilot Concepts • Sensors: provide data to remote processes, allowing real-time monitoring • Actuators: provide remote processes the ability to invoke local functions or update data, allowing remote steering • Properties: key-value pairs that are associated with and used to identify a sensor or actuator, allowing remote processes to be selective about the sensors and actuators they connect to

5. Pablo Research Group - Department of Computer Science - UIUC Basic Autopilot Concepts • Sensor Client: a process that connects to one or more sensors with matching properties and receives data from those sensors • Actuator Client: a process that connects to one or more actuators with matching properties and sends data to those actuators, causing application variables controlled by the actuators to be updated

6. Pablo Research Group - Department of Computer Science - UIUC Basic Autopilot Concepts • Autopilot Manager: a daemon process that is responsible for handling registration requests from sensors and actuators, and matching sensor client and actuator client requests to registered sensors and actuators

7. Pablo Research Group - Department of Computer Science - UIUC Autopilot Component Interactions Autopilot Manager 1. sensors and actuators register with their properties 2. clients request matching sensors and actuators 3. global pointers returned for matches 4. sensor and actuator controls and actuator data Instrumented Tasks Monitor/Control Task 5. sensor data

8. May be application code, library (MPI, HDF), or infrastructure resource monitor (lmon). Pablo Research Group - Department of Computer Science - UIUC Instrumented Tasks • May contain multiple sensors and/or actuators • Many instrumented tasks may be active at any given time • May register sensors and actuators with multiple Autopilot Managers running on different hosts Instrumented Tasks

9. Pablo Research Group - Department of Computer Science - UIUC Monitor/Control Tasks • May contain multiple sensor clients and/or actuator clients • Many monitor/control tasks may be active at any given time • May query multiple Autopilot Managers running on different hosts • May implement “human in the loop” (Autodriver, Virtue) or automated decision server (PPFS II) • May be monitor only,writing collected data to a file or displaying it Monitor/Control Task

10. Pablo Research Group - Department of Computer Science - UIUC Autodriver Monitor and Control Architecture Autodriver Java GUI Java Remote Method Invocation Autodriver - Autopilot Adapter Task Unix Instrumented Task Autopilot Manager

11. Autodriver Startup Pablo Research Group - Department of Computer Science - UIUC • User specifies hosts for Autopilot Manager and, if remote, Adapter • Main window displays currently registered sensors and actuators • User selects sensors and/or actuators they are interested in

12. Autodriver Field Selection Pablo Research Group - Department of Computer Science - UIUC • When a tagged sensor is selected, a new window showing the list of fields in that sensor are displayed • The user selects the field(s) they want to view

13. Autodriver Numeric Display Pablo Research Group - Department of Computer Science - UIUC • Data can be displayed as numeric values • The user can choose to save the data values to a file for later analysis

14. Autodriver Plot Display Pablo Research Group - Department of Computer Science - UIUC • Using ptplot packagefrom Berkeley, values can be plotted as connected or unconnected points • Multiple fields can be plotted to a single window • User can control number of points to display in window and zoom in on area of graph

15. Autodriver Actuator Interaction Pablo Research Group - Department of Computer Science - UIUC • User may enter value for selected actuator and transmit it to the remote process • Interface may be customized for non-numeric data entry such as pull-down menu choice of LRU or MRU for actuator controlling cache replacement policy

16. Pablo Research Group - Department of Computer Science - UIUC Virtue Monitor and Control Architecture Virtue Actuatorcontrols Tagged Sensor data Instrumented Tasks Autopilot Manager

17. Pablo Research Group - Department of Computer Science - UIUC Virtue Display and Control • Each sphere in the ring represents a workstation • lmon collects processor utilization data and makes it available via sensors • Virtue maps the data to the display • Data transmission frequency can be adjusted via slider connected to lmon actuator

18. Autopilot/Cactus Integration Plan • Autopilot sensors added to MPI and IO (HDF) libraries; automatically get sensor data when you link with these libraries. • Near term plan: create Autopilot thorn; sensors will provide timing measurements for each thorn • Customize Autodriver to be Cactus specific and display thorn performance information • Long term: Add actuators to provide remote steering of thorns • Any sensor/actuator data from thorns can be visualized with Virtue today.