Plans and Procedures Essential Functions Delegations of Authority Orders of Succession Alternate Facilities Interoperable Communications Vital Records Human Capital TT&E Devolution Reconstitution Elements of a Viable COOP Capability
AGENDA • EXERCISE OBJECTIVES • EXERCISE BACKGROUND • EVALUATOR ROLES • EVALUATOR CRITERIA • AGENCY SPECIFIC AAR/HOT WASH
Type of Exercise • FEMA identifies three types of exercises • Table Top • Functional • Deployment • RICE 05 is a little of all of the above • What type of exercise depends on the individual agency • Best described as a “table top on steroids”
EXERCISE SIZE • 52 COOP plans to be exercised • 278 participants on site (estimated) • 50+ exercise staff • 165 participants at various off sites (estimated)
KC RICE 05 OBJECTIVES • Test Agency Orders of Succession • Test Agency delegations of authority • Test Execution of Agency Minimum Essential Functions • Test practicality of procedures surrounding use of Agency Alternate Facilities • Test use of agency interoperability communications • Test practicality and access to agency vital records • Test practicality of Agency plans and procedures in the COOP Program (reference: Federal Preparedness Circular 65)
ADDITIONAL OBJECTIVES KC RICE 05, NOT TESTED BEFORE • Test and develop Agency plans and procedures for Phase II COOP Operations Sub-objectives: • Demonstrate that all action items are logged and assigned with actions taken recorded. • Demonstrate that multi-tasked actions are coordinated properly and effective. • Demonstrate information flow to and from SERT/Senior Leadership of the Agency
ADDITIONAL OBJECTIVES KC RICE 05 • Test plans and procedures for both internal and external communications Sub-Objectives: • Demonstrate establishment of public information function • Demonstrate capability to coordinate the development and dissemination of clear, accurate, and timely information • Illustrate process of handling media inquiries • Demonstrate uses of Family Plan throughout the organization
ADDITIONAL OBJECTIVES KC RICE 05 • Test plans and procedures for communications with Disaster Ad-Hoc entities: 1)FEMA Resource Coordination Center, formally the ROC 2) Principle Federal Official (PFO)
ADDITIONAL OBJECTIVES KC RICE 05 • Test plans and procedures for devolution of operations. Sub-Objectives: • Identify policies and procedures to transfer essential operations to a backup organization. • Demonstrate a clear implementation plan of how and when operations should be transferred.
ADDITIONAL OBJECTIVES KC RICE 05 • Test plans and procedures for reconstitution and return to normal operations. Sub-Objectives: • Demonstrate ability to identify logistical needs to reconstitute services after the emergency has ended • Identify policies and procedures necessary to implement Phase III of COOP Plans
Differences • Interaction with elements of the new NRP • Regional Resource Coordination Center (RRCC) • Joint Information Center (JIC) • Principal Federal Official (PFO) • Phone lines to stimulate cross talk • Existence of a simulation cell • Existence of an off site element for 12 agencies
Scenario • Terrorist Group Strikes • Forces activation of COOP plans via • Real Loss of the Building • Fear of More Attacks • Scenario involves multiple attacks • New MO for groups • 9/11, other broken up attempts to all involved multiple attacks • Scenario a little wild in order to actively involve all agencies
Exercise Play • Action items will come in from several different angles • White Cell via paper • Various ways from the simulation cell • Come in various ways from other agencies • We are breaking new ground, it will be rough • Agencies will be challenged on new issues like devolution and reconstitution
Exercise Action Item Flow Agencies Controllers carry the paper White Cell Couriers carry the paper Simulations Cell
Exercise Artificialities • Sped up exercise clock • Communication complexities fudged • Either simulated or • Assumed it will always work • Organizational inertia assumed away • (the work to get something done internally)
Exercise Simulations A cell of players exist to simulate • Washington Headquarters for Organizations • US Coast Guard • Secretary of Agriculture’s office • City of Kansas City MO • The White House • Various DHS entities • HHS Secretary’s office • U.S. Congress
Exercise Assumptions • The Exercise is in compressed time. • At the start of the exercise all communications and IT infrastructure will be intact and operational. Exercise controllers might render them unavailable from time to time to test viability of other methods of communications. • All Agency alternate facilities survive the event and are available. • The nature of the threat is ever constant and uncertain. • Responses will have to be based on accepted standards, practices and policies for agencies. • It is to be assumed that Washington always has good communication lines to Kansas City to deliver their instructions. • Communications with people not participating in the exercise will have to be simulated. • If you get “killed” off during the exercise you may watch, take notes, but you are not to speak. The same applies if you are stricken with an illness. Then your Agency can request your replacement if they so see fit. Then you will come back in as your replacement for the exercise. • Responses to action items and inquiries should be fleshed out with as much detail as possible. • Participants can expect some feedback from the simulation cell.
What Is My Role? Observe and collect data Research & provide answers to Agency Questions if possible Try to minimize interfering with Agency operations
EVALUATOR DUTIES • Observe: Observe Agency performance in planning, preparation, and execution of the COOP plan based off of Agency and FEMA guidance • Control: Help the Agency focus on events to get the most out of the experience • Coach: Begin as necessary/requested • Analyze: What’s broken / How to fix • Provide Feedback: AARs
Evaluator Training Heisenburg Uncertainity Principle As an evaluator you may cause inordinate concern and activity if you ask a direct questions. Agencies consider you an evaluator and often react to your questions. Instead, attempt to answer your questions by observing—looking at charts, listening to conversations, checking maps, logs, etc.
Evaluator Training Heimlich Maneuver Sometimes AAR requirements will drive you to return for specific information. The degree of importance and urgency will warrant violating the Heisenburg Principle. You may have to get the information by “brute force,” far exceeding the Heisenburg effect by figuratively applying the Heimlich Maneuver. We will try to funnel action items through you so you can see what is going on.
Evaluator Training Stockholm Effect As a fellow Agency person, you will want your fellow workers and the Agency to do well during the exercise. This prompts a temptation to help the Agency. The danger is that the unit will do whatever you say, and concurrently, they will hold you responsible for any outcome, favorable or unfavorable. Remain somewhat distant and impartial.
Evaluator Criteria (THE HOW TO PART)
ESTABLISHING THE BASE • Review Plans, Policies, Procedures • Conduct Needs Assessment • Define Program Scope • Select Event Types • Develop Program Purpose • Develop Program Schedule PROGRAM FOLLOW-UP • Establish a Corrective Action Program (CAP) • Assign Tasks, Schedules & Responsibilities • Monitor Corrective Action Progress • Prepare Multi-Year Development Plans PROGRAM DEVELOPMENT • Identify Resources • Define Objectives, Key Events, Points of Review • Develop Materials (COOP TT&E Plan, Training Schedule) • Prepare Facilities & Logistics • Identify & Train Participants PROGRAM EVALUATION & CRITIQUE PROGRAM MANAGEMENT & IMPLEMENTATION • Review and Analyze Participant Questionnaires & Critiques • Review Notes From Observers & Data Collectors • Review After-action / Evaluation Reports • Conduct TT&E Events • Market TT&E Program • Coordinate with Staff and TT&E Planning Group Program Development Process Constructing a TT&E Program Your Role Here
Evaluation Requirements • Determine what outcomes will be evaluated, based on exercise objectives • Identify activities to be evaluated • Identify which functions should be observed • Determine where observations will take place • Identify the appropriate evaluation tools
Individual Evaluator Responsibilities • Try to observe exercise objectives • Monitor player actions and record them • Report any exercise execution problems • Monitor Action Item play to ensure exercise is progressing as planned • Provide observations on actions • Guide people in the agency specific AAR
Recording Observations • The emphasis is on Who? What? When? Where? How? Why? • Record observations through: • use of Evaluator Guides • blank sheets of paper • Collect exercise documents
Evaluator How To Notes • Compile observations into chronological narrative of events • Describe outcomes achieved or not – use questions below and evaluation guides: • What happened? • What was supposed to happen? • If there is a difference, why? • What is the impact of that difference? • What should be learned from this? • What improvements might be recommended?
Timeline Development Identify the appropriate outcome for each activity Make a team timeline of actions Focus on significant actions
Record Significant Activities • Initiating scenario events • Facility activities • Response actions • Key decisions made by Players • Deviations from plans and procedures • Completion time of events
Analysis of Performance • Analysis of activities • What tasks were to be accomplished • How well were they performed • Root causes of differences between expected and actual performance • Recommendations
Root Cause Analysis 1. Why did it happen? 2. Why did that happen? 3. Why was that? 4. And why was that? 5. And why was that? 6. And so on… ***Root Cause*** Each step must completely explain the step above… …down to the basic underlying causal factor.
Data Analysis • Conduct Hotwash • Develop timeline of significant events • Analyze performance: • Individuals • Teams/Functions • Outcomes
Levels of Analysis • Performance is analyzed at three levels: • Task level • Agency/discipline/function level • Mission level (within and across communities)
Task Level Performance Answers the question: did the person or team do the right thing the right way at the right time? Helps assess need for training, equipment, personnel, etc. Task = work with measurable output that has utility Levels of Analysis
Agency/Discipline/Function Level Performance — Multiple teams Answers the question: did the larger team or organization perform duties in accordance with plans and policies? Levels of Analysis • Helps assess communication, coordination, planning budgets, etc.
Levels of Analysis • Mission Level Performance • Answers the question: were the mission level outcomes achieved? • Addresses jurisdictional preparedness Outcomes = results
Event Development Process Evaluate the TT&E Event • Collect all comments, observations, and notes recorded during or after the event • Carefully review and assess this information • Identify any issues that require further study • Identify any TT&E needs that may have surfaced • Develop an after-action report Constructing a TT&E Program
Event Development Process (cont.) Evaluate the TT&E Event (cont.) • Enhancement of mission readiness of team members • Enhancement of team members’ understanding of COOP mission • Improvement in skill level of participants • Completion of identified/established objectives • Overall value of the event to the TT&E program • Degree to which the event met team members’ needs • Quality of instruction Constructing a TT&E Program
Title slide AFTER ACTION REVIEW I
Evaluator Training Specific Agency AARs After Lunch each Agency will have 30 minutes to conduct their own agency specific after action review (AAR). As the evaluator we ask you to help facilitate the AAR. This AAR will help you fill out the written AARs.
After Action Reports • There are three after action reports to fill out. Copies of the reports are in your handbook. • Individual: • Ask each individual to fill out one for themselves and turn it in. • External: • Each agency will report out on the lessons learned and smart practices so that other agencies may benefit. Do not put sensitive information in this AAR. That will remain internal. This report will go public. • Internal: • Each agency has a format to fill out a detailed after action report. The detailed report is for your use only.
AAR Elements • The after-action review (AAR) consists of four parts: • Establish what happened. • Determine what was right or wrong with what happened. • Determine how the task should be done differently the next time. • Perform the task again
AAR Agenda INTRODUCTION TRAINING OBJECTIVES EXERCISE SUMMARY SIGNIFICANT EVENTS WHAT HAPPENED? WHY IT HAPPENED? HOW TO IMPROVE? CONCLUSIONS INTRO 002
AAR Elements • Establish what happened. • The evaluator and the participants determine what actually happened during the exercise. Members assist in describing the flow of the exercise and related events and discuss outcomes from their points of view