Comprehensive Overview of ADDIE Model and EOTA Qualification Standards

1. NA-40 Training Overview January 31, 2014

2. EOTA Qualification Standards

3. Qualification Process Analysis Curricula Exercise Evaluation Tasks Objectives

4. ADDIE Model • Systematic instructional design model used globally: • Analysis • Design • Development • Implementation • Evaluation

5. Analysis • Performs Feasibility Study • Identifies learning problem, the goals, and objectives • The audience’s needs • Existing knowledge • Learning Environment • Any constraints • Delivery options • Timeline

6. Design • A systematic process of specifying learning objectives based off of tasks • Design Document – outlining details of course or exercise, including types of evaluation Exercise Design Document

7. Development • The actual creation (production) of the content and learning materials based on the Design Phase/Document Exercise Design Document

8. Implementation • Training is executed utilizing ISO registered processes for: • Web-Based/Computer-Based • Instructor-Led • Self-Study • Blended • Webinars • Mobile Training Teams • SharePoint and Learning Management System (Defensibility) • Repository • Version Control • Easy access from remote locations

9. Evaluation • Formative • Is present in each stage of the ADDIE process • Monitors student learning to provide ongoing feedback throughout curriculum • Summative • Evaluates student learning at the end of the course • Tests • Feedback from students

10. Kirkpatrick’s Levels Four Levels of Evaluation • Reaction - immediate satisfaction • Learning - results in increase of knowledge/skills/attitudes • Behavior - transfer from classroom to job (3-6 months post training) • Results - can be measured in reduced cost, improved quality, increased production, efficiency, etc.

11. Learning Management System • Handles course distribution and course management • Facilitates course exams and allows item analysis • Course registration/repository • Creates custom reports • Generates certificates

12. NA-40 Training NA-40.2 Technical Training Liaison MANAGEMENT RESPONSIBILITY - Provide Direction/Set Standards - Quarterly Reviews - Reports/Metrics - Provide SMEs - Monitor Students/Customers Satisfaction Ratings DOE EOTA INPUT - Management Support - Feedback - NA-40 Program Knowledge - Subject Matter Expertise - Lessons Learned from Training, Exercises and Real World Events OUTPUT - Defensibility - Clearinghouse - Repository for all NA-40 Courses - Credibility - Customer Trust - Consistent Products - Standardized Methods - Reliability RESOURCES - Experienced Staff - Objective Assessments - Standardization - Formal Review of Courseware - Partnerships - Training Technology - Tracking/Reporting Capabilities - Document Repositories - Collaboration Tool (SharePoint) QUALITY - Maintain ISO Registration - Corrective/Preventive Action - Audits/Reviews (internal/external)

13. Exercise Planner Support • EOTA offers exercise capabilities and a performance-based exercise program for exercise design, development, conduct, evaluation, and improvement planning • Standardized policies • Methodologies • Terminologies • “Reachback” to Subject Matter Expertise • On staff controllers, evaluators, and facilitators

14. Exercise Types • Discussions-based • Seminar • Workshop • Tabletop • Operations-based • Drill • Functional Exercises • Full-Scale Exercises

15. Implementation • Development of resource documents • Situation Manual • Exercise Plan • Controller/Evaluator Handbook • MSEL • Player Handout • Exercise Evaluation Guides (EEGs) • After Action Report

16. After Action Reports • After Action Report (2 sections) • Capture observations/recommendations (process driven) • What was accomplished • What worked well • What requires improvement (tactics/techniques) • Participant Reaction Form Summary • Qualitative • Quantitative After Action Report Observations Recommendations Participant Reaction Form Summary

17. Corrective Action Plan • ISO registration includes: • Corrective Actions • Preventative Actions • Improvement Opportunities • Assistance with CAP for Watch Office • Discussion • Path forward • Outline responsibilities • Tracking Corrective Action Plan

18. EOTA Exercise Progress • Conducted a Feasibility Assessment • Analysis Completed • Needs Analysis (helped identify if training was truly the answer) • Training Requirements Analysis (focused on what the regulatory requirements are that pertain to training and or areas of responsibility) • Resulted in Standard Curriculum • Note: Review sample DIF surveys

19. Exercise Curriculum • Web-Based Training • EXR101DW: Intro to the DOE/NNSA Exercise Program • EXR102DW: Exercise Design • EXR103DW: Exercise Scenarios • EXR104DW: Completing the Exercise Plan • EXR105DW: Post Exercise Activities • EXR106DW: Optimizing Your Site Exercise Program • EXR107DW: Controlling Security Exercises • EXR108DW: Designing Exercise Evaluations • Basic Exercise Controller and Evaluator Tutorial (ORISE) • Instructor-Led Training • EXR 207: Controlling Security Exercises Practicum • EXR 231: Exercise Design Laboratory (Pilot course January 14-16, 2014)

20. Where we are going… • Clearinghouse • Validate content and format for consistent results across all vested customers • Repository for All NA-40 courses • One area to look for all NA-40 available training as well as direction to locate other training offerings available (FEMA, DHS) • Defensible Training