Training Effectiveness/Evaluation and Training Assessments

1. Training Effectiveness/Evaluation and Training Assessments Fiona Hughes, Pfizer, Little Island

2. Training Team Structure/Resources

3. Where we were with training 3 years ago • Paper training records – filed by SOP name • Transcribed data to a database - errors • No standard course names numbers – duplicates, poor traceability • Training was done as SOPs were updated every 3 years • No formal definition of SOP Read and Understand, Course Instructor Led or On the Job Training OJT

4. Introduced new electronic system • Cross checked old vs new system • 3 years of data was uploaded manually • Introduced an electronic Learning Management System – Plateau • Put together Job Function Curriculum JFC for all colleagues • Verified data was correct - corrected • Training % complete was 37% - planned to close

5. Metrics • Training % complete measured every month on Quality Metrics board • Ran reports for % complete by • Dept • Person • Item • Analysed largest gaps • Planned to close

6. Plan to close training gaps • SOP Read & understand – online link to Documentation system + paper training record • Course Instructor Led - Scheduled 2 weeks of training – lunchtime sessions + paper training record • Assessments in place for some course e.g. GMP Refresher training

7. Training System

8. Training Policy • Overarching document defines that training system is made up of: • Induction • Job descriptions • Job Function Curriculum • Methods of training delivered • Importance of Training

9. Induction and Onboarding • Buddy system – point of contact • Booklet for new starter • Overview of systems and access • Map of site • Safety Policy • Sports and social aspects • Aimed as reference book

10. Methods of Training • Defines types of training and when to use • SOP Read and Understand • Course Instructor Led • On The Job Training – OJT • Observe • Assist • Be Observed

11. JFC Creation and Review • Core JFC • Job Specific JFC • Template for JFC & format • Quality, safety, technical considerations • Revision Control • Review to reassess content

12. Assessment and Evaluation of training • All SOPs will have assessment questions • Assessment in body of SOP & electronic – gets approved by Dept, QA and EHS • Minimum of 3 questions per SOP • Multiple choice to align with electronic system • One set of questions for SOP & Course • Assessment can be done electronically or on paper

13. Multiple Choice Questions • Clear, grammatically correct questions • No trick questions • Mix up the position of the correct answer • Ensure all answers are the same length approximately (same no. of words) • Avoid “All of the Above” & “None of the above” answers

14. Self Management of Training For SOP Read and Understand Trainee can • Log on to LMS • Open up SOP – online link • Read SOP • Take assessment online • Sign training record electronically

15. Our Criteria • Pass rate is 100% • Assessments in SOP and electronic for LMS • 2 attempts at assessment • If trainee fails, trainee must go to SME/Instructor for instruction

16. Course Instructor Led Assessment • Trainer delivers training course • Trainee may use case studies, group exercise • Use assessment in SOP after training complete • All trainees are given opportunity to review Q&A and correct any wrong answers • Instructor signs off • Trainees sign electronic training record – online, gaps closed immediately, no transcription errors, no paper gets lost

17. OJT Assessment • Observe x times • Assist x times • Be Observed x times • Instructor & trainee sign off when the trainee is competent and has demonstrated

18. Significant Achievements 1)Increase in % Training Complete: • 37%complete in Apr’08 96% Mar’10 2) Self Management of training: • 410SOP Read & Understand items are set up online on LMS (~500 SOPs) • 6,649SOPgaps closed by colleagues online • 9,913 Course gaps closed in 2009 Very successful 18 months!

19. Focus areas for 2010 Launching in 2010 • Auto-attendance on LMS • Allows us to close gaps immediately – log on to LMS to create electronic training record • Update LMS through our SOP System • Reduce need for paper forms – all new and revised SOPs will be updated on LMS automatically • Scheduled offerings on LMS • Plan training, use as a calendar and review at end of year

20. Challenges for 2010 • Assessments in place for all SOPs • Questions must be to high standard • Must test knowledge • Must keep questions up to date in future • JFC Review for all JFCs - 200 • Need to review JFCs for content • May create new JFCs / smaller JFCs / phased basis • Scheduled offerings on LMS • Plan training, use as a calendar and review at end of year • Optimise use of LMS