PRESENTATION FORMAT • Review of the Canadian Forces Individual Training & Education System (CFITES), • Validation terminology, • Purpose of Validation, • Fundamental Processes of Validation, • Validation Model, • 3 Types of Validation Surveys, • Impact of Validation Surveys, • Review of CFITES Model, and, • A Program Evaluation Experience at CFLS.
3 FUNDAMENTAL PRINCIPLES TO CFITES • Performance orientation, • System approach, and, • Optimum efficiency.
VALIDATION TERMINOLOGY • Validation vs Verification, • Validation vs Evaluation, • Role of Program Evaluation, and, • Validation vs OA/FA.
PURPOSE OF VALIDATION • Verify adequacy of training and education (T&E), • Identify excessive or unnecessary T&E activities, and, • Keep T&E programmes open to changes.
FUNDAMENTAL PROCESSES FOR VALIDATION • Scope and Plan, • Collect and Analyze Data, and, • Report and Recommendations.
3 TYPES OF VALIDATION SURVEYS • Routine Check, • Comprehensive Review, and, • Focus Review.
Functional Analysis Impact of Validation on Management System a Non-TrgProblems Analysis CFITES Design QuantityControl Validation Development Evaluation ResourceMgt Conduct Verification
CFITES Model Verification OA\FA Analysis Design Validation Development Conduct Evaluation
Program Evaluation Experience at CFLS • Mrs Lucie Ratté, Chief of Standards.
Programme Evaluation A programme is effective to the extent that the learner has achieved the performance objective(s). A programme is efficient to the extent that the methods, materials and the expenditures of resources are optimized.
MSLTP Mid February End of June ENTRY August 100 % 80 % 750 HOURS 300 HOURS PL1 / PL 2 / PL 3 / PL 4 / PL 5 PL 6 / PL 7 Intermediate Level Continuous Course Elementary Level Elementary Level One Progress Level = 150 hrs Optimum duration ??? Reading Writing Oral Interaction Qualifications AAA BBB
Military Second Language Training Programme Evaluation • AIM : to modify and improve an aspect of the programme in order to optimize the training. • Elementary Level Evaluation (duration) .
Intermediate Level Continuous Course, 2002-2003. Sample : 51 Students (Officers and Non-Commissionned Officers). Groups of 6 to 8 students. 4 Different Schools. Training Plan : 750 hours to achieve AAA Level. Conditions: MSLTP curriculum and EC, Testing after 600 hours, Reading + Writing without preparation, Oral Interaction -20 min preparation, and, The certification was given by the SLE (Second Language Evaluation). Elementary Level Evaluation (duration)
Analysis Criterion • Rank, • MLAT (aptitude), • Testing Over the phone / face-to-face, • Location, and, • SLE results.
Data Analysis Results • 75 % of the students obtained an Elementary Qualification A in 3 skills : Reading, Writing and Oral Interaction, • 60 % of the students obtained an Intermediate Qualification B in Reading and in Writing, • 25 % of the students failed in one skill, and, • Possible overtraining for students with highest aptitudes (Superior and Above-Average) High correlation between results and the aptitude.
MSLTP AAA February BBB June August Intermediate Level Continuous Course 750 HOURS 300 HOURS 11 months posting Qualification BBB PL1 / PL 2 / PL 3 / PL 4 / PL 5 PL 6 / PL 7 Xmas End Jan New Proposal 600 HOURS 450 HOURS 5 Month courses Before and After Xmas. Elementary Lvl Crse – Qual – AAA. Intermediate Lvl Crse – Qual –BBB. PL1 / PL2 / PL3 / PL4 PL 5 / PL 6 / PL 7 Elementary Level Course Intermediate Level Course PL 5 / PL 6 / PL 7 PL1 / PL2 / PL3 / PL4 Extra training (90 hrs)+ retest.. Intermediate Level Course Elementary Level Course AAA BBB QUALIFICATIONS
REVIEW - QUESTIONS • Review of the CFITES, • Validation terminology, • Purpose of Validation, • Fundamental Processes of Validation, • Validation Model, • 3 Types of Validation Surveys, • Impact of Validation Surveys, • Review of CFITES Model, and, • A Program Evaluation Experience at CFLS.