1 / 14

Training Evaluation- Design Issues

Training Evaluation- Design Issues. Basic Designs Post-test only Pre-rest/Post-test Complex Designs Post-test with control group Pre-test/Post-test with control group Time Series Solomon 4 group. Basic Designs. Post-test Only

siegelr
Télécharger la présentation

Training Evaluation- Design Issues

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Training Evaluation- Design Issues • Basic Designs • Post-test only • Pre-rest/Post-test • Complex Designs • Post-test with control group • Pre-test/Post-test with control group • Time Series • Solomon 4 group

  2. Basic Designs • Post-test Only • occurs when training is provided and then a post-test is given X T2 X – training T2 – post-test evaluation

  3. Basic Designs - Post-test Only (cont’d) • Goals of evaluation: • to determine if change has taken place • to determine if a level of competence has been reached • Post-test only design is sufficient if the evaluation is for the second goal • e.g. a training which is developed to equip employees with certain legal requirements

  4. Basic Designs - Post-test Only (cont’d) • Post-testing can be turned into a pre-test/post-test design by utilizing data from the following sources: • needs analysis • archival data, e.g. • performance appraisal • measures of quality • An equivalent group could also be chosen and given the same post-test and the evaluation becomes post-test only with control group

  5. Basic Designs - Post-test Only (cont’d) • Post-test only design is problematic for assessing change because change could be caused by a number of other competing courses

  6. Basic Designs- Pre-test/Post-test • The design can be represented as: T1 X T2 T1 – Pre-test X – Training T2 – Post-test • Main criticism is that it is without a control group – difficult to determine the impact of training on the changes that occur

  7. Basic Designs- Pre-test/Post-test (cont’d) • A method of dealing with the lack of a control group is known as Internal Referencing Strategy (IRS) • IRS includes using relevant and non-relevant test questions in the pre- and post-test • relevant : questions that deal with training contents • non-relevant : questions with aspects not trained (these questions serve as control)

  8. Complex Designs • Two factors to consider in developing a sound evaluation design: • control group: a group of similar employees who do not receive the training • random assignment: the assignment of employees to either the control group or training group by chance

  9. Complex Designs – Post-test Only with Control Group • This design can be represented by the following Trainee Group X T2 Control Group T2 • Differences in test scores between the groups will provide evidence of the effectiveness of the training • Weakness: there is no pre-test score to indicate equivalence of the groups before training • Representative sampling: matching of employees on specific variables e.g. age, educational background, experience etc. may reduce the weakness

  10. Complex Designs- Pre-test/Post-test with Control Group • It is represented by the following expression: Trainee Group T1 X T2 Control Group T1 T2 • Provides indication of equivalence between the two groups (through T1) • Post-test differences will provide evidence for training effectiveness

  11. Complex Designs- Time Series Design • The expression for this design is: Trainee Group T1 T2 T3 T4 X T5 T6 T7 T8 • It uses a series of measurements before and after training • The design can be made more powerful by using a control group Trainee Group T1 T2 T3 T4 X T5 T6 T7 T8 Control Group T1 T2 T3 T4 T5 T6 T7 T8

  12. Complex Designs- Multiple Baseline Design • Expression: Trainee Group A T1 T2 T3 X T4 T5 T6 T7 T8 T9 T10 .. Trainee Group B T1 T2 T3 T4 T5 X T6 T7 T8 T9 T10 .. Trainee Group C T1 T2 T3 T4 T5 T6 T7 X T8 T9 T10 .. Trainee Group D T1 T2 T3 T4 T5 T6 T7 T8 T9 X T10 .. • Multiple measures are taken (as in time series) but each group is trained at different times • If each group improves after receiving training, it can be said that the changes are due to training

  13. Complex Designs- Solomon 4 Group Design • Expression: Group 1 T1 X T2 Group 2 T3 T4 Group 3 X T5 Group 4 T6 • Members of the groups should be based on representative sampling • If T2 > T1 and T4, and if T5 > T6, and if T5 > T3 then inference about the effects of training can be made with confidence • If T6 = T1 and T3 (history and maturation are not the cause of improvement) • If T2 and T5 are equal (both higher than T1) then the reactive effect of testing is not the cause

  14. Evaluation– The Choice of Design • The true impacts of training depends on the validity of evaluation results • The more complex the design, the more valid the results • When multiple measures are not possible, the following design may be useful: Trainee Group A T1 X T2 Trainee Group B T1 X T2 Trainee Group C T1 X T2 Trainee Group D T1 X T2

More Related