Download
1 / 10
100 likes | 224 Vues
This document provides a concise overview of key concepts and definitions related to measurement, assessment, and evaluation within the context of educational programs and research. It differentiates between formative and summative evaluations, emphasizing the significance of data collection and processing through statistical methods. The text highlights the importance of accurately describing and evaluating outcomes against established goals, as well as discussing types of assessments and statistical techniques suitable for analyzing collected data.
E N D
Assessment, Evaluation, & Statistics:A Brief Overview WISTPC 2014 Florida International University Miami, FL Steven J. Condly, PhD United States Military Academy at West Point scondly@gmail.com
Definitions • Measurement (What do you know?) • Assigning numbers to things, events, people, actions, etc. • Assessment (How do you know?) • Measurements, actions, processes, data that answer the question. • Evaluation (How are we doing?) • Comparing results and observations with goals and objectives (implied or otherwise).
Implications • Measurement (assigning #s) • Need for consistency, accuracy, and precision. • Assessment (collected data) • Should relate to what it is purportedly describing. • Evaluation (comparing results to standards) • Not entirely objective, but should be reasonable and logical.
Types of Evaluation • Formative • Serves to strengthen or improve the program being evaluated • In-process • Summative • Examines the effect(s) program • Examines how well program goals and objectives were met • Contains implications for corrective action
Formative Evaluation • How well identified and defined is the problem? • How well does the program deal with the problem? • How well does the program progress? [feedback loop]
Summative Evaluation • Similar to Formative, but looks back over the entire life of the program. • Reaches conclusions regarding effectiveness, cost, overall success, and likelihood of generalizability (or moving on to the next level). • Easier to perform if formative evaluations are being performed and data/results collected.
Types of Assessment • Assessment is data collection with a purpose • Really only two ways to do it: • Question • Observe • Raw data have to be processed (statistics)
Statistics • Select a good comparison criterion or group • Standard statistical techniques are alright for Likert-scaled survey data • Don’t use p-values (NHST) • Strongly influenced by sample size • Small p does not necessarily indicate a stronger relationship or effect, or practical significance • What people think it is: P(H0=0|sample) • What it actually is: P(sample|H0=0) • How much there is there?
Effect Size Statistics • For Likert or interval-level data, when comparing two groups, use Cohen’s d • M1 - M2 / [(s1 + s2) / 2] • For ordinal data, when comparing two groups, use Probability of Superiority • MWU / (n1n2) • For correlations between two groups, use r2 • (r) (r) x 100 gives % of variance explained
Websites • http://oerl.sri.com/ccli_resources.html • www.socialresearchmethods.net/kb/contents.php • http://www.uccs.edu/~lbecker/ • Grissom, R. J. (1994). The probability of the superior outcome of one treatment over another. Journal of Applied Psychology, 79(2), 314-316.
