1. Construct validity - feedback An author’s responses to your criticisms!

2. Purpose • Feedback and learning • This is one way of increasing understanding… • Critiques and the peer review process • Peer review can be pleasant or nasty, but you usually learn something • Levels of mastery • See handout posted on web

3. A. What the study intends to measure • This first part is about identifying the correct target… • Any criticisms you mentioned that are not about the constructs are a matter of external or internal validity

4. A. i. the relationship of interest • The relationship between feedback type, children’s ability rating, and age • In essence increased age is supposed to reverse the relationship between feedback type and ability rating

5. A. ii. The dependent measure(s) • Ability rating

6. A. iii. The independent measure(s) • Age, feedback type

7. B. i. Operationalizations - DV • Ability rating: • Watch a video of others performing • Extract answers via either one-to-one verbal or small groups filling in individual surveys • Rate ability via semantic differential scale of 1-5: • Not good at all…very good (at hitting the softball)

8. B. ii. Operationalizations - IV • Age: • selected people from within two age groups (7-10, 12-15) [no means or SDs given] • Feedback type • Manipulated by different comments on a video • E.g. criticism: “come on, that wasn’t even close”; neutral: “you missed”

9. C. Critique (at last!) • Ability rating concerns (your comments) • The kid’s ability to understand the rating system • Response will capture effort, not just ability • Test in other sports • Unclear scale/”leading the participant” • Only measured in one way (mono-method bias) • Only used softball to generalize to other sports • Occurred in groups – interaction of testing and treatment, or social effects • Possibility of copying each other

10. C. Critique (at last!) • Age concerns (your comments) • Older kids’ ability levels are higher, so differences might exist in what they rate as good/not good compared to younger kids • (I didn’t see anyone mention the similarity in age of the 2 groups, but this could clearly be mentioned)

11. C. Critique (at last!) • Feedback type concerns (your comments) • Neutral feedback confusing • Isn’t “you missed” still negative? • Body language/tone of voice of person giving feedback • Group size a social concern

12. C. Critique (at last!) • Other concerns (your comments) • Sample size • Control groups • Age used in three different ways • Only females, generalized to young children • Only used softball…generalized to all sports • Watching a video of one performance – don’t know prolonged effect of evaluation, nor firsthand evaluation • Influence of evaluator reading responses • Break groups up so that each only hears one comment type from the coach

13. My summary • Many of the real concerns with this study are about whether we were looking using the right settings and contexts, and so were external concerns • Clearly, several of your concerns were well spotted…my rating is we’re on the right track, but need more practice of the last step (D)