Outline

1. Outline • Ethics vs. Law • Ethical use of tests – issues: • Competence • Informed Consent • Knowledge of Results • Confidentiality • Test security • Divided Loyalties Ethics

2. Outline • Professional Issues • Theoretical issues • Test adequacy • Actuarial vs. clinical prediction • Social Issues • Dehumanization • Access to benefits of psychological testing Ethics

3. Outline • 6. Two examples of bad tests • Handwriting analysis • TOEFL • 7. The CPA Code of Ethics (synopsis) • Respect for the dignity of persons • Responsible caring • Integrity in relationships • Responsibility to society Ethics

4. Ethics: what one should or should not do, according to principles or norms of conduct Ethics codes are not produce by democratically-elected legislatures Enforcement mechanism usually informal, may be complex, even unconscious Ethics vs. Law Ethics

5. Law: what one must or must not do, according to legal dictates Laws are created by democratically-elected legislatures in democracies such as Canada Laws come with explicit penalties for infractions and a formal enforcement system Ethics vs. Law Ethics

6. Ethical use of tests – issues: • Competence • Informed Consent • Knowledge of Results • Confidentiality • Test security • Divided Loyalties Ethics

7. develop competence in assessment concepts and methods recognize boundaries of your competence competence is very important if test result affects someone’s life 1. Competence Ethics

8. client/subject must voluntarily consent psychologist must inform client about nature and purpose of assessment in understandable language 2. Informed Consent Ethics

9. Sometimes, it is acceptable to test without getting consent – e.g., you have given implied consent to be tested by registering in this course, but I have never sought your explicit consent Sometimes it is necessary to test without getting consent, even when consent is explicitly refused by person to be tested – e.g., when mandated by law Informed consent - exceptions Ethics

10. must fully disclose test results in understandable language but do not do so in a way that reveals the content of test items, because that content is proprietary information (e.g., somebody owns it – it’s not yours to reveal) 3. Knowledge of Results Ethics

11. PIPEDA Principal 4.9: Upon request, a person must be informed of the existence, use, and disclosure of his or her personal information and must be given access if requested. Section 9.3b says that access can be denied if it would mean revealing confidential commercial information (unless latter is severable). Constraints on client access to results Ethics

12. Ontario Personal Health Information Act: access rights do not apply to a record that contains raw data from standardized psychological tests or assessments, unless those data are reasonably severable. CPA and leading Canadian test publishers say that Test Materials are not “personal information” so should not be released. Constraints on client access to results Ethics

13. Test results are confidential information Release of results should only be made to another qualified professional after client’s consent 4. Confidentiality Ethics

14. Test materials must be kept secure Test items are not revealed except in training programs and when mandated by law, to protect test integrity Test items are private property 5. Test Security Ethics

15. Psychologist is still responsible for proper interpretation of test results Psychologists are professionals – they take personal responsibility for their work Psychologists are like engineers, not like assembly-line workers 6. Automated Scoring/Interpretation Systems Ethics

16. Who is the client? The person being tested, or the institution you work for? What if these parties have conflicting interests? How do you maintain test security but also explain an adverse decision? 7. Divided loyalties Ethics

17. Suppose you are asked to test a child who might have disorder X or Y Suppose X is very common and the government provides resources to help schools deal with it Suppose Y is not common and no resources are available in school to deal with it If your testing suggests Y, will you report Y – or X, so the child gets some kind of help? Divided loyalties Ethics

18. Professional Issues • Theoretical issues • Test adequacy • Actuarial vs. clinical prediction Ethics

19. Are you measuring a stable characteristic of person being tested? If so, differences in scores over time reflect measurement error or subject variables such as fatigue But we have little evidence that this is true, so what is the value of your test result – will it still be true next year? Especially problematic for personality tests 1. Theoretical Issues Ethics

20. Suppose you are a human resources person, testing candidates for a job, using one of the tests described in Chapter 18 If the test result says that someone does not have the characteristic you’re looking for, does that mean they could never acquire that characteristic? 1. Theoretical issues Ethics

21. How do we measure test adequacy? (We’re supposed to be good at measuring things!) What should go into an assessment of test adequacy? 2. Adequacy of tests Ethics

22. So far, in considering the relative merits of various tests, we have asked whether the tests are psychometrically adequate We haven’t asked: is the best test available good enough? Society uses that standard when testing becomes a legal issue 2. Adequacy of tests Ethics

23. U.S. Supreme Court Justice Potter Stewart said in a 1964 case that he didn’t know how to define pornography, but he knew it when he saw it. The Casablanca Test Is it possible for us to make good judgments on a question when we cannot articulate the basis for the judgment? 3. Actuarial vs. clinical judgment Ethics

24. Actuarial judgment occurs when we feed test scores into statistical formulas to diagnose a psychological condition or predict future performance. Clinical judgment occurs when we have a trained psychologist interpret test scores to diagnose a psychological condition or predict future performance. 3. Actuarial vs. clinical judgment Ethics

25. In actuarial judgment, we cannot make accurate predictions tailored to individuals Instead, our conclusion will be the same for every person with a given set of test scores In clinical judgment, the claim is that you can determine “what caused what” in an individual’s person’s life (Dawes, 1994) The evidence does not support this claim No experience effect 3. Actuarial vs. clinical judgment Ethics

26. Meehl’s work in the 1950s: feed test scores into statistical formula to predict future performance – or have trained psychologist interpret scores statistical formula does a better job of prediction still, someone has to be legally responsible – that has to be a trained professional 3. Actuarial vs. clinical prediction Ethics

27. Social issues • Dehumanization • Access to benefits of psychological testing Ethics

28. Does computerized testing and analysis of test results create a danger of minimizing human uniqueness? Humans are very complex – which allows us to be individuals, to be different from each other But testing and interpretation generalize 1. Dehumanization Ethics

29. Is generalization a problem? Computer-scored and administered psychological tests might determine important aspects of your life – by generalizing results to your case If you think this is a problem, why do you think so? What evidence do you have that this tactic hurts people? What alternative approach would you use? 1. Dehumanization Ethics

30. We’ve considered the possibility of being hurt by tests – but what about possible benefits of tests? Who gets those benefits? WAIS-III kit costs $775 per person – for tester to buy the kit. Cost must be passed on to someone. Who should that be? 2. Access to psychological testing Ethics

31. Psychological services are not covered by OHIP; Psychiatric services are Psychiatrists have opposed extending OHIP coverage to psychological testing Is that just turf protection? There is rivalry between the professions – but we could make a stronger case for billing OHIP if we used only actuarial judgment 2. Access to psychological testing Ethics

32. Two examples of bad tests • Graphology (handwriting analysis) • TOEFL Ethics

33. Used in hiring decisions in business Used to make assessments of personality type and to predict on-the-job performance Simner & Goffin (2003): very little evidence of validity for graphology Studies using non-autobiographical writing are the most useful and show the least evidence of validity. Graphology (handwriting analysis) Ethics

34. Dean (1992) meta-analysis of handwriting studies: average correlation of .16 between graphological judgment and job performance, using autobiographical writing. Correlation drops to .09 when using non-autobiographical writing. Graphology (handwriting analysis) Ethics

35. Simner & Goffin: both general mental ability testing and structured employment interviews correlate around .50 with job performance. When used together, the two correlate .63 with job performance. Thus, GMA testing and interviewing are much more likely to lead to sensible hiring decisions than handwriting analysis. Graphology (handwriting analysis) Ethics

36. Widely used to select among students who are not native speakers of English for admission to Canadian universities. TOEFL manual presents a distorted picture of validity studies Simner (1998) presents CPA’s recommendation – don’t use TOEFL for university admission decisions Test of English as a Foreign Language Ethics

37. TOEFL manual leaves out studies that show low to moderate correlations between TOEFL scores and teachers’ ratings of English language proficiency (.34 to .55) TOEFL manual leaves out studies that show low to moderate correlations between TOEFL scores and oral interviews (.30 to .60) TOEFL – Criterion Validity Ethics

38. Generally, these studies assess concurrent validity, not more relevant predictive validity Simner: some criterion validity tests for TOEFL involve the Michigan TELP, very similar to TOEFL – so this is really parallel forms reliability, not validity evidence TOEFL – Criterion validity Ethics

39. ETS: Native speakers of English do much better than non-natives on TOEFL. Simner: in these studies, native speakers are a more highly-selected group. Non-natives were unselected examinees from a range of backgrounds. Natives were selected from among entering freshmen Test anxiety differences TOEFL – Construct validity Ethics

40. Unpublished ETS study: 5000 English speakers had an average score of 590, below the 600 cutoff used for non-natives at many Canadian universities. If native English speakers can succeed with scores < 600 why can’t non-natives also succeed with scores below 600? TOEFL – Construct validity Ethics

41. Simner: Since 1995, 10 Ontario universities have raised their TOEFL cutoff from around 550 to 600. This means that only the top 10% of examinees are accepted, instead of top 30%. Taylor-Russell tables: r = .20, now 10% accepted, before 30% Net gain in percentage likely to be successful is 4%, at a cost of rejecting an additional 20%. Test of English as a Foreign Language Ethics

42. make admissions decisions on the basis of all relevant information, not just TOEFL scores do not use rigid cutoff scores consider the kinds and levels of English proficiency required in different fields and levels of study consider resources available for improving English language skills TOEFL: Simner’s recommendations Ethics

43. In thinking about the CPA Code of Ethics, pay attention to words such as benefit, beneficial, dignity, welfare, straightforward, caring, conscience, and other such words. In order for this Code to guide our behavior, we have to be able to agree on what these words mean in particular, real-life circumstances CPA Code of Ethics Ethics

44. CPA Code of Ethics • The following slides show the four basic principles advanced in the CPA Code of Ethics. All of the material on the next 15 slides is quoted directly from that Code. • There is significantly more information on each principle at this web address: http://www.cpa.ca/cpasite/userfiles/Documents/Canadian%20Code%20of%20Ethics%20for%20Psycho.pdf Ethics

45. CPA Principle I: Respect for the Dignity of Persons. • In the course of their work as scientists, practitioners, or scientist-practitioners,psychologists come into contact with many different individuals and groups, including: research participants; clients seeking help with individual, family, organizational, industrial, or community issues; students; trainees; supervisees; employees; business partners; business competitors; colleagues; employers; third party payers; and, the general public… Ethics

46. CPA Principle I: Respect for the Dignity of Persons. • In these contacts, psychologists accept as fundamental the principle of respect for the dignity of persons; that is, the belief that each person should be treated primarily as a person or an end in him/herself, not as an object or a means to an end… Ethics

47. CPA Principle II: Responsible Caring. • A basic ethical expectation of any discipline is that its activities will benefit members of society or, at least, do no harm. Therefore, psychologists demonstrate an active concern for the welfare of any individual, family, group, or community with whom they relate in their role as psychologists. This concern includes both those directly involved and those indirectly involved in their activities. Ethics

48. CPA Principle II: Responsible Caring. • However, as with Principle I, psychologists’ greatest responsibility is to protect the welfare of those in the most vulnerable position. Normally, persons directly involved in their activities (e.g., research participants, clients, students) are in such a position. Psychologists’ responsibility to those indirectly involved (e.g., employers, third party payers, the general public) normally is secondary. Ethics

49. CPA Principle III: Integrity in Relationships. • Psychologists are expected to demonstrate the highest integrity in all of their relationships. However, in rare circumstances, values such as openness and straightforwardness might need to be subordinated to the values contained in the Principles of Respect for the Dignity of Persons and Responsible Caring. Ethics

50. CPA Principle III: Integrity in Relationships. • The relationships formed by psychologists in the course of their work embody explicit and implicit mutual expectations of integrity that are vital to the advancement of scientific knowledge and to the maintenance of public confidence in the discipline of psychology. These expectations include: accuracy and honesty; straightforwardness and openness; the maximization of objectivity and minimization of bias; and, avoidance of conflicts of interest. Psychologists have a responsibility to meet these expectations and to encourage reciprocity. Ethics