RUSH TO ASSESSMENT:

1. RUSH TO ASSESSMENT: Designing Assessment to Produce Failure Roberta Madison, Gloria Melara and Robert Lingard 3/12/2003

2. Session Overview • An example of a recent assessment project at CSUN • A group activity designed to gain insight into effective approaches to assessment • Summary and review of group results 3/12/2003

3. Session Objectives At the end of the session participants should be able to: • Identify potential problems in designing outcomes assessment • Select/create appropriate assessment measures and tools • Identify the steps in designing a productive assessment project 3/12/2003

4. Common Problems in Assessment • Assessment results are often needed quickly • Insufficient resources (people, tools, etc.) are usually applied • Assessment is sometimes viewed as an end in itself (e.g., an accreditation team must be convinced that assessment is being done) 3/12/2003

5. An Assessment Example • Examine an approach taken toward assessing Computer Literacy at CSUN • Look at the difficulties and problems encountered • Suggest some recommendations for improving the assessment process 3/12/2003

6. Why Was Computer Literacy Chosen for Assessment? • Ensuring the Computer Literacy of students was viewed as an important University goal • The effectiveness of CSUN’s general education course in Computer Literacy was unknown 3/12/2003

7. The Approach Taken • To get results quickly, an existing instrument, TekXam, was selected to measure student’s computer skills • Students taking the General Education course in Computer Literacy and a control group not taking this course were given pre tests and post tests • The results were analyzed 3/12/2003

8. Study Results 3/12/2003

9. TekXam Passing Rates 3/12/2003

10. Completion Rates 3/12/2003

11. Problems Encountered • The TekXam instrument did not test what our Computer Literacy course covered • TekXam was difficult for students to use • Administration of TekXam was tedious and rigid • Student participation rate was low 3/12/2003

12. Conclusions and Recommendations • A clear definition of Computer Literacy is needed before it can be assessed • Clear (measurable) objectives need to be established for the assessment • Incentives are needed to increase student participation • Better assessment tools are needed 3/12/2003

13. A Sample Problem • Assume you are given the task of assessing the effectiveness of online instruction for a particular program • What needs to be done? 3/12/2003

14. First Group Task(Risk Analysis) • Identify potential problems or obstacles that might be encountered in trying to do this assessment • List them in order of importance (high probability & high impact) 3/12/2003

15. Problems/Obstacles Identified 3/12/2003

16. Second Group Task • For one of the problems identified suggest possible solutions • List the advantages and disadvantages of each solution 3/12/2003

17. How Can We Ensure a Successful Assessment? • Make sure the goal of the assessment is clearly understood • Establish measurable objectives toward achieving the goal • Determine how to measure each objective (in multiple ways) 3/12/2003

18. GOAL OBJECTIVE OBJECTIVE OBJECTIVE MEASURE MEASURE MEASURE MEASURE MEASURE MEASURE MEASURE MEASURE MEASURE 3/12/2003

19. OBJECTIVE DATA DATA DATA DATA DATA DATA DATA DATA DATA ANALYZE GOAL Obstacles CHANGES? MEASURE MEASURE MEASURE MEASURE MEASURE MEASURE MEASURE MEASURE MEASURE 3/12/2003

20. Final Group Task • Suppose your task was to assess the effectiveness of online instruction in teaching written composition • Specify one measurable objective and identify two or more ways to measure it. 3/12/2003

21. Measurable Objective Example Students will be able to write a grammatically correct English sentence with a subject and a verb. 3/12/2003

22. Conclusions • A successful assessment project must be carefully planned with sufficient resources allocated (we must avoid rushing to assessment) • The goal or goals of the assessment must be clearly understood • Specific measurable objectives must be established • Appropriate tools must be found/created to measure each objective in multiple ways 3/12/2003

23. References • A short class feedback/assessment. It assesses students’ perception of the classroom environment. http://www.cod.edu/outcomes/SoaClsDs/StopStartCont_Assessment.PDF http://www.cod.edu/outcomes/SoaClsDs/feedback.PDF • This assessment activity clarifies expectations and mutual responsibilities for student learning and success. http://home.att.net/~p.klassen/SuccessAttribution.htm • An extensive list of many college assessment sites from North Carolina State University http://www2.acs.ncsu.edu/UPA/assmt/resource.htm • Nine Principles of Good Practice for Assessing Student Learning from AAHE, Alexander W. Astin, et.al http://www.aahe.org/assessment/principl.htm • A good resource with information from classroom to institutional assessment http://www.cod.edu/Dept/Outcomes/AssessmentBook.pdfhttp://www.cod.edu/Outcomes/ 3/12/2003

24. Contact Information • Roberta Madison - roberta.madison@csun.edu • Gloria Melara - gem@ecs.csun.edu • Robert Lingard - rlingard@ecs.csun.edu 3/12/2003