Planning for Assessment

Planning for Assessment Marta Colón de Toro, SPHR College of Business Administration UPR-Mayagüez

On Planning… “Whatever failures I have known, whatever errors I have committed, whatever follies I have witnessed in private and public life have been the consequence of action without thought.” Bernard M. Baruch 1870-1965, American Financier “To be prepared is half the victory.” Miguel De Cervantes 1547-1616, Spanish Novelist, Dramatist, Poet “Before beginning, plan carefully.” Marcus T. Cicero c. 106-43 BC, Great Roman Orator, Politician

Purpose of the plan • Gives us direction • Establishes accountability • Helps measure progress • Keeps us in track • Facilitates execution • Increases probability of achieving goals • Demonstrates commitment to institutionalize assessment. at least…it Should!!!

Levels of Assessment of Learning Institutional Learning Goals Program Learning Goals AACSB’s Focus Course Goals Lesson Objectives

Writing a Program Assessment Plan: Steps 1-5 • Revisit Vision and Mission Statements • Express rationale and purpose • Identify audiences you wish to address • Set assessable learning outcomes • Identify (or design) enabling experiences

The Plan: Steps 6-10 • Select/design assessment instruments and establish performance criteria • Set schedule for assessing each goal. • Assess learning outcomes • Interpret and report results • Introduce change strategies

Step #1: Revisit Program’s Vision and Mission • Must take into consideration external and internal reality • Every effort must be based on your program’s mission • Mission must be owned by faculty, students and administrators • Must be revisited periodically

Step #1: Revisit Program’s Vision and Mission • Is it aligned with your institution’s mission? • Is it inspiring? • Does it delineate the scope of academic programs? • Does it define your geographic market? • Does it profile your completing students and faculty? • Is it unique?

Step #2: Express Rationale and Purpose for Assessment Rationale: Why are you writing this plan? “As a liberal arts college, we seek to enhance the personal development of our students.” Purpose: Who will be assessed? What will you assess? When will assessment takes place? “The primary purpose of the assessment program is to describe the cognitive and personal development of our students from matriculation to registration.”

Let’s practice Step #2: • Answer the following: • Why are you doing this plan? ________________________________________ ________________________________________ ________________________________________ • What main purpose will it serve? ________________________________________ ________________________________________ ________________________________________

Step #3: Identify Audience(s) • This section addresses: • To whom you plan to communicate your assessment results. • The message that you want to convey. • The media through which you will communicate results. Example: Who: Prospective Students Message: “We regularly assess students’ performance, and provide them feedback and guidance to improve.” Media: Program brochure and recruiting materials.

Let’s Practice Step #3:Identify at least two audiences… • Audience #1- _______________________ • Message - ________________________________________ ________________________________________ ________________________________________ • Media - __________________________________ ________________________________________ ________________________________________

Step #4: Establish Program Learning Outcomes • State the broad educational expectations for each degree program. • General, not specific. • general knowledge and skills • management-specific learning goals • Specify the intellectual and behavioral competencies a program is intended to instill. • Clarify how we intend for graduates to be different as a result of their completion of the program.

Learning Outcomes… • Need to be defined operationally in order to be assessable. • Normally, four to ten learning goals will be specified for each degree program. • Express what students will be able to dowhen they complete your program requirements. • Must be focused on the student’s learning outcomes • Specify (or suggest) level of learning.

Hierarchy of Learning Outcomes “Apply scientific inquiry methods.” “Think logically and critically.” “…basic and applied research.” “…apply data collection techniques.” “list steps in the scientific process.” Institutional College (Dept) Program Course Session

Focused on Teaching Focuses on the professor’s effort, intent, and interests or Focused on Learning Focuses on the student’s achievement Example: “Expose the student to recent research in organizational behavior”. Example: “After completing this course the student should be able to evaluate recent research in organizational behavior”. The focus! The student’s learning. Objective #4

Types of Learning (According to Bloom) Objective #4 What is to be learned? • Cognitive: mental skills (Knowledge) • “…list three major EEO laws…” • Affective: growth in feelings or emotional areas (Attitude) • “…value diversity…” • Psychomotor: manual or physical skills (Skills) • “… use the keyboard…”

Levels of learning… Objective #4 • Compare & contrast, critique, justify Evaluate Bloom’s Taxonomy (Cognitive Domain) • Adapt, combine, compare, contrast, design, generate Synthesize • Correlate, diagram, distinguish, outline, infer Analyze • Determine, develop, compute, utilize, conduct Apply • Classify, explain, discuss, give examples, summarize Comprehend • Define, describe, list, reproduce, enumerate Know

Dealing with “understand”… • Always ask yourself… • How do you know someone understands? He/she can: • Give examples of … • Determine the “correct” method to use… • Discuss pros and cons… • Identify elements in a given case…etc.

Let’s Practice Step #4:Write at least 2 Program Learning Outcomes “Graduates of the Business Administration Program at UPR-___________ will be able to: 1. (Do what?) (At which level?) 2. Use action verbs!

Step #5: Identify/Design Enabling Experiences • Curriculum • Focus on Core Courses • Taken by all students • Extra curriculum • Student associations • Seminars • Guest speakers • Forums • Internship/coop

Program Goal Matrix Objective #4 Goal #2 - “The students will use the appropriate software packages and hardware as a working tool in the daily operations of a business”.

Lets Practice Step #5:Choose 1 Goal and complete the following table:Goal: ________________________________________

Step #6:Select Assessment Instruments and Establish Performance Criteria • For each learning outcome choose an instrument Entry Level Midpoint Exit Level Assessment Assessment Assessment • Essay included Paper written Senior Essay in admission in 2nd yr. course package Criteria: 50% 70% 85% • Diagnostic Test Repeat Repeat Criteria: 44% 70% 85%

Direct Measures of Learning Diagnostic test to incoming students on learning goals Course-embedded assessment Capstone courses Projects Seminar Writing assignments Internships/Coop Proficiency exams Professional exams Student Portfolio Indirect Measures of Learning Employer Reports on Alumni Exit Surveys Focus groups Interviews Student reports on Internships/Coop Employer performance evaluation of interns/coop Student evaluation of courses Program Assessment Instruments

Examples of Measures of Achievement Example 1 • School A has defined a learning goal in ethical reasoning for each of its four undergraduate majors. Student achievement on this goal is relevant to demonstrating satisfaction of Standard 16. The school’s faculty has defined the goal. Learning Goal • “Each student can recognize and analyze ethical problems and choose and defend resolutions for practical situations that occur in accounting, human resource management, and marketing.” Demonstration of Achievement • The school uses course-embedded exercises in three required introductory-level courses. Faculty in the three disciplines have developed different methods for instructing and assessing achievement toward this learning goal. AACSB

Examples of Measures of Achievement In Accounting • A two-week module near the end of the introductory course is devoted to “Ethical standards and fraud in accounting.” • A topic outline has been developed by faculty members to structure an exam on the materials of this module, and a standard set of expectations has been created for grading the exam. • In addition to this exam’s contribution to the course grade, it provides a pass/fail indication on the learning goal. AACSB

Examples of Measures of Achievement In Human Resource Management • Students must provide four written analyses of problem situations during the course. On three of these analyses (on the topics of selection, reward systems, and job design), students are asked to respond to ethical issues. • A standard scoring key on the ethical component provides evaluation toward the course grade and a pass/fail indication on the learning goal. AACSB

Examples of Measures of Achievement In Marketing • Each student must compose a term paper analyzing a current national or international marketing campaign. • The analysis must include a specified set of components, and ethical issues that have been presented in lectures are among the required components. • In addition to the overall grade of the paper, each student receives a pass/fail indicator on the ethics component. AACSB

Let’s practice Step #6: • Choose 1 learning goal and brainstorm with colleagues on assessment instruments that may be used to directly measure student achievement of the goal. • Establish criteria (“passing grade”)

Step #7: Create a Calendar for assessing each goal

Let’s Practice Step #7:

Step #8:Assess Learning Outcomes • Conduct assessment according to schedule • Remember you can use samples

Step #9:Interpret and Report results • Collect, process and analyze results. • Envision sitting before the curriculum committee with the assessment results in your hand. • Engage them in analyzing and reflecting on the meaning of these results and implications to them. • KISS • Descriptive statistics, graphs…

Reporting results • Use your plan in Step #3 • Audience • Message • Media Change the culture! Communicate, communicate, communicate!

Step #10: “Close the loop”Introduce Change to Improve • In curriculum • Faculty development • New learning experiences

Determine the gap and narrow it! Objective #1 Assessment Reveals the Gap Intended Outcomes 1. Act ethically 2. Quantitative Analysis 3. Apply models 4. Decision making skills 5. Communication skills Actual Outcomes 1. 2. 3. 4. 5. ?

Congratulations! You did it!Now, complete it and put it to work!

Remember to Assess your assessment plan! We wish the best! colon-m@rigel.uprm.edu

Planning for Assessment