Breadth and Depth of Data: Assessment of the Curriculum Using Quantitative and Qualitative Approaches

1. Breadth and Depth of Data: Assessment of the Curriculum Using Quantitative and Qualitative Approaches

2. University of Puget Sound • Tacoma Washington is home, located between Mt. Rainier and Puget Sound • Primarily undergraduate liberal arts college • Small graduate programs in Education, Occupational Therapy, and Physical Therapy • 2,800 students (250 are graduate students) • 300 faculty and 500 staff • This fall implemented a requirement that students live on campus for their first four semesters

3. University of Puget Sound

4. Our Office Changes • Our office doubled in size in May of this year • Participating in more collaboration between other offices • Collecting data from a larger and more representative part of the population • Encouraging campus to take a different look at data and how they use it • Aim to affirm the dependability of institutional data, • Enable a critical look at data • Foster the use of data across campus

5. Vision, Mission, and Values • We have redefined our vision, mission, and values • Vision: We aspire to be an integral part of a Puget Sound culture where data is sought after, creates conversation, and is used to actuate positive change on campus and throughout higher education • Mission: The Office of Institutional Research provides accurate data, clarifies its meaning, and through the active use of that data, advances the mission of the University of Puget Sound • Values: Integrity, Collaboration, Flexibility, Courage, Awareness, and Collegiality

6. Role of IR in Core Curriculum Assessment • Asks seniors questions related to the core on the annual Senior Survey administered in the spring • Facilitates Senior Focus Groups every spring to obtain qualitative data about the 2 areas of the core that the Curriculum Committee is assessing in the upcoming year • Analyzes the quantitative & qualitative findings and presents them to the Curriculum Committee

7. Who/What is the Curriculum Committee? • Currently consists of: • 14 faculty members • 2 staff members • 1 student member • Tasked with assessing the core curriculum on a rotating cycle • Assess 2 of the 8 core areas every year • This year, also assessed the Upper Division Requirement

8. 8 Educational Goals for Undergraduates • The ability to think logically and analytically; • The ability to communicate clearly and effectively, both orally and in writing; • Intellectual autonomy and the accompanying capacity to learn independently of a formal educational structure; • An understanding of the interrelationship of knowledge; • Familiarity with diverse fields of knowledge; • Solid grounding in the special field of the student's choosing; • An acknowledged set of personal values; • Informed appreciation of self and others as part of a broader humanity in the world environment.

9. 8 Areas of the Core Curriculum • Writing & Rhetoric • Scholarly & Creative Inquiry • Fine Arts Approaches • Humanistic Approaches • Mathematical Approaches • Natural Scientific Approaches • Social Scientific Approaches (2012) • Connections (2012) • 1 unit – 1st year • 1 unit – 1st year • 1 unit – 1st through 3rd year • 1 unit – 1st through 3rd year • 1 unit – 1st through 3rd year • 1 unit – 1st through 3rd year • 1 unit – 1stthrough 3rd year • 1 unit – 3rd or 4th year

10. Methodology and Design

11. Guiding Questions • Do the courses that satisfy the core serve the goals of the core? • Does the “rubric” describe what we are or should be doing?

12. Social Scientific and Connections Rubrics SOCIAL SCIENTIFIC APPROACHES: The social sciences provide systematic approaches to understanding relationships that arise among individuals, organizations, or institutions. Students in a course in the Social Scientific Approach to Knowing acquire an understanding of theories about individual or collective behavior within a social environment and of the ways that empirical evidence is used to develop and test those theories. CONNECTIONS: Students in Connections courses develop their understanding of the interrelationship of fields of knowledge by exploring connections and contrasts between various disciplines with respect to disciplinary methodology and subject matter.

13. Survey Questions Thinking back to the course you took to fulfill your Social Scientific Approaches Core requirement, to what extent did that course enhance your ability to: • Understand relationships that arise among individuals, organizations, or institutions. (base level) • Apply theory and evidence as a way of understanding individual or collective behavior in a social environment. (higher level) Thinking back to the course you took to fulfill your Connections Core requirement, to what extent did that course enhance your ability to • Understand connections among different things you were learning • Analyze a subject from multiple approaches

14. Focus Group Question Themes Why does is this core required? Which course did you choose and why? How did the course help you understand the goal? To what degree do you see your other course differently? What impact has the core had on your in and out of class perspectives?

15. Results • Use of survey and focus groups results to compose comprehensive report • Allowed for breadth and depth of information • Report focused on two key areas of core curriculum

16. Results • Example of comprehensive approach • Survey asked about Connections course’s capability to enhance student’s ability to analyze a subject from multiple approaches • Focus group asked how Connections course meets goal of analyzing subject from multiple perspectives

17. Results • Delivered the report and a presentation to the curriculum committee • Expectations (taken from rubric) • Results (taken from report)

18. Results • Social Scientific • Our expectations • Understanding of theories about individual or collective behavior within a social environment • Exploration of assumptions embedded in social scientific theories • Understand the ways that empirical evidence is used to develop and test those theories • Examine importance of simplifying or describing observations of the world in order to construct a model of individual or collective behavior

19. Results • The results • Students’ understanding and exploring assumptions • Faculty demonstrations of behavioral theories, including topics such as gender, race, economics, freedom, labor • Examples: employment by wage and gender demonstration by faculty; case study for grade assignments

20. Results “It is about critical thinking, being able to take a lot of different perspectives that you read or hear and being able to synthesize them, and being able to form your own opinions. There is never a right answer, and that can be frustrating, but it allows you a lot of flexibility, to really develop what you learn, and question what you think, and why you think the way you do.” “When we leave here, we become part of a collective, and even within the university, we are part of a smaller collective. It is important to understand… how [things] work intrinsically.”

21. Results • Strengthen the Social Scientific • For a few students, the level of academic challenge was less than they expected or anticipated • Some students noted that newer faculty with less Puget Sound experience used a less engaging teaching style

22. Results • Connections • Our expectations • Interdisciplinary expertise of one faculty or collaboration of faculty • Develop an understanding of the interrelationships of fields of knowledge by exploring connections and contrasts between various disciplines with respect to disciplinary methodology and subject matter • Students engage by identifying multiple disciplinary approaches to a subject • analyzing the subject from these perspective • participating in cross-disciplinary dialogue • exploring the integration or synthesis of these approaches to foster understanding on the subject

23. Results • The results • Students found value in approaching a topic from multiple perspectives • Students applied this type of approach to other in and out of class experiences • Students appreciated cross disciplinary dialogue • Students were able to integrate knowledge based on bringing together multiple perspectives • Students relayed a feeling of this course acting as a capstone

24. Results “We have been trained to be students for such a long time, we start off really broad, and then we go into specific majors, and I think what Connections is trying to do is put you in the real world, intellectual stage, having lots of different people from different disciplines… at the same time, it starts discussion between different disciplines; professional students talking to professional students.” “I appreciate other disciplines so much more, having, for example, been [specific to] one discipline, and going into Connections; I realized an appreciation for the use [and discussion] of other disciplines.”

25. Results • Strengthen the Connections Core • While two thirds generally agree they are getting out of the courses what is expected, how do we strengthen that even further? • Some students reported not gaining depth and not being pushed out of their disciplinary comfort zone

26. Results • Committee posed the following questions • Is the feedback about Connections more favorable than in the past? • Are students taking particular Connections courses in order to fulfill a major, minor, or upper division requirement? • If yes, does this explain a lack of challenge for students?

27. Results • What next? • Use questions from committee to inform further assessment efforts • Use model with other campus partners

28. Questions? • Alanna Johnson, amjohnson@pugetsound.edu • Emily Mullins, emullins@pugetsound.edu • Kate Cohn, kcohn@pugetsound.edu • Ellen Peters, epeters@pugetsound.edu