Course Development S tarts with Course G oals : Defining, Refining , Collection , and Analysis

1. Course Development Starts with Course Goals: Defining, Refining, Collection, and Analysis David W. Stockburger U.S. Air Force Academy, CO Faculty Orientation - 4 days 2005 2006 2007 2008 2009 2010 USAFA Outcomes To commission leaders of character who embody the Air Force Core Values and are committed to Societal, Professional, and Individual Responsibilities • Ethical Reasoning and Action • Respect for Human Dignity • Service to the Nation • Lifelong Development and Contributions • Intercultural Competence and Involvement empowered by Integrated Intellectual and Warrior Skills • Quantitative and Information Literacy • Oral and Written Communication • Critical Thinking • Decision Making • Stamina • Courage • Discipline • Teamwork grounded in essential Knowledge • Heritage and Application of Air, Space, and Cyberspace Power • National Security and Full Spectrum Joint and Coalition Warfare • Civic, Cultural, and International Environments • Ethics and the Foundations of Character • Principles of Science and the Scientific Method • Principles of Engineering and the Application of Technology Learning-Focus Initiative Begins Introduction For the last four years, the U.S. Air Force Academy (USAFA) has been in the process of moving from a teacher-centered to a learning-focused environment. This process has involved a coordinated effort of administration, academic assessment, faculty development, and departmental faculty members. In our implementation of the learning-focused environment, we have become convinced that course and lesson development must begin with clear and measurable goals. USAFA has created a database of course goals that was used to assess the extent to which the cultural shift has proceeded. This poster presents our journey and a preliminary analysis of our course goals. Faculty Orientation - 5 days Learning focus introduced to new faculty Outcome Teams Formed Inter-Departmental Four-year Assessment orientated Departmental Assessment Plans • What’s in the database? • The current course goals database has the submitted course goals for three semesters, Fall, 2008, Spring, 2009, and Fall, 2009. • The Fall, 2009 semester has course goal for 576 different courses with over 14,000 total words. • At USAFA there are four divisions: Engineering, Science, Social Science, and Humanities, with five departments within each division. Courses are located within a department. In addition, each course can be categorized as a core (required) or non-core course with a given level (100, 200, 300, or 400.) Please see handout for analysis of course goals using these variables. Committee to revise cadet feedback formed Faculty Orientation - 5 days using Ken Bain’s book , small groups, and experienced mentors Revised Online Cadet Feedback to Faculty Course developers workshop Course design workshop Leadville, CO – 3 days Faculty Orientation - 5 days Dr. Ken Bain presents in person. Goals Database Begins Course developers workshop Director of SoTL hired NCA Accreditation Visit Faculty Orientation - 6 days Course design workshops at Academy Course design workshop in Leadville, CO 3 days using Dee Fink’s book Goals Database Analysis Change in Word Frequency in Course Goals of Bloom’s Cognitive Domain Categories over Three Semesters and Core and Non-Core Courses Analysis of Course Goals A lexical analysis of the goals database was done to view changes in level of cognitive domain for core and non-core courses. In addition, responses to each USAFA outcome were organized into a report for outcome assessment (see handout). Conclusion/Discussion Little change in relative word frequency for different cognitive domains was observed over the three semesters, in part because past course goals are used as a default for the current semester. Cultural change occurs slowly, and these data will provide a baseline for future comparisons. Application and knowledge verbs were the most frequently observed words in the course database. Creation verbs were more frequently observed in non-core (majors) courses. Our goal is to move a greater proportion of verbs in the course goals to a higher cognitive level. Core Courses Non-core Courses Acknowledgements: Many individuals have contributed to this effort and the following list is necessarily incomplete: Dr. Steve Jones, Dr. Ken Sagendorf, Lt Col D. Brent Morris, Dr. Rolf Enger, Dr. Evelyn Patterson, Dr. Rich Hughes, and Brig Gen Dana Born