Multi-Section Assessment of One Course: English 407A

1. Multi-Section Assessment of One Course: English 407A Ed Nagelhout Julie Staggers Department of English

2. Presentation Goals • Describe steps for assessing multiple sections of a single course • Use English 407A: Business Writing as an extended example • Offer recommendations for other multi-section courses • Answer Questions

3. Eight Steps • Think Programmatically • Select Outcomes • Develop Tools • Simplify Evaluation Measurements • Construct Report Template • Examine Findings • Improve Teacher Support Structure • Review/Evaluate Assessment

4. 1. Think Programmatically Every student should have basically the same experience in every (multi-section) course regardless of teacher or platform. • Coherence • Consistency • Project-Centered • Pedagogical Emphasis

5. English 407A: Business Writing 70-75 Sections per year Taught primarily by PTIs, GAs, and Adjuncts Business Writing Course Delivery Platforms: • Traditional Face-to-Face sections • Hybrid sections • Distance Ed sections

6. English 407A: Business Writing To Meet Programmatic Goals • Common Syllabus • Common Outcomes • Common Projects • Mentoring Program • Teacher Support • Teacher Choice

7. 2. Select Outcomes How much can we legitimately do in 16 weeks? • Determine what’s important • What do you want them to know? • What do you want them to do? • Promote primary and secondary objectives • Be explicit • Maintain focus

8. English 407A: Business Writing Common Outcomes • Write to multiple audiences, for various purposes • Plan and manage short- and long-term writing projects • Draft, design, revise, and edit documents • Design and implement appropriate research strategies • Write collaboratively (e.g., co-authoring, peer revising) • Follow and adjust to business writing conventions • Design documents for both content and visual appeal • Use writing to help prepare and deliver oral presentations • Write ethically and responsibly

9. 3. Develop Tools Tools should be designed to determine the achievement of outcomes as effectively and as efficiently as possible • Think programmatically • Align explicitly with outcomes • Use a range of tools

10. English 407A: Business Writing • Quizzes • Exercises • Introductory Project (2) • Case Project (4) • Employment Project • Major Collaborative Project (3) NOTE: Each project includes multiple deliverables and a project assessment memo

11. English 407A: Business Writing • Primary Criteria • Project-Specific Criteria

12. 4. Simplify Evaluation Measurements Evaluation measurements provide the data to help you determine the success of the course • Take cut/paste mentality • Allow for multiple lenses • Provide different ways into the data

13. English 407A: Business Writing Common Outcomes • Write to multiple audiences, for various purposes • Plan and manage short- and long-term writing projects • Draft, design, revise, and edit documents • Design and implement appropriate research strategies • Write collaboratively (e.g., co-authoring, peer revising) • Follow and adjust to business writing conventions • Design documents for both content and visual appeal • Use writing to help prepare and deliver oral presentations • Write ethically and responsibly

14. English 407A: Business Writing • Primary Criteria • Project-Specific Criteria

15. English 407A: Business Writing

16. English 407A: Business Writing

17. English 407A: Business Writing

18. English 407A: Business Writing

19. English 407A: Business Writing

20. English 407A: Business Writing

21. 5. Construct Report Template Reporting the data in a way that will allow you to improve your course is paramount, so you need to be proactive • How do you report the data? • Where do you report the data? • Why do you report the data? • Create template for application over time

22. English 407A: Business Writing • English 407A Annual Report • Department Queries • College Queries • University Queries

23. 6. Examine Findings Examine findings through the lenses you find most applicable • Student Learning • Teacher Effectiveness • Teacher Support • Program Effectiveness

24. English 407A: Business Writing • Preliminary Findings • Analyze by Outcome • Analyze by Criteria for a Single Project and across Projects • Analyze by Platform • Analyze by Section and across Sections

25. English 407A: Business Writing Sample Findings Pilot Study – Spring 2008 – 7 Sections Outcome: “Design documents for both content and visual appeal” Criteria: Format: All projects; All sections • Overall Average = 4.23 • Low Average = 3.98 (Case Project) • High Average = 4.39 (Employment Project)

26. 7. Improve Teacher Support Structure Strong teacher support structures are imperative to successful course delivery and assessment • Teacher Effectiveness • Teacher Efficiency • Teacher Workshops • Teacher Materials

27. English 407A: Business Writing 10-Hour per Week Commitment • If four courses is considered full-time, then no teacher should spend more than 10 hours per week per course. • This includes time in class, time in office hours, time preparing for class, time responding / evaluating student writing, and time in training. • As a program, our primary objective is to help teachers meet this expectation.

28. English 407A: Business Writing Teacher Workshops • Classroom Activities • Responding Effectively AND Efficiently to Student Writing (Egg Timer Workshop) • Norming Sessions • Focus Groups • Staff Meetings

29. English 407A: Business Writing Teacher Materials • Course Materials • Syllabus, Schedule, Evaluation Sheets • Handouts • Project, Pedagogical, Process/Product • Classroom Materials • PPT Slides, Samples, Activities, Mini-Lectures

30. 8. Review/Evaluate Assessment • Why are we doing assessment? • Are we achieving our goals? • Are the outcomes appropriate? • Are we improving our course? • Are we improving our teaching? • Are we looking at the right things? • What can we do differently? • What can we do better?

31. English 407A: Business Writing Program Recommendations • Revise Project Assessment Memos • Incorporate Focus Groups • Update/Revise WebCampus • Review/Revise Outcomes • Review/Revise Criteria • Improve Methods • Assessment and Pedagogical

32. Conclusions/Recommendations • Design to fit your needs • Go slowly • Make incremental progress • Think long-term • Imagine reduced teacher workload and improved student learning • Questions?