From Creating the Form to Analyzing the Results:

1. From Creating the Form to Analyzing the Results: Anatomy of a Campus-Wide Instructor Evaluation Julie E. Thomas, Ph.D. Central Texas College Jump!

2. Background • Items drafted by a faculty committee • Draft reviewed by IE office • Form approved by faculty senate Go to 7

3. Preliminaries • Download course listing • Delete classes with <10 enrolled • Exclude labs (except nursing, EMT) • Sort by department, then instructor • Send course list to department chairs to verify accuracy

4. Sample definition • Classes are sampled units • Random sample with restrictions • Instructors may switch with permission • Overall sample is slightly more than half of courses meeting criteria

5. Sample Selection • First stage: randomly select one class for every instructor • Second stage: randomly select a group from the remaining classes • Third stage: delete classes if more than three for an instructor; substitute courses where necessary

6. Issues & Accommodations • Course listings don’t tell the whole story! • Team-taught courses • Last-minute instructor changes • Special requests

7. Customizing the form • Teleform edit menu: Copy page to clipboard • Paste into MS Word as a picture • Overlay with text boxes • Insert merge fields in text boxes • Course listing file is used as data file for mail merge Go to 11

9. Other details • Mailing labels for the evaluation packets • Keep the same sort order! • Copies of forms will work, but try to avoid second-generation copies • Packets distributed by campus mail

10. Receiving Completed Surveys • Completed forms collected by department offices • Returned to IE office when all forms have been received by department • BOCTAE!

11. Scanning Forms • Excel file produced by Teleform • May also be set to generate other types of data files • Class ID is included on each form: used to match results with course listing file

12. Compiling & Analyzing Results • Statistical software such as JMP is very helpful • Data summary • One-to-many and many-to-one sorting • Can be done entirely in Excel! (not the most efficient way) • Trimmed means used for class summary data

13. Data Reported • A page of results in generated for each class, including for each item: • Class means (20% trimmed) • Frequency distributions (per class) • Comparison data

14. Comparison Data • Instructor means • Course means • Discipline means • Department means • Campus Means • All based on weighted averages of class trimmed means

15. Reporting • Mail merge again! • Automatically produces a one-page report for each class evaluated • To fix formatting, global find-and-replace commands are used

16. Individualized Reports • Each faculty member receives summary pages of his/her own classes • Department chairs receive the summary pages for classes in their departments • Complete report distributed to a very limited audience • “No-names” edition is more widely available

17. Custom Reports • Special requests can be accommodated as time permits • Standard deviations • Rank ordering • Graphical reports

18. QUESTIONS? • Julie.Thomas@ctcd.edu • Central Texas College • PO Box 1800 • Killeen, TX 76540 • 254 526 1484 • To view the latest “no-names” edition: http://www.ctcd.edu/ie/seci_spring2005.pdf