Evaluation Report

2. Evaluation Report A preliminary report to the faculty and administrators of the online distance learning program in the Department of Educational Research in the College of Education at Florida State University. April, 2002 “There is no royal road to learning” Euclid, 300 B.C.

3. Agenda • Evaluation Team • Description of the Students • Procedures Used • Results • Conclusions • Preliminary Recommendations • Reactions – Discussion and Written

4. Overall, How Would You Rate the Program? Overall, How Would You Rate the Curriculum? Overall, How Would You Rate the Faculty? M = 3.29 SD = .68 M = 3.29 SD = .72 M = 3.49 SD = .68 Key Evaluation Results (4 = Excellent)

5. I. The Evaluation Team

6. Michael Simonson, Ph.D. Evaluation Coordinator

7. Judy Converso, Ph.D. Comparisons With Similar University Programs

8. Maggie Crawford Information Specialist Telephone Interviews

9. II. The Students

11. Ages of Students Responding Median = 41

12. Number of Online Courses Taken + Taking Median = 6

13. Hours/Week Studying – One Course Median = 10 hours/week

14. III. Procedures • Student Survey (online) • Telephone Interviews • Faculty Interviews and Meetings • Literature Searches and Program Comparisons • “Lurking”

15. IV. ProceduresV. Conclusions

16. 1. Student Survey Response = ~70%

17. 1. A. Are You Satisfied with Your Courses and Curriculum? Not SatisfiedMean • Grading Requirements? 1 3.3 • Required Course Content? 1 3.2 • Required Course Selection? 1 3.2 • Course Difficulty? 1 3.2 • Elective Course Content? 10 3.1 • Relevance of Courses ? 2 3.1 • Online Teaching? 2 3.0 • Elective Course Selection? 10 2.9 N = 41 4 = Excellent

18. 1. B. Are You Satisfied with Your Instructors? Not SatisfiedMean • Content Knowledge? 1 3.6 • Feedback? 3 3.2 • Skills at Online Teaching? 1 3.1 • Interaction? 3 2.9 N = 41 4 = Excellent

19. 1. C. Are You Satisfied With? Not SatisfiedMean • Number of Required Courses? 2 3.4 • FSU Technical Support 4 3.2 • Prep. For Profession? 6 3.1 • Portfolio Experience? 10 3.1 • Number of Electives? 5 3.1 • FSU Library Support? 9 2.9 N = 41 4 = Excellent

20. 1. D. General Questions: How Would You Rate: Mean • Faculty? 3.5 • Program – Overall? 3.3 • Curriculum? 3.3 • FSU’s Support? 3.3 • Texts and Printed Materials? 3.1 • Program Support Services 3.0 N = 41 4 = Excellent

21. Open-Ended Questions-Student SurveyWhat Improvements Would You Recommend? • More electives • More teaching and less discussion • More courses each semester • Improve Blackboard • Eliminate discrepancies in course policies

22. Open-Ended Questions-Student SurveyWhat Do You Like Best? • Content • Faculty • Reputation of the program • Freedom • Flexibility • Relevance

23. Open-Ended Questions-Student SurveyWhy Is Your Favorite Course Your Favorite? • Quality interaction with the instructor • Excellent professor • Interesting Content • Various instructional Strategies used • Relevant content

24. Open-Ended Questions-Student SurveyHow Likely Would You Be To Choose An Online Program, Given Free Choice? Almost evenly split between those who prefer face-to-face and online learning

25. Open-Ended Questions-Student SurveyOther Comments? • Do something about Blackboard • Offer courses in the summer • Provide more interaction with faculty

26. 2. Telephone Interviews • 10 Students Called ~ Randomly • 5 Females and 5 males (30 Females/18 Males) • Age = ~45 • 15 – 45 Minutes Each • 3 Questions

27. Summary Statements – Telephone Interviews Why did you enroll? • Reputation of the program • Convenience • Content • Skill Building

28. Summary Statements – Telephone Interviews Major Program Strength? • Faculty • Organization • Flexibility • Students

29. Summary Statements – Telephone Interviews What Changes Would You Recommend? • Get a better system than Blackboard • Improve consistency of policies between courses • More immediate feedback • Help us connect with other students • Be realistic about time commitments • Consider synchronous instruction

30. 3. Faculty Interviews • 7 Interviews • Face-to-face and Telephone • 30 – 60 Minutes

31. Summary StatementsFaculty Interviews • Instructional Systems Design is the key to the program • Rigor and quality are critical • The program is very labor intensive • Synchronous instruction should be considered • Various instructional platforms should be available • Identify program benchmarks • Provide training in distance education for faculty • Support for faculty is needed – graduate assistants and local production equipment • Camaraderie among students is important – add synch. • 3 faculty promised, only one hired

32. 4. Literature & Program Review • 11 programs reviewed • Titles of courses compared • Similarities and differences noted

33. VI. Preliminary Recommendations Policy & Management Faculty & Curriculum Instruction & Delivery

34. RecommendationsPolicy & Management • Tuition and Fees • Program Policy Statements • Incentives and Involvement of Faculty • Entrance Requirements – Skills AND Resources • 5-Year Plan

35. RecommendationsFaculty & Curriculum • Move from text-based instruction to media • Explore various management platforms in addition to Blackboard • Review the capstone experience • Consider specializations w/in the degree • Involve full-time faculty

36. RecommendationsInstruction & Delivery • Offer Summer courses • Increase selection of online materials • Consider use of adjuncts, supervised by full-time faculty • Secure local production resources for faculty • Offer synchronous instruction, and consider face-to-face activities

37. VII. Reactions