1. The Online Academy(TOA) John Baek Michelle Didier Dan Feinberg Lisa Herbert Danalyn Robinson Heather White

2. Overview The project • An online high school • A partnership between GMU and Northern VA county school systems

3. Overview Review of last semester • Focus on performance analysis, design & development Mission Statement • Develop high quality courses, correlated to the Virginia SOL and COPLS model • Support the expert mentors

4. Overview • Redirection of focus • Focus this semester • Framework for continued development: code library, site structure, consistent design using style guidelines and templates • Field testing • Evaluation & usability testing • Complete the Intro module/ Resource guides

5. Overview • Our process • Team meetings • SME (teachers) meetings • Project management • Divide & conquer

6. Presentation Outline • Evaluation • Evaluation needs & goals • Methods • Findings & recommendations • Future of TOA • Framework for future development • Project recommendations

7. Evaluating TOA Need for Evaluation Methods of Evaluation Findings and Recommendations

8. Need for Evaluation • Inform and direct current and future development • Examine the usability of the checklist • Organization and self-regulation • Determine effectiveness of navigation • Locating resources • Intuitiveness—do the learners know where to go next? • Review the course modules • Clarity of instruction • Engagement and motivation of instruction • Adequacy of the resources, skills, and content for challenge • Pedagogical re-engineering (approach and format)

9. Evaluation Goals • Analyze instructional strengths • Determine the effectiveness of the design and transfer of learning • Evaluate the usability and the benefits of the checklist to the learner • Locate ease of use problems (navigation)

10. Methods of Evaluation • Quality assurance • Field testing • One-to-One evaluation • Expert review • Case study

11. Meeting the Evaluation Goals • Effectiveness of the design and transfer • Quality assurance • Case study • Expert review • Instructional strengths • One-to-one evaluation • Expert review • Case study • Checklist Usability • Expert review • Field test • Locate ease of use problems • Field test • One-to-one evaluation

12. Quality Assurance

13. For each course, at least two modules were field tested. • English 11 • The Two Macks: A Harlem Renaissance Club • Regionalism, Realism, and Nationalism • World History I • India and China • African Civilizations • World History II • Imperialism • Between the Wars Chemistry Mars Matters Sappy Solutions Earth Science Passing Gas…For Effect Only Like It…Weather or Not Algebra Graphing – Slopes Geometry Right Triangles/ Trigonometry Points, Lines, Planes, Segments, and Angles

14. Before anything was field tested, the modules were evaluated for Quality Assurance. • Forms that don’t work • Typing mistakes • JavaScript errors • Form elements not properly labeled • Pages that don’t print correctly • Consistent styles • Missing page titles or code comments

15. The result is a much more polished looking product.

20. Field Testing

21. Field Testing • 14 Modules • Technical problems • Student response • Self regulation • Synchronous discussion (Digichat)

22. Digichat • Encourages peer interaction • Encourages mentor-learner interaction • Mentor office hours

23. Intro Module

24. Intro Module • Need for Intro Module • Elements of most modules • Use basic TOA functions

25. Mentor Resources • Overview • Objectives • Answer keys

26. Challenge • Problem based context for entire module • Background building knowledge • Culminating activity

27. The Checklist • Self regulation • Mentor-student negotiation • Links to activities

28. TOA Introductory Module • Research for the module was gathered from several sources • Teachers’ feedback from their field testing • Results from one-to-one evaluation • Review of related literature • Maintain theme of the story introduced in the Challenge

29. Working with a Mentor

30. Working with a Mentor Activity

31. Being an Online Learner

32. Being an Online Learner Activity

33. Online Research

34. Online Research

35. Online Research

36. One-To-One Evaluations A face to face meeting with members of our target audience where we ask them to use TOA and give us their opinion on the site.

37. One-To-One Evaluations • Is the instruction engaging and motivating? • Will the student know the references? • Is the content at the appropriate level? • Does the checklist provide self-regulatory benefits? • Are the instructions clear? • Can the user locate resources? • Can learners navigate? • Are the resources, skills, and content adequate for completing the challenge?

38. One-To-One Evaluation Process • Recruit students. • Prepare evaluation questions. • Design data collection tools. • Write script. • Set up testing area. • Conduct evaluation. • Analyze results. • Write evaluation report.

39. 1. Recruit Students • We were provided access to two students at a Washington, DC private school • “Ralph” – A 16 year old white male currently enrolled in Chemistry • “Kelly” – A 16 year old white female who had taken Chinese History

40. 2.Prepare Evaluation Questions

41. 3. Design Data Collection Tools • Tape recorded and transcribed • Field notes were taken by interviewer and observer • Follow up questionnaire was given

42. 4. Write Script

43. 5. Set Up Testing Area6. Conduct Evaluation

44. 7. Analyze Results

45. 8. Write Evaluation Report