Sang Chan Weber State University Jane Strickland Idaho State University

1. Examining the Multimedia Redundancy Effect among Education Majors:Assessing Pedagogical Usability of Instructional Videos Sang Chan Weber State University Jane Strickland Idaho State University • AACE, • New Orleans, LA October 29, 2014

2. Introduction • Many studies (e.g., Craig, Gholson, & Driscoll, 2002; Kalyuga, Chandler, & Sweller, 1999, 2000; Mayer, Heiser, & Lonn, 2001, etc.) compared redundant groups (graphics, text, and narration) with non-redundant groups (graphics and narration). • Results: Non-redundant group significantly outperformed the redundant group.

3. Introduction • Used a short treatment time • Measured only short-term learning • Rarely used a topic of a formal course • Rarely discussed pedagogical usability

4. Purpose • To further examine the redundancy effect by comparing posttest scores, delayed posttest scores, and usability perceptions of education majors enrolled in an online technology class. • Redundant (R): Narration, graphics, & text • Non-Redundant(NR): Narration& graphics

5. Research Questions • Is there a significant difference in posttest scores in learning Microsoft Access 2013 between R and NR? • Is there a significant difference in delayedposttest scores? • Isthere a significant difference in pedagogical usability perceptions?

6. Theoretical Framework • Cognitive load theory

7. Participants • The study used convenience sampling. • 42 students participated in the study. • Female, traditional students (majority) • No students with disabilities • Native English speakers

8. Research Design • T1: Narration, graphics • T2: Narration, graphics, and text

9. Materials • Designed and developed the videos • Developed a posttest and delayed posttest • Adapted the pedagogical usability survey • Piloted with a small group of four

10. Procedure • Obtained the informed consents • Assigned students randomly • Administered treatments • Completed surveys (αR= .83, αNR= .88) • Took posttest • Took delayed posttest (6 weeks later)

11. Data Screening • Little’s MCAR, χ2 (252) = 263.08, p = .303 • Test scores • Violated normality & equal variances for the independent-t test • Outliers in test scores • Survey responses: • Met the independent-t test assumptions

12. Data Analysis • Posttest and delayed posttest • Mann-Whitney U(adjusted α= .025) • Survey • Independent t-test (α = .05) Note: We found similar results for parametric and non-parametric tests with and without outliers.

13. Result: 1 • No significant difference between the two treatments on posttest scores, U = 164.50, p = .489 • Small estimated effect size, r = Z/ = .11

14. Result: 2 • No significant difference between the two treatments on delayed posttest scores, U = 118, p = .538 • Small estimated effect size, r=.11

15. Result: 3 • No significant difference between the two treatments on survey responses, t(40) = .60, p = .549 • Small estimated effect size, d = .18

16. Discussions • No significant differences in posttest and delayed posttest • Not consistent with previous studies (i.e., Craig, Gholson, & Driscoll, 2002; Mayer, Heiser, & Lonn, 2001) • Consistent with McNeill, Doolittle, and Hicks (2009) and Wu (2011)

17. Discussions • Useful text: Short and easy-to-read with slow narration • “The text was so helpful in understanding the content. I have never used access [sic] before & [sic] I think it can be helpful.” • “. . . The visuals were great and i appreciate having the subtitles to keep me on track.” • “Seeing you perform the task but also having the words on the bottom helped reinforce the idea.”

18. Discussions • Read text to reinforce narration. • Eight words, on average, per line • Processed the elements in different orders. • Anecdotally,text was useful.

19. Discussions • Insignificant results on posttest and delayed posttest may result from: • Small sample size • Low power • High data variability • Low motivation • Prior preparations before tests

20. Discussions • Useful aspects of the videos: • Review what was learned: “… Reviewing after each topic helped reinforce the concepts.” • Clear content and logical sequence: “It is useful that the objectives built on one another. I also thought it was helpful to see what the end product would look like.” • Multimedia elements: “I really liked the reading at the bottom of screen so I could follow along with the speaker.”

21. Implications • Present the text slightly faster (1 or 2 seconds) than narration (120 words/minute) • Keep the text available on the screen with graphics although the narration was silent • Keep the text short (8 words) and easy to read (large font)

22. Recommendations • Investigate such use of text • Explore ways to use text as a supporting element • Use a large sample size • Use a more reliable instrument to reduce data variability • Consider motivation: Low motivation may result in high data variability.

23. Thank You!Q & A

24. Revising Survey • Primary analysis: 6 subscales (75.63%) • Low reliability for some subscales • Low correlations among items • Conceptual overlapping • Unstable factor • Revised survey suggested 5 subscales.

25. Exploratory Factor Analysis

26. Exploratory Factor Analysis

27. Exploratory Factor Analysis

28. Revised Survey • Factor 1: Design Considerations (α = .84) • Factor 2: Learning Confidence (α = .96) • Factor 3: Learning Format (α = .86) • Factor 4: Learning Motivation (α = .87) • Factor 5: Multimedia Elements (α = .78) • Overall reliability α = .89 • Total variance = 75%.