Computers in Human Behavior, 2006

1. The effect of previous experience with information and communication technologies on performance in a Web-based learning program Computers in Human Behavior, 2006

2. 1. Introduction • At universities, there is a growing trend in the number of blended learning projects that combine training with virtual training supported by the Web. • There are great expectations surrounding online training because of its versatility (it is appropriate for all types of contents or subjects), its flexibility (at time and in any place) and its personalization (it can be adjusted to meet the user’s requirements). • It is necessary to analyze how the roles of trainer and student are affected and to develop procedures to evaluate the impact of such changes on the efficiency of the learning processes.

3. 1. Introduction • Web-based learning requires greater effort, involvement and skill from the student when managing information (Hokanson & Hooper, 2000). • Learning concepts requires both carrying out the corresponding activities and being competent in information technology. • Students who are confident in their computing skills and suffer less anxiety when using computers are more likely to continue with the training course and to achieve better results (Osborn, 2001.

4. 1. Introduction • Not paying sufficient attention to these factors could explain the “failure” in training which results in students abandoning the programs. This is one of the main threats to the development of online training. • The lack of skill working in a Web environment causes the user to undergo negative experiences. • The phenomenon of being “lost in hyperspace” (Danielson, 2002; Otter, 2000), happens when a user gets to a point in which the structure of the hypertext has no meaning for him and he has no idea how to get to a place with which he is familiar (Aedo, Diza & Montero,2001). • Disoriented users find it extremely difficult to capture the global structure of the information, which means that they to continue to feel disoriented and this greatly hinders their understanding of the main content (McDonald & Stevenson, 1998; Rada & Murphy, 1992).

5. 1. Introduction • Although this is acknowledged as one of the greatest threats, little is known about the reasons for students dropping out. • Sometimes it is associated with low performance, which in turn indicates factors such motivation, self-efficiency, perceived control, ICT experience, and navigation strategies (Wang & Newlin, 2002). • Previous experiencein the use of ICT tools seems to be a differentiating factor between students who participate in Web-based training programs. • Researchers found that individuals with less previousexperience in the use of computers navigated in a more linear manner；individuals with more hypermedia experience navigated more nonlinearly (Ervin et al., 1995).

6. 1. Introduction • Some found that a negative relationship existed between experience with hypermedia and time on task within the hypermedia environment (Reed & Geissler, 1995; Reed & Oughton, 1997), in some instances, it is found that those users with more previous experience in using hypermedia programs spent more time using the hypermedia program which allowed them to navigate in a less linear way and to make more use of exploring the possible links (Kraus, Reed, & Fitzgerald, 2001). • Nevertheless, previous computer experience was unrelated to user performance on measures of learning through the hypermedia case study program (Fitzgerald & Semrau, 1997). • Taking these considerations into account, the present study analyze the behavior of a group users in a Web-based learning programtesting whether previous experience with ICT influences performance and online activity.

7. 2. Method • 2.1. Participants • 120 college students were enrolled in “Personnel Training and Development”, the course was designed to train the skills that the “new” professional trainers need to develop in order to apply ICT. • 2.2. The Course • The course included conventional (face-to-face) and Web-based sections. • All students were encouraged to take part in the online cyberclass about the role of ICT in personnel training. • Asynchronous online communication (forum, e-mail etc..) was built-in to the virtual environment. • Basic bibliographic references, Web links, FAQ’s and a glossary were supplied as additional resources.

8. 2. Method

9. 2.3. Procedure • The virtual classroom was open for 36 days without interruption in the first term of the course. • At the beginning of the Web-based course, students were given a survey asking them to indicate the frequency of their use of different ICT tools. • These data were useful to report on students’ experiences in new technologies and virtual courses. • At the end of the study, participants were asked to fill in a questionnaire in order to collect the user’s assessment of the Web-based course.

10. 2.4. Measures • 2.4.1. Previous experience • A global index of previous experience with a range of 0-1 was setup, taking into account the frequency of use of basic Internet tools (e-mail and browser) and previous participationin Web-based training projects.

11. 2.4. Measures • 2.4.2. Online activity • Logged data allowed us to compute five measures of online activity: - Total hypertext sessions: Total number of sessions - Total Web-site hits: Total number of Web pages visited - Average path length: Average number of Web pages visited per session - Total online time: Total time of online activity - Average path time: Average online time per session

12. 2.4.3. Performance • An index of classroom performance was created, based on the total number of activities carried out. A total of eight possible activities were proposed: - E-mail: Using Web browsers - Navigation: Using Web browsers - Forum: Participating in the electronic forum - Characteristics: Thinking about main features of using ICT in training projects - Models: Thinking about Web-based learning and teaching models - Examples: Visiting and testing other ICT training programs - Design: Designing an ICT training project - Assessment: Assessing an ICT training project • In order to pass the online course, students must carry out at least four activities correctly.

13. 3. Results • 3.1. Previous experience • The tool most frequently used by the participants is e-mail 60% of the participants regularly use e-mail (weekly or daily), 36.8% use it occasionally and 2.1% never use it. • The 24.5% of the participants navigate daily on the Internet, 27.7% do so once a week and about half of the participants never or almost never navigate on the Internet (47.9%). • Only 3.3% of the participants that they had previously visited a virtual classroom. • Applying the global indicator of previous experience in the use of basic ICT tools, with individual differences which allowed us to group them according to low (n=32), average (n=40) and high (n=22) previous experience.

14. 3.2. Online activity • Descriptive statistics about online activity are presented in Table 1.

15. 3.3. Performance • With respect to the number of activities carried out, 76.7% of the participants completed at least four of the eight proposed activities, which was the criteria used to pass the course. • The activity with least participation (59.2%) corresponds to ‘visiting and testing other ICT training programs’. • The activity most frequently carried out was the ‘e-mail activity’, completed by 72.5% of the participants.

16. 3.4. Online activity and performance • The analysis of the relationship between the navigation variables, online activity and performance shows (Table 2) that performance is directly related to the total time devoted to the learning program, the number of sessions carried out, and the total number of pages visited. • The indicators of work per session present, between them, high and significant relationships: the number of pages visited per session has a correlation of 0.76 with the time used per online session.

17. 3.4. Online activity and performance • The results show that among participants with higher performance levels, a greater number of Web-site hits and more total online time were registered (Table 3).

18. 3.4. Online activity and performance • The analysis of online activity measures and performance data, through the principal component analysis with rotation, shows two conceptual categories of online course activities: the work carried out (quantity) and the organization of the work (how). 75.3% of the variance is accounted for by these two components (Table 4). I. The work carried out (quantity) II. The organization of the work (how)

19. 3.4. Online activity and performance • Performance in the virtual classroom is significantly related to the amount of effort and work invested: the more pages visited and more time spent in online activity, the greater the effort and the better the results. I. The work carried out (quantity) II. The organization of the work (how)

20. 3.4. Online activity and performance • A relationship between previous experience in the use of ICT tools and the factor of organization of the work in the classroom (r=-0.28;p=0.001) • People with greater previous experience in the use of ICT tools organized their work in a different way to users with less time; more previous experience used less time and less visits to pages per work session. • Moreover, the lack of significant relationship between the variables of previous experience and performance, indicate that the results obtained or the previous experience in the use of ICT tools, do not influence users’ positive learning performance of the virtual classroom.

21. 4. Conclusion • The results of the study indicate that the difference between students with low and high previous experience in the use of Internet tools does not lie in the amount of activity carried out during the training program. • Thus, users with greater experience distribute their work in more sessions of shorter duration and visit fewer pages per session, without this meaning less amount of total work carried out.

22. 4. Conclusion • These results show that although users with more experience with Internet tools have a different pattern of activity, this do not influence their performance at the program. • Good results with respect to performance and satisfaction with the learning experience is a successful combination, which should lead to a reflection on the need for participants in Web-based learning programs to have a basic level of computing skills before joining a Web-based training program.