Perception and Reality:

1. Perception and Reality: Creating a Baseline for Longitudinal Study of Information Literacy Skills at the Outset of a High School Laptop Program Roberta Jordan, Presenter Glickman Library, University of Southern Maine December 3, 2010

2. What’s the premise? PERCEPTION V. REALITY

3. Perception: I am able to use the terms AND and OR correctly in my searching.

4. Check one: _____ DO NOT UNDERSTAND _____ NEVER _____ SOMETIMES _____ FREQUENTLY _____ ALWAYS

5. 62% of surveyed students expressed high confidence (i.e. answered “frequently” or “always”) in their ability to use Boolean operators in an information search.

6. Reality: Which of the following would retrieve the MOST results in an on-line search? a. Stars NOT planets b. Stars OR planets c. Stars AND planets d. Stars INSTEAD OF planets e. Don’t know.

7. Just under 13% of students who took a follow-up information literacy “skills” test chose the correct answer (b) to the multiple choice question on Boolean operators.

8. Perception v. Reality = BASELINE The difference between the two numbers (62 versus 13 percent) constitutes a realistic baseline against which to measure gains in student achievement.

9. PROJECT BACKGROUND

10. My Interests • Information Literacy/Skills Testing as a means to inform instruction • Real-world application for capstone project for my M.L.S.

11. An Opportunity November 2009: 700 students in grades 9 through 12 at Morse High School received laptops for school and at-home use through the Maine Laptop Technology Initiative (MLTI). Approximately 65 staff members also received laptops the year before. Commitment by R.S.U. 1: $750,000+ over four years Goal: Be a successful participant in a pilot program in selected high schools; support attainment of Maine Learning Results, and “help prepare students for a future economy that will rely heavily on technology and innovation.” (Task Force on Maine’s Learning Technology Endowment , 2001, p. vi).

16. The Challenge -Document student skill levels at the program start; -Demonstrate increased student achievement attributable to laptop use (the “holy grail” of research efforts on 1:1 laptop use) to help justify the community investment.

17. Project Purpose Develop tools for collecting baseline data on information literacy skills of Morse High School students at the outset of 1:1 laptop initiative.

18. Why information literacy is a potentially good curricular area for demonstrating achievement: -a subject that integrates use of technology and other important academic skills (research, synthesis); -ample evidence suggesting student skills are lacking in this area; -there are widely accepted standards that guide instruction against which to measure improvement; -there are cost-effective ways to collect data.

19. Two Tools Were Developed Over the Course of the Project: • A “Perceptions” Survey • -intended to provide a snapshot of student and teacher confidence in their ability to integrate technology, research, and information-seeking skills • An Information Literacy “Skills” Test • -intended to measure how much students’ perceived skills matched up in reality.

20. PERCEPTIONS SURVEY Based on the work of Morrisette (2007) and Grimble and Williams (2004), the MHS perceptions survey asks students to express their confidence levels in five areas relating to information-seeking and technology. Point Access Skills Research Skills Database Skills Internet Skills Technology Skills

27. Student Participants: Total Number Completing Survey: 299 Male Respondents: 133 Female Respondents: 166 Totals by Class: Grade 9: 68 (36M, 32F) Grade 10: 84 (39M, 45F) Grade 11: 80 (30M, 50F) Grade 12: 67 (28M, 39F ) Faculty Participants: • Total Number of Respondents: 23 • Male Respondents: 10 • Female Respondents: 13

28. Table 3. Frequency of Student and Faculty Survey Responses “Frequently” plus “Always” and Frequency of Student and Faculty Skill Assessments “Intermediate” plus “Advanced”

33. Table 5. Comparison of Student and Faculty Aggregated “High Confidence” Responses (Ranked from highest to lowest.)

34. Table 6. Comparison of Aggregated High-Confidence Responses By Skill Category and Grade Level

35. Most frequently reported uses of the Internet: -The most frequently reported uses by students were e-mail, Facebook, search engines, and watching videos (YouTube, Hulu, DVDs.) -For faculty, e-mail, news, weather, search engines, and on-line banking. -Both faculty and students indicated more frequent use of search engines over databases when seeking information or doing research.

36. Developing the Skills Test A pool of 46 skills test questions was assembled from a variety of existing tests, including those used by the Wooster School Library, Carleton College, Bowdoin College, Wichita State University, TRAILS (Kent State University), and the University of Maryland, Baltimore County (UMBC).

37. Other Valuable Sources for Test Questions

38. Question Selection: -School technology committee members (including the school librarian) were asked to select 25 to 30 questions that: -were of an appropriate level of difficulty; -matched well with items from the perceptions survey; -represented a balance of questions relating to and testing ACRL or AASL information literacy standards. -School librarian, library aide assisted in final question selection (25 questions total). - Version one of the skills test was piloted with 37 students (using Survey Monkey); final test reduced to 20 questions.

39. Student Participant Profile Maximum Respondent Pool*: 302 students Grade Levels: Grade 9: 109 Grade 10: 91 Grade 11: 30 Grade 12: 72 Gender: Male: 138 Female: 163 No Response: 1 *A pilot survey of 26 questions was administered to 37 student respondents. Based on the feedback on the pilot test, the skills test was revised and shortened to twenty questions, which was administered in two separate surveys of ten questions each. The tests were administered on-line, through Survey Monkey. 302 students participated in the first part of the skills test; 202 students participated in the second part of the test. The demographic and background information (above) was taken from those who participated in the first part of the skills test.

40. Students were also asked about: Their Source of Knowledge of Library Resources: • A course or project in which a teacher scheduled a class with a librarian: 198 • Self-taught: 129 • Sought help from librarian on my own:57 • Reference librarian at a public librarian:47 • None of the above: 23 Library Resources Used in the Last Six Months • Electronic Library Catalog 99 • MARVEL 92 • Student Resource Center Gold 76 • SIRS Online 37 • Infotrac 15 • Skipped the question 100

41. Perception Versus Reality: Baseline Comparisons

42. POINT SOURCE SKILLS: PERCEPTION I feel competent using the on-line library catalog to search for books. 41.1% of students gave high-confidence responses on this statement.

43. POINT SOURCE SKILLS: REALITY • The screen shot, above, is the search page for the Morse High School’s library catalog. If you were to hit the “Search” button using the terms shown, the list of search results would be: • a. A list of magazine and journal articles about both Germany and World War II • b. A list of different web sites about both Germany and World War II • c. A list of databases with information about both Germany and World War II • d. A list of books in the Morse High School library about both Germany and World War II. • e. Don’t know.

44. REALITY 37.4% (113) of students chose the correct response for this question. 26.5% (74) of students answered incorrectly. 15.9% (48) of students said that they did not know the answer. 22.2% (67) of students skipped the question. Comparison of Perception and Reality: 41.1% were confident about their ability to use the OPAC. 37.4% answered the corresponding skills test question correctly.

45. RESEARCH SKILLS: PERCEPTION I am able to tell the difference between a primary and secondary source. 60.9% of students gave high-confidence responses on this statement.

46. RESEARCH SKILLS: REALITY • Which of these items relating to Walt Whitman is the best example of a primary source? • a. A biography of Walt Whitman • b. A recently published book of letters Whitman wrote to friends and family • c. A Library of Congress collection of Whitman’s personal notebooks • d. A book of critical analysis about his poetry. • e. Both (b) and (c).

47. REALITY: 36.1% (109) of students chose the correct response for this question. 28.8% (87) of students answered incorrectly. 11.6% (35) of students identified one of the two primary sources. 23.4% (71) of students skipped the question. Comparison of Perception and Reality: 60.9% were confident about their ability to identify primary sources. 36.1% answered the corresponding skills test question correctly.

48. DATABASE SKILLS: PERCEPTION I am able to perform a successful search in SIRS online. 28.1 % of students gave high-confidence responses on this statement.

49. DATABASE SKILLS: REALITY • The Morse High School Library database that would be the best source of information for an in-class debate on the Iraq war is: • a. Student Resource Center Gold • b. Infotrac • c. MARVEL • d. SIRS online • e. Don’t know

50. REALITY • 5.9% (12) of students chose the correct response for this question. • 45.5% (92) of students answered incorrectly. • 31.2% (63) of students said they did not know the correct answer. • 17.3% (35) of students skipped the question. Comparison of Perception and Reality: 28.1% were confident about their ability to use SIRS online successfully in a search. 5.9% answered the corresponding skills test question correctly.