1 / 24

Testing the Impact Assessment Framework in Lisbon Municipal Libraries Network

Libraries for Lifelong Learning. Testing the Impact Assessment Framework in Lisbon Municipal Libraries Network. Leonor Gaspar Pinto Library Management Division Lisbon Municipality PORTUGAL. The agenda. Lisbon Municipal Libraries Network: a brief introduction

kirk-underwood
Télécharger la présentation

Testing the Impact Assessment Framework in Lisbon Municipal Libraries Network

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Libraries for Lifelong Learning TestingtheImpactAssessment Framework inLisbon Municipal LibrariesNetwork Leonor Gaspar Pinto Library Management Division Lisbon Municipality PORTUGAL

  2. The agenda • Lisbon Municipal Libraries Network: a brief introduction • Testing the Impact Assessment Framework • Test 1 – Basic ICT Learning Skills • Test 2 - Story Telling • Lessons learnt

  3. LISBON Municipal Libraries Network 17 libraries – 1 central library, 10 branch libraries, 3 special libraries, 2 mobile libraries and 1 public garden service point First municipal library = 1887 The Network • LISBON, the capital city of Portugal S. Lázaro Library

  4. LISBON Municipal Libraries Network To respond and anticipate users’ information and knowledge needs; Incentive them to use the information recourses in a Lifelong Learning perspective; Promote reading and information literacy as basic competencies for an active citizenship; Give them the necessary support in leaser activities. L M D Goals

  5. Testing the Impact Assessment Framework Selection of 2 Different Activities • Basic ICT Learning Sessions (ULISSES Programme) • Adults and elders • “Give me a Book… I will tell you a story” • Reading promotion initiative: story telling • Children, Primary School classes

  6. Pre-testing: methodological aspects • Study the framework (GLO + summary of the Framework) • Portuguese version / adaptation of the Framework Testing the Impact Assessment Framework • 5 Learning Areas Knowledge & Understanding Attitudes & Values Skills Activity, behaviour & progression Enjoyment, Inspiration & Creativity • 5 Steps 1)Developing your Story 2)Selecting SMART Indicators 3)Deciding on the right Data Collection Methods 4)AnalyseResults5)Presentand Use Findings

  7. Testing the Impact Assessment Framework Basic ICT Learning Sessions • ULISSES Programme Aims • To reinforce the role of Municipal Libraries as places for Lifelong Learning, as well as privileged access points to information. • To help users developing the competencies they need to use information in an effective and critical manner. • To promote users’ information literacy competences, improving their ability to access, use and evaluate information in every format. • To contribute to the development of computer basic skills among library users and citizens in general. Named after Ulysses, Homer’s hero Lisbon = Olissipo, the city of Ulisses

  8. Testing the Impact Assessment Framework 524 users/learners 103learning sessions 226leaning hours 27 elements of Library Staff directly involved 52 computers used Cited inEducating Cities International Documents Databank http://w10.bcn.es/APPS/eduportal/pceVeureAc.do ULISSES Programme Outputs - 2008

  9. Testing the Impact Assessment Framework Basic ICT Learning Sessions • Step1)Developing your Story • Retrospective testing • ULISSES Programme available performance data • Time scope: February-December 2008 • 5 Libraries • At least 10 Sessions • At Least 50 Participants • Expected Outcomes • Basic ICT competences development • Comprehension of sessions content • Users/learners satisfaction • Change in users/learners attitude and behaviour towards ICT

  10. Basic ICT Learning Sessions Testing the Impact Assessment Framework • Step 2)Selecting SMART Indicators • Step3)Deciding on the right Data Collection Methods Knowledge and Understanding Ex. Outcome Indicator Collection Method At least half of the participants will understand the sessions contents Nº of participants that point a level 2, in a scale from 1 (min) to 3 (max.), in the skills framework of each learning Module Skills Evaluation Form

  11. Testing the Impact Assessment Framework Basic ICT Learning Sessions • Step3)Deciding on the right Data Collection Methods • Interview Guide • Control Sheets • Enrolment Forms • Learning Resources Evaluation Form • Skills Evaluation Form

  12. Testing the Impact Assessment Framework Basic ICT Learning Sessions • Step4)AnalyseResults Outcomes 5 Libraries involved 70 participants involved 15 Sessions monitored Per Generic Learning Outcome Knowledge and Understanding • 90% of the participants consider they reached a level 3 in terms of competences acquisition... • This means: participants understood all the contents (although they feel they need more time to practice)

  13. Testing the Impact Assessment Framework Basic ICT Learning Sessions Per Generic Learning Outcome Skills • 66% of the participants indicated a level 2 for all the skills listed per Module... • This means: all skills were acquired by the participants, although they felt still insecure and needed more practice Attitudes & Values • % of participants that pointed a level 3 - GOOD (Scale 0 to 4) in the following items: • 64,1% – Sessions structure (contents, duration, objectives) • -87,2% – Resources Evaluation (Facilities, Equipments) • 94,9% – Staff Performance (communicating skills, group dynamics...) • 41% – Participants’ Self-Evaluation (skills acquisition, participation)

  14. Testing the Impact Assessment Framework Basic ICT Learning Sessions Per Generic Learning Outcome Activity, behaviour & progression • A panel of 7 participants were interviewed about habits and routines or significant changes in their every day life, with the following results: • A) • 5 bought a Computer; • 5 are now connected to the Internet; • 2 consider having more chances to get a new job or a part-time job. • B) • 7 use the computer more frequently and with more confidence; • 7 continued the learning process.

  15. Testing the Impact Assessment Framework Basic ICT Learning Sessions • Step5)Presentand Use Findings “I have bought a computer with Internet connection. Perhaps, I can find a part-time job this time.” Testimonials “It is very nice to be able to send and receive E-mails or Chat with my grandchildren living abroad.” ”

  16. Testing the Impact Assessment Framework Story Telling • ReadingPromotionProgramme Aims • To consolidate and expand the social and educational role Lisbon Municipal Libraries of Lisbon play on the development of reading habits and information literacy skills. • To strengthen the external image of Lisbon Municipal Libraries as information and knowledge hubs and reading promoters. Action Lines • I – Promotion of Reading and Library Collections • II – Promotion of Information Literacy • III – Promotion of Citizenship

  17. Testing the Impact Assessment Framework Story Telling • Step1)Developing your Story • “Give me a book... I will tell you a story”: a reading promotion activity targeted at children (primary school classes) • Sample • 3 municipal libraries, 50 participants (47 children, 3 teachers/facilitators), 3 sessions • Expected outcomes • The activity promoted reading skills and habits of children • Children understood the story and the characters’ roles • The activity stimulated children's fondness of reading • Children found the activity fun

  18. Testing the Impact Assessment Framework Story Telling • Step 2) Selecting SMART Indicators • Step3) Deciding on the right Data Collection Methods Skills Ex. Outcome Indicator Collection Method Nº of teachers/ facilitatorsthat point a level 2, in a scale from 0 (min) to 4 (max.) in the item “Contribution to reading skills improvement” At least ¾ of the teachers/facilitators think the activity helped to improve children reading skills Teacher/ facilitator Evaluation Form

  19. Testing the Impact Assessment Framework Story Telling • Step3)Deciding on the right Data Collection Methods • Adapting the collection methods to the target audience • Redefining the “story”… • Groups of children participating in summer-camps • Different age groups • Accompanied by summer-camp facilitators

  20. Testing the Impact Assessment Framework Story Telling • Step4)AnalyseResults Outcomes 3 Libraries involved 47 participants involved 3 Sessions Monitored Per Generic Learning Outcome Knowledge and Understanding • 79,5% of the children referred level 3 (maximum conformity in a scale 1 to 3) in relation to the item “Understand the story and the characters” • All adults considered the activity contributed to create habits and develop reading competences

  21. Testing the Impact Assessment Framework Story Telling Per Generic Learning Outcome Skills • Every adult considered the activity useful to improve reading habits and competences (a rating of 3, in a 0 to 4 scale) Attitudes & Values • 72,7% of the children were very satisfied with the activity (a rating of 3, in a 1 to 3 scale, in terms of facilities, the story and the characters, the complementary activities and the story teller performance) • All adults were very satisfied with the activity (a rating of 3, in a 1 to 3 scale, in terms of facilities, the story and the characters, the complementary activities and the story teller performance)

  22. Testing the Impact Assessment Framework Story Telling Per Generic Learning Outcome Enjoyment, inspiration, creativity • 61,3% of the children considered the activity was very fun (a rating of 3, in a 1 to 3 scale) • All adult considered the activity useful to improve children's fondness of reading (a rating of 3, in a 0 to 4 scale) Activity, behaviour & progression • 47,7% of the children were willing to read more stories after the activity (a rating of 3, in a 1 to 3 scale)

  23. Lessons learnt Selection of 2 Different Activities • Basic ICT Learning Sessions • “Give me a Book… I will tell you a story” • It was easier to use the IAF in Test 1 than in Test 2 • The more structured an activity is in terms of its planning and evaluation methods and routines, the easier the use of IAF. • The less pedagogical structured the activities are, the bigger the effort needed to (re)think and frame them into IAF • All learning initiatives and activities in libraries should be goal-oriented.

  24. The ENTITLE Impact Assessment Framework TEST TEAM We welcome any comments Learning Promotion Service spa.dgb@cm-lisboa.pt

More Related