MEASUREMENT OF PROCESS EFFECTIVENESS

MEASUREMENT OF PROCESS EFFECTIVENESS THE USE OF DYNAMIC INDICATORS IN MONITORING AN ADULT LITERACY PROGRAM IN SOUTHERN INDIA Meera Sundararajan & V. Vinod Kumar CARE India, Tsunami Response Program,Chennai- India

The Context • Post tsunami rehabilitation efforts in livelihoods identified gaps in functional literacy as one of the reasons of vulnerability and poverty condition among women. • Though female literacy rates for the two southern states affected by disaster was 61% and about 73% respectively, it was found that in the case of older and economically more active women literacy rates were much lower. • Therefore it was felt that enabling functional literacy was critical to ensuring better returns from livelihood activities.

The objectives of the project Goal • To develop and promote enhanced capabilities of women of the Tsunami affected villages for better livelihoods and empowerment Objectives • To impart and improve functional literacy and numeric skills • To provide learning experiences which promote better livelihood preparedness

Course outline • The course period for a batch of 1200 learners consists of a core learning period of 6 months and is divided into 3 phases each focusing on the following inputs. Phase-1: Basic Literacy & Numeric Skills with Projects/Practical Phase-2: Reading, Writing & Calculation Skills with Projects Phase-3: More of Projects for ‘Experiential Learning’ • Each phase has eight weekly teaching guides (except the last one which has 2 weekly guides) with each guide being focused on the learning objectives for the week.

Outreach • 4 coastal districts- Nagapattinam and Cuddalore in the state of Tamil Nadu and Nellore and Krishna in the state of Andhra Pradesh • 10 learning centres in each district • About 30 learners in each centre. • Total number of learners enrolled in the current batch 1221

Monitoring system Targeting Learning abilities : Tracking the learning milestones for each fortnight (based on the learning objectives for the week) Learners’ are graded through testing of Written material Class room observation One to one observation Categorization is done based on ability to achieve learning milestones. Abilities classified as “outstanding”, “Good”, “Doubtful” “Poor” with respect to the key learning milestones for that fortnight. Baseline is generated on the core competencies that are expected at the end of the 6 month course

Levels of analysis • At the level of individual learners • At the level of each learning centre • At the district level • Consolidation at the project level

“ Outcome” indicators • 9 Literacy and 11 numeric abilities that are expected to be exhibited by all learners at the end of the course. • There are 5 sets of intermediary level literacy and numeracy milestones that the learners have to go through over the first two phases. • The outcome indicators are tracked regularly from the 4th week of phase 2 right up to the end of the course.

Targeting: Key features of 1221 learners enrolled in batch 2 ( current batch) • Social vulnerabilities : Predominance of vulnerable castes and tribal groups and women headed households

Targeting: Key features of learners enrolled in batch 2 ( current batch) • Age profile : Predominantly 26-45 yrs of age- Economically active age group

Targeting: Key features of learners enrolled in batch 2 ( current batch) • Occupational profile : Mostly daily wage workers

Key learning competencies expected at the end of the course Literacy related • Ability to express/speak out opinions on a given topic fluently and with relevance in front of the other learners in the class • Ability to make & narrate stories from pictures in a logical sequence. • .Ability to read and understand simple poems and stories • Ability to read & understand short and simple prose texts based on health, livelihoods, govt. schemes, notices • Ability to read written instructions and do the intended activity on their own • Ability to read and find desired information from print materials • Ability to write sentences that describe the passage of events. • Ability to write a relevant text/para (short) given by the teacher based on a topic that relates to their day to day life or livelihoods or general issues, etc. • Ability to understand, sequence and respond to questions.

Key learning competencies expected at the end of the course • Numeracy related • Ability to arrange a set of 3-digit numbers (eg. 100 and above) in to ascending & descending order • Ability to write at least 5 given numbers in words • Ability to say the number in sequential order from a given number (starting) to another given number (ending number) • Ability to solve 2 problems with carry-over in each (ie. addition, subtractions and multiplication) • Ability to do 3 simple division without reminder • Ability to understand write down the fraction figures as the teacher dictates the following: 1/2 Kg, 1/4 Kg, 3/4 Kg, 1/1 Kg also 2/3…) • Ability to solve word problems involving time & days calculations • Ability to identify 5 basic shapes out the objects seen around and write down their shapes with name • Ability to measure the given objects using appropriate units of measurement • Ability to identify the next number by understanding the patterns (of numbers) given by the teacher • Ability to solve at least 3 currency based word problems involving basic mathematical operations such as addition, subtraction, multiplication & decimal usage

70% 60% 50% Poor 40% Doubtful 30% Good Outstanding 20% 10% 0% L1 L2 L3 L4 L5 L6 L7 L8 L9 Baseline on core literacy competencies at enrolment Nagapattinam Cuddalore

Baseline on core literacy competencies at enrolment Krishna Nellore

Baseline on numeric abilities during enrollment Cuddalore Nagapattinam

Baseline on numeric abilities during enrollment Nellore Krishna

After 2 weeks of attending classes Literacy milestones expected • Be able to recognise a large number of words, know more than 10 letters, and be able to combine some of these to read / make new words. • Be able to write 7-10 letters and also some words made from these letters. Numeric milestone expected • Count up to 20, recognize written numbers up to 20, and write them. • Understand the basics of addition, using 1-9, with the total being below 20.

Consolidated performance of all learners on the literacy milestones at week2

Consolidated performance of all learners on the numeracy milestones

Implications of analysis at each level

Learners at week 4- Milestones to be achieved • Literacy (7 key milestones) that primarily focus on measuring • Public interaction and comfort levels with that • Recognition of new letters, words and symbols of vowel sounds and identify them in a given text. • Ability to read the text based on recognition • Ability to copy a given text meaningfully • Numeric milestones (9 key milestones) that aim at measuring ability to • Understand and apply measurement concepts • Read and write numbers, from 1-20 definitely and 21-50 if possible • Internalization of basic math concepts like larger and smaller numbers, • Observe and record information in numbers • Do addition and subtraction without carry over; orally and written

Milestones to be achieved by week 8 Literacy milestones • Read and understand small, familiar and interesting prose/poetry passages and answer questions based on that. • Understand pictures given with texts and make stories based on that • Write down dictation of known words and sentence • Work with a library • Numeracy milestones • Expand given numbers up to 3 digits (e.g. say that 134 = 100 + 30 + 4) • Perform operations (+, -, x) up to 50, and perhaps 100 (without carry over). • Count objects, and arrange different groups of objects in sequence of numbers of objects they 3contain • Understand a number pattern • Answer word-based problems that relate to their daily experiences • Measurement: with non-standard units (hand span) • Use units of time

Milestonesto be achieved by week 10 • Literacy milestones • Ability to make stories from pictures and narrate them in sequence and write sentences on pictures • Ability to read loudly, fluently with attention to punctuation • Ability to answer small questions from text • Ability to identify and read livelihood related words and words relating to bank, hospital etc • Interest in taking notes especially when new words are introduced • Numeracy milestones • Able to write 1-100 • Recording information in numbers • Clear understanding on the carry over method in calculations • Ability to identify place value of numbers upto 3 digits • Use of standard measurement systems like scale etc Others • Understanding and clearly articulating the concept of self and their role as women • Interest in involving in social activities

Week 10 onwards • From the end of week 10 onwards till the end of course at week 20 the learners are rated on the core competencies ( 9 literacy and 11 numeracy milestones) • The last month is used for learners who have fallen behind to catch up.

Learning milestones and knowledge generation for the project • Collation of learning milestones and comparison with attendance data for each learner is able to tell us reasons behind consistently poor performance of a learner. • It also helps in strengthening the hypothesis that regular attendance fosters good learning. • Often when regular attendance does not match up with a reflection in good learning milestone achievement- the teacher’s ability to deal with the learning challenge often emerges as the reason. • Learner profile often has a bearing on the attendance and consequently on the achievement of learning milestones.

Additional monitoring mechanisms used • Collation of attendance data across centres and comparing it with the level of achievement of learning milestones. • Use of class room observation tool COT for Dec 08.xls • Comparison of performance of learners involved in livelihood projects with those learners who are not part of the learning initiative

Challenges • Some subjectivity in the grading of the learners. • Some times time becomes a challenge as the teacher has to grade 30 learners in two to three hours of class. • Teachers abilities often pose a challenge in enabling milestone achievement. • All learners have not been included in livelihood projects due to various operational reasons so it is difficult to compare it across the board- the external variables governing this are a little difficult to control.

Thank you

MEASUREMENT OF PROCESS EFFECTIVENESS