The Problem with Reflection: Lessons Learned

1. The Problem with Reflection: Lessons Learned Alfredo Gaitan (alfredo.gaitan@beds.ac.uk_ University of Bedfordshire Cathy Buyarski (cbuyarsk@iupui.edu) Indiana University-Purdue University Indianapolis Presentation delivered at the AAEEBL meeting 25-28 July 2011.

2. Session Overview • Introduction of presenters and portfolios • Three sections: • Using reflection – institutional contexts and overview of portfolios • Assessing reflection – how, when, how well • What we learnt about reflection – in our projects and from each other • We will aim to share two different projects/ perspectives, e.g. • Campus versus department • Freshman versus across enrolment • Summative assessment versus formative assessment • Insights on the students’ experiences of constructing eportfolios

3. Institutional contexts of PDP IUPUI UoB E-portfolios fit with the PDP model of learning, support the personalisation agenda and can also serve to structure learning for students studying remotely from the University. “We will further develop the use of e-portfolios as a framework to enable all students to record and review their progress and e-PDP as a means for structured and supported development.” CRe8. The ‘realistic learning’ dimension emphasises a learning experience which is meaningful, active, reflective, collaborative and challenging. Personal development planningis a process which will enable first year students at IUPUI to understand, implement, and mark progress toward a degree and career goal by creating and following a personalized plan that is open to revision and reevaluation every semester in collaboration with an academic advisor, faculty member, or other mentor. RISE Each undergraduate student is challenged to include at least two of the four RISE experiences - research, international, service learning, and experiential learning - into their degree programs. Principles of Undergraduate Learning

4. Overview of ePortfolios IUPUI UoB Implemented at levels 1, 2, 3 of the Psychology curriculum with 225 students In Year 1 and 342 in Year 2. Reflection (with prompts) in a optional structure: Background, My Learning Experiences My Learner Development My Employability (includes a resume) Showcasing my performance • Implemented in first-year seminar course across the entire campus (342 students in fall 2010). • Reflection (with very specific guided prompts) in seven areas: • About Me • Educational Goals and Plan • Career Goals • Academic Showcase • Campus and Community Connections • My College Achievements • Resume

5. Portfolio Examples IUPUI • UoB Example Examples

6. Assessment of Reflection IUPUI UoB Assessment was summative of students’ planning, experience, reflection, prospective view. Part of the assessments in each of three units (30-credit modules): 5%, 20%, 10%. Assessed at the level of sections or whole portfolio. Rubric loosely based on experiential learning (e.g. Kolb). Feedback: comments written on each page/section and the portfolio. Moderation in some units. • Assessment to date has been formative and designed to improve prompts and rubrics. • Main assessment of the First Year Seminar (1 credit): 100%. • Assessed at the level of each prompt. • Rubrics based Bloom’s Taxonomy. • Feedback provided to students on each section of portfolio. • Conducted through content analysis process.

7. Lessons LearnedIUPUI Table 1: 2010 e-PDP Compared to Not E-PDP First-Year Seminar Sections: Student Characteristics and Academic Success Indicators. Note 1: Missing cases were excluded from analyses. Note 2: Students who Withdrew or who were Administratively Withdrawn from Seminars were excluded (N=66 students). Note 3: Bolded items are significantly different based on independent samples t-test or chi-square results (p < 05). 

8. Lessons Learning University of Bedfordshire Table 2. Submissions of e-portfolios over the two years of the project.

9. Lessons LearnedIUPUI • Assessing reflection in developmentally appropriate ways • Scaffolding the use of reflection • Faculty development • Rubrics need to include cognitive development AND critical thinking • Grading versus assessment

10. Lessons Learning University of Bedfordshire An emergent ‘grounded theory’ 1 of the students’ experiences: interconnected themes (1) Glasser & Strauss, 1967; Pigeon, 1996; Pigeon & Henwood, 1997)

11. What We Learnt from Collaborating Cathy Alfredo The eporfolios at UoB could be more directly linked to the coursework. Our portfolios are snapshots of a year, and only the Year 3 takes a long view (3 portfolios rather than one). The strong ethos and framework (e.g. RISE) may provide an environment favorable to e-portfolios. What can eportfolios do for your institution? • Moving the PDP to embed in academic programs is an important next step • Faculty buy-in and development will be key factors in our success • Use of themes from UoB will inform our thinking about student participation

12. Let’s talk! • Any questions? • What have you learnt from your own experiences that you can offer us? Thank you!