1 / 10

To RCT, or not RCT: that is the question

To RCT, or not RCT: that is the question. Mandy Crow, Lucy Lernelius, Alex Webb Widening Participation and Outreach Team, The University of Manchester. “Craft a patchwork of evaluation methodologies” *.

rsmart
Télécharger la présentation

To RCT, or not RCT: that is the question

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. To RCT, or not RCT: that is the question Mandy Crow, Lucy Lernelius, Alex Webb Widening Participation and Outreach Team, The University of Manchester

  2. “Craft a patchwork of evaluation methodologies”* • WP programmes aim to change the lives of children and young people by intervening in complex systems • Complexity aware evaluation “positions evaluation as an internal, team function integrated into action and ongoing interpretive processes” (Douthwaite et al, 2017) • Harrison and Waller (2017) recommend a ‘realist small steps approach’ whereby evidence is built incrementally, based upon an evolving theory of change framework *Connelly, 2018

  3. Manchester Access Programme (MAP) MAP is a structured scheme for local post-16 students from less advantaged backgrounds

  4. Manchester Access Programme (MAP) Progress and Options workshops – to discuss higher education options with a member of university staffSupport with writing UCAS Personal StatementSubmission of pre-UCAS form and feedback from admission decision-makersInterview skills workshops Decision Manchester Opportunity for students to get advice and an early decision on their application to The University of Manchester, before submitting their UCAS application

  5. Post programme questionnaire 2018 (E2019)As a result of completing MAP do you feel more confident about your university applications? 94% Yes, I have gained so many invaluable skills from MAP as well as application support and guidance, especially with writing a personal statement …it has given me so much confidence to progress to the next stage of my life. It was a privilege to be part of it and I feel the lower offer has given me a chance to achieve my goals. Yes, it is because of MAP that I am more confident in applying for University

  6. 379 initially successful 560 course applications through Decision Manchester 402 successful Applications to Manchester Pre-UCAS applications – Decision Manchester 2018 (E2019) 181 initially unsuccessful 23 reapplications successful

  7. Around1/3of MAP entrants to the University need the 2 A-level grade (or equivalent) reduction. Twice as many MAP students are accepted onto courses at Manchester compared to non-MAP applicants from the same colleges. Around 30% of students who complete MAP will progress onto a course at another university. Analysis of previous cohorts has shown

  8. 95% of students who successfully complete MAP will make an application to Manchester – with around 75% being made at least one offer 29% of MAP entrants to the University do not need the 2 A-level grade (or equivalent) reduction Following expansion in 2013, around 500 students successfully complete MAP each year Impact Since 2005, 1,974 MAP students have successfully gained a place at The University of Manchester And when they have completed University over 90% of MAP graduates are in a positive employment destination Typically ¾ of our MAP undergraduates come from the lowest household incomes MAP undergraduate entrants have a continuation rate of nearly 95%

  9. Studentship • 3 year funded PhD studentship • To further develop understanding of: • the effect of MAP on raising attainment - does the work we do to raise expectations and support students in developing a range of skills to demonstrate their preparedness for university study have any impact on their attainment? • the choice to engage/not engage with MAP -what are the challenges and opportunities in engaging schools and recruiting students? • the nature of the impact of MAP, in the context of everyday life and experiences. • Key theories: Habitus, cultural capital, fields (Bourdieu); transactionalism, pragmatism • Methods: Interviews, case study group followed through the programme and beyond, quantitative analysis of attainment data

  10. So.. To RCT, or not RCT? Context is really important– ‘one size does not fit all’ Ethical and logistical considerations An RCT may tell us that an intervention has worked – however it will not tell us how an intervention has worked. Ideally would also need qualitative work to happen at the same time RCTs are not the only way to evaluate impact - a rigorous mixed methods approach is better than a poor RCT! It is possible to show how the intermediate outcomes make a strong case for a longer-term impact – but may not be able to generate a causal conclusion Resources – cost of funding an RCT

More Related