ADVANCED MASTERY OF THE e-PORTFOLIO -A COMPETENCE BASED FRAMEWORK

1. ADVANCED MASTERY OF THE e-PORTFOLIO-A COMPETENCE BASED FRAMEWORK TRAINERS WORKSHOP 12.10.10 RIPON SPA

2. SET OBJECTIVES • WHAT DO YOU WANT TO GET OUT OF TODAY?

3. Programme • 930 Agenda setting • 945 Update in e-portfolio v5 /Clustering –the RDMP model • 1000 How do we fill in an ES report. Difficulties.Hot tips. • 1030 The journey from ES to panels • 1100-1115 Coffee • 1115 Validation and reflection-exercises in groups • 1230 Update from Deanery • 1245-1330 Lunch • 1330 Trainees in Difficulty • 1500-1515 Tea • 1515 E-portfolio practice and validity tasks.

4. An exercise in Observation • http://www.youtube.com/watch?v=Rbmai1b__i4

5. UPDATE Eportfolio v5 RCGP update-Positive ideas • Expanded rating scales-relate to stage- • NFD – Below Expectations; NFD – Meets Expectations; NFD – Above Expectations • Competent for Licensing • Excellent • Each part is on a separate web page - helps to concentrate on each before a summary in the seventh part. Each can be saved • All the evidence for each section is provided on same page. • Eg the curriculum coverage now provides a curriculum map which can delve into before making comments. • ES rating scales - all the evidence has been pooled together quite nicely. No more clicking out of the 'review' section and having to delve into other menus and options first. • Ensures you sign the educational work contract. You can't do a review without signing it. Means that we are now less likely to be crucified for not having signed the ed. work contract when it comes to ARCP time.

6. EDUCATIONAL SUPERVISON • What do we find hard? • Preparation STs • Preparation ES • Documentation-ES report • Ensure that judgments are fully evidence based and referenced. • Controversies

7. ES Controversies • Time • Pay • Validity • Lack of Guidance • Do we need to spend as much time for all? Or concentrate on those with difficulties? • Giving feedback. • Who reads? CS or ES? • Validation and linkage • Reflection

8. STs-PRE ES meeting • Arrange meeting! • log and share an e-portfolio entry entitled Ed Sup Rev current date. Attach: • COT & CBD competency mapping • HDR spreadsheet and sick leave/all leave spreadsheet. List complaints. • Ensure CSR report done. • Ensure self rating assessment and PDP up to date. • Ensure compulsory assessments (inc MSF in modular posts)?include NOE • Ensure last objectives achieved.

9. ES pre meeting • How much do we look at ? • How long should we spend? • How to gain the evidence for competences? See appendix • ESR report when looking at evidence or after meeting? • Ensuring evidence is referenced not feelings

11. Deanery Guidelines ES How Many ES Meetings and When? • ST1: 2 meetings in first post, 1 meeting in second  (i.e. 3 for that year: 1 informal + 2 formal) • ST2: 1 per 6m post (ie 2 for that year: both formal) • ST3: 1 per 6m post (ie 2 for that year: both formal) •  So, especially during the period Feb-Aug of every year (as that is when most trainees will move onto the next ST stage) make sure you have had your second ES meeting before the end of May

12. Deanery Guidelines ES “The deanery expects all trainers to be an ES so we need to make sure it is achievable “ • read entries, link these to competencies and provide feedback • provide other feedback re: number of log entries/ whether getting behind on OOH requirements, formal assessments numbers of entries/ style of entries etc. • assess the Naturally Occurring Evidence at the times of the ESRs • There should be a high level of supervision throughout the final year as the year will be entirely in GP

13. Deanery expectations ES • Educate trainees about the quality and reflectiveness of the log entries from ST1 • You need to be more vigilant at the ST2 stage when in hospital often to ensure do not lose sight of attaining GP competencies . • There is a move to expect more virtual supervision of e portfolio by the ES… more reading and responding to log diary entries, more use of 'educators notes' (to indicate progress ). we do not see doing the ESR meetings as whole role of the ES but that these meetings are punctuations on a continuous ( if intermittent) interaction between ES and GPSTR! • Those who are behind/ under performing will need more meetings , as well as more continuous checking and reading and feeding back etc.

14. After ES meetings • MUST generate an "Educational  Supervisor's Report" (ESR). • This report will highlight any difficulties - either personal, educational or with the post.. Educational supervision is not about disciplinary procedures but more about helping you overcome or see you through the difficulties. • The report has an "agreed action plan" to help your training journey become smoother. • The report must be uploaded onto the e-portfolio.  

15. ESR suggested framework • Create a review-in advance-reasons • So that the Trainee's evidence appears in the correct Review period • For an Educational Supervisor to release the PSQ and MSF results to the Trainee, they need to have the current review set. The system will not allow these to be released if this is not currently set. • Under the new system, in order to make the Trainee “Self Rating” Review specific, a Review needs to exist to give the trainee this option. • View and comment on Curriculum Coverage • View and comment on Skills Log • Review and comment on the PDP • Review the Trainee’s Self Assessment of Competence

16. ESR Review 2 • Review each of the 12 areas of Competence • Make an "evidence-based" judgement about the progress of the trainee in each of the twelve areas, looking at evidence from the linked forms and learning log entries. • Decide on a rating • Give evidence to support this rating • Decide on specific actions to help with progression before the next review • The trainee is required to go through the same process in their “self assessment” of competence • You will not be able to continue to the next stage of the review process until all sections have been completed • Finish Review • Review the summary of all comments and actions • Comment on the quality of the evidence provided • Make a Recommendation • Agree a Learning Plan • The review details can then be edited, saved and edited later, or the review can be completed and submitted.

17. ESR • How do we do it? • Suggestions re framework-RAM idea • a) click on learning log and review; look at the NOE (Audit, SEA, reflection of post)b) click on evidence - review CBDs, COTs, CEXs - minimum numbers, competence mappingc) stay on evidence - review PSQ, MSF and CSR - look for themesd) click on the 'Educators' Notes' and see what themes there are theree) Now click on 'create review' and go through the ESR form in order using the evidence made available to you on each page and bringing in other stuff noted from steps a)-d).

18. Post coffee • http://www.youtube.com/watch?v=6EElqrgk4N0&feature=related

19. Log entries • Log entries should on average show: • evidence of critical thinking & analysis, describing own thought processes • self awareness demonstrating openness and honesty about performance and some consideration of feelings generated • evidence of learning, appropriately describing what needs to be learned, why & how • appropriate linkage to curriculum • demonstration of behaviour that allows linkage to one or more competency areas

20. Gibbs Reflective Cycle

21. Reflective Writing: role and functions • To maximise the effectiveness of experiential learning • To evaluate one’s practice • To promote critical thinking • To facilitate the integration of theory with practice • To generate theory • To evaluate a learning activity • To demonstrate that learning has taken place

22. Reflection How do I decide which competence area to attach? • You may select more than one area if you believe the entry merits this. • Trainees will be expected to demonstrate progression in all 12-competency areas (and hence to have a corresponding spread of validated entries). How do I decide whether the entry shows ‘meaningful reflection’? • The trainee should demonstrate the impact on their practice of the experience they describe. This may relate to new knowledge and skills or a0n effect on insight and attitude. • The quality of the reflection should be judged against the learner’s stage of training with sophisticated reflection as progress. How should I comment on the log diary entry? • Your comments vital feedback and should respond to their reflection. • If you have decided to validate an entry against a competence area, you should describe in your view why it has addressed the area you have chosen (and this may include your suggestions for further development)

23. Reflection template

24. Validation Why does validating entries matter? • The learning log helps to balance the educational portfolio and provides additional evidence of learning and progression, capturing evidence from learning opportunities in the workplace. • There is no limit to the number or quality of entries that trainees can make in their eportfolios, but not all of them can or should be validated. For example, attendance at VTS seminars. Entries which cannot be validated may still be useful for curriculum coverage. • Once validated by you, each entry then forms part of the trainee’s evidence of progression. • Entries are validated against the 12 areas of the competency framework.

25. Validation What does validating an entry mean? • It means that you, in your role as trainer or educational supervisor, are confident that the entry fulfils the following two requirements: • a) It addresses one or more of the 12 competence areas • b) It demonstrates meaningful reflection • By validating a log entry you are confirming that this is valid evidence of learning in an appropriate competency area. You are not making a judgement about whether that competence has been achieved.

26. Reflection and validation exercises • Eportfolio j smith2 • Password jsmith2 • Dr Pauline example Task 1- first 3 entries 13.2-7.4.08 • Appropriate no of entries? • Critically appraise the entry and justify • Curriculum linkage-acceptable • Advice re further entries -reflection,linkage etc

27. Task 2 • Log entries 7.6.08-1.7.08 • How many log entries do we read? Does it cahange at different stages of training. • ES comments • Try making your own pre read comments • What is function?/how often? • Comment on quality presented and how may improve • Validation • Appropriate? • Alternatives? • How many validated entries per week/?

28. Task 3 • Last 2 log entries 4.7.08 9.7.08 • How to validate • Comments? • Plenary? Anything to ask or share?

29. Lessons from ARCP panels • Best composition of panel-TPD(Chair)/Trainer who is and ES/lay member /Deanery rep • Evidence given by somebody who does not know ST well • The ‘progress to certification’ view on the portfolio is effectively the Educational Supervisor’s report. The reviewer’s job prior to the panel is to check that the ES’s judgement is backed by evidence in the portfolio and is appropriate. • A Note on OOP-Those trainees ‘Out of Programme’ (eg maternity, sick or abroad), still need an ES review. If there is substantial evidence in that review period, it needs commenting on. If not, mark the review as ‘refer to panel’

30. Lessons from ARCP panels 2 • For CCT-four critical items: attainment of Competences, AKT, CSA, and the must-dos of OOH and NOE . If on track ,evidence appropriately backed up-minimal discussion & quick look at learning log. • If less straightfoirward –illustrative not exhaustive, evidence based not anecdotal • Curriculum coverage /PDP more for ongoing supervision than satisfactory progression.

31. ARCP PANEL OUTCOMES • SATISFACTORY –FOR PROGRESSION OR CCT • OUT OF PROGRAMME • MAINLY SATISFACTORY-ARCP NOT COMPLETED;FEEDBACK TO TRAINEE TO OBTAIN MISSING EVIDENCE-PANEL CHAIR REVIEWS EVIDENCE 2W LATER. • POSSIBLE OR LIKELY UNSATISFACTORY-ARCP NOT COMPLETED ;WRITTEN EVIDENCE TO DEANERY RE REASONS • ONLY ONE OUTCOME PER TRAINEE IN EACH ARCP CYCLE

33. WHY do STs get referrred to central Deanery panel • Majority incomplete evidence eg • out of hours sessions, • patient satisfaction questionnaires and • other workplace based assessment tools. • NOE(Naturally Occurring Evidence) • Confusion modular posts /LTFTT • Clustering minimal evidence.

34. OOH • There is the service commitment to out of hours work that is specified for each training post. Not attending OOH sessions is a probity issue. • In an Innovative Training Post (ITP) most ITPs will have the same monthly (6 hour) session of OOH work as normal GP training posts. Some will have on call commitments to the modular component of their post – eg on labour ward or hospice. make clear in the portfolio. If no OOH sessions logged panels will find the portfolio unsatisfactory. (PSQ also due in modular posts) • Documentation of learning in OOH sessions -linking that to chapter 7 of the GP curriculum – Care of the Acutely Ill. • One trainee documented 2 OOH sessions in two months prior to panel. A total of only 3 patients had been seen in these two sessions. This is considered to be unsatisfactory. • Clustering/demand

35. OOH cont Advise STs to document for each OOH • The type of session – telephone triage, visiting doctor, base doctor • The number of patients seen. • A selection of the most interesting patients • The significant learning points and, • Link these to the curriculum(esp care acurtely ill)

36. NOE-Naturally Occurring Evidence • Deanery gives advice/ guidance as to evidence that occurs naturally or “Records experience” that should be submitted annually • Despite multiple emails –commonest area that Sts fall down on. • Is it legally enforceable as a local Yorkshire criteria?

37. Naturally Occurring Evidence 1) Significant Event Analysis – 3 per 6 month post – file under Significant Event Analysis 2) Reflection on key learning points from each post – file in Reading – expected length 1 side A4 3) Audit or QoF review or NPMS Project – x1 in 3 year training – file in Audit/ Project

38. NOE (cont) 4) Case study – 2 per year – file in Audit 5) Statement of Total Leave Taken – file in courses/certificates 6)Attendance Record at VTS teaching – supplied by VTS administrator 7) Complaints and adverse incident reports – if any. File in Professional Conversations

40. E-portfolio • Post tea • What happened next? • http://www.youtube.com/watch?v=mhRxyGlU2mQ

41. Using the e-portfolio In small groups • Practice using new ES report set review own ES-and look at how presented and changes. • Decide on a framework to present back for the most efficient way to assess evidence/ESR rv. • Look at examples of ES descriptors-use sheets and own examples. • Find alternatives • Give examples of good and weak evidence in ES reports (use appendix criteria)

42. Table 1. Criteria used for assessing EducationalSupervisors’ Reports.

43. PLENARY • ANY LEARNING TO SHARE • FEEDBACK FORMS • FUTURE LEARNING NEEDS

44. CLOSE

45. Reflective Writing: role and functions • To maximise the effectiveness of experiential learning • To evaluate one’s practice • To promote critical thinking • To facilitate the integration of theory with practice • To generate theory • To evaluate a learning activity • To demonstrate that learning has taken place

46. Reflective writing: description • What were the significant background factors to this experience? • Describe the experience – Sequence of events – Actions – Observations • What essential factors contributed to the experience?

47. Reflective Writing: analysis • What were the consequences of my actions? • How do I feel about the experience? • What factors influenced my decision and actions? • What knowledge influenced my decision and actions?

48. Reflective Writing: evaluation What went well; what went badly? Could I have dealt better with the situation? What other choices did I have? What would have been the consequences of acting on these other choices?

49. Reflective Writing: new perspectives • What have I learnt from this experience? • How has this experience affected my thinking?

50. Reflective Writing: action plan How should I change my practice? Behaviour Standards, procedures • Should I suggest changes in policy? • What constraints may exist? Review changes and their effects!