Compliance Monitoring and Enforcement Audit Program - The Audit Process

1. Compliance Monitoring and Enforcement Audit Program - The Audit Process

2. Introduction This presentation is intended as an introduction to the audit process for employees of entities being audited by MACD. Please refer to the manual for more detail

3. Introduction to Compliance Audits • Audit Notification • Pre Audit Meetings • Data Submission • The Audit • Exit Meeting • Audit Report

4. Introduction to Compliance Audits • Audits are not “out to get you” • “Trust but Verify” is the philosophy for the process • The Audit team must, in their professional judgment, be satisfied that the evidence demonstrates compliance • If the Audit team cannot confirm compliance to a standard, the report will indicate potential non compliance which MAY trigger an investigation

5. Key Roles Your company’s Audit Coordinator • The Single Point of Contact for the audit • Will do all coordination with MACD • The person you should talk to with any concerns or questions • The only person who should identify concerns or issues to the MACD Audit Team • The person responsible for all logistics with the MACD Audit Team

6. Key Roles Subject Matter Expert • The person your company has identified as being able to discuss audit information with the audit team • Essential to your company’s ability to demonstrate compliance with standards • May be interviewed by the Audit Team to discuss reliability standards • Should discuss any questions about the audit in advance with your Audit Coordinator

7. Key Roles Audit Team Lead • An employee of MACD • Coordinates all audit efforts for MACD with your Audit Coordinator • Discusses any issues or concerns with your Audit Coordinator • The local decision maker for MACD

8. Audit Notification • At least 90 days before your audit, you will receive a notification • Audit Team • Scope • Off Site vs. On Site • Date of Audit • Pre Audit Survey • Initial Data Requests • Pre Audit Schedule of Activities

9. Notification - Audit Team • Audit Team Members: • are responsible for producing an opinion on adherence to standards • have experience in the industry and/or auditing • have completed NERC Audit Training • have confidentiality agreements in place • are independent, have not worked in your company in the recent past • issues? Discuss with MACD in advance

10. Notification - Audit Scope Scope - Standards, Audit Period, Sample Size • Standards: • Responsible for all applicable Reliability Standards • Audit will focus on a sub set chosen by MACD • Subset reflects industry trends, impact on reliability • Period is a rolling six years, audit team can examine compliance during entire six years • Data requested is usually for previous 12 months

11. Pre Audit Activities Pre Audit Survey • A tool for the audit team to learn more about your organization. • Generic, some questions may not apply to your organization. • Questions about assets, organization structure, physical location and neighbours • Some logistical information and contact information • Helps the audit progress more smoothly

12. Pre Audit Activities RSAWs – Reliability Standard Audit Worksheet • Tool to help you demonstrate your compliance • Guides you through providing evidence to demonstrate your compliance • Assists you in organizing and packaging the information • Clearly identify appropriate sections in the files for evidence, reference the standard • Submited to MACD through the Portal

13. RSAWs

14. Pre Audit Activities • It is important to annotate documents • If using the same document for multiple requirements, put a copy of the document in each folder!

15. Pre Audit Activities • The IESO Portal is a secure system already in use for providing this information to the audit team • Portal training material is available on the IESO website • http://www.ieso.ca/imoweb/marketplaceTraining/showCourses.asp?id=15

16. Pre Audit Activities • It is important to organize information to show where it ties to the standard (to show compliance) • Include the RSAW for each standard in the top folder, and then evidence documentation with annotation for each requirement.

17. Pre Audit Meetings • Open dialogue is important during the audit process. Opportunities for 3 pre audit meetings. • Pre submission meeting – date agreed to, allows for general questions and discussion • Pre Audit meeting, optional, done after MACD has reviewed the data submission • Pre Audit Logistics – At least one week prior to audit to discuss the logistics of the audit week

18. On Site Logistics Audit Team will require: • Rooms to work in • Lunch/coffee (working lunch delivered to room) • MACD to be invoiced • Access to projector, internet, printers, if possible via our laptops – All evidence should be electronic unless otherwise agreed by the MACD audit team • Access to site as needed, with appropriate escorts

19. Audit Presentations The Audit team will start the audit with a brief opening presentation. • Introducing the team • Expectations for the process It can be helpful if the MP makes a brief opening presentation to • Introduce the MP audit team • Provide some background on the company and the facility • Explain any safety/security requirements

20. Interviews • The Audit Team may want to talk to SMEs (Subject Matter Experts) and other staff • Your Audit Coordinator will arrange interviews • The Audit Team may want to ask questions about procedures, their implementation, data

21. Exit Meeting Attheendoftheauditperiod,theteamwillgiveanexitpresentation.Thispresentationwillgiveanoverallsummary: • Recapoftheauditreview • TheAuditTeamobservations,preliminaryfindingsandrecommendations • Scheduleforremainderofprocess,includingwhenthefinalreportwillbeissued

22. Final Report • The audit report would identify if the team was unable to confirm compliance with a reliability standard • The potential non compliance is turned over to the investigation unit in MACD for follow up. • There is no “penalty” from the audit itself

23. Potential Non Compliance Findings • If the Audit Team identifies an immediate risk to reliability, the Team Lead will immediately communicate this to your Audit Coordinator and then to MACD for investigation • For other potential non compliance observations that do not have an immediate impact on system reliability, these will be reported at the conclusion of the audit

24. Escalation Process Any questions or concerns during the audit: • Discuss with your company Audit Coordinator • The Audit Coordinator from your company should discuss with the Audit Team Lead as soon as possible. • If not satisfied after discussing with the Team Lead, your Audit Coordinator can escalate the issue to the MACD supervisor/director • If still not satisfied, your company can take issues to the Dispute Resolution Panel (very formal process) – OEB process

25. Summary • A minimum of 90 days notice • Initial Information Request - before the audit begins, submit information to demonstrate compliance • Subsequent Information Requests may come during the audit • SMEs and other staff may be interviewed • The Audit Team will provide an exit interview with preliminary results • The Audit Report will follow with details

26. Compliance Monitoring and Enforcement Audit Program - The Audit Process End Of Presentation