Download
1 / 232
3.48k likes | 5.31k Vues
Audit Principles. NERC Auditor Training Introduction to Audit Principles and Techniques. REMG Compliance Auditor Training. Please Remember…. This is not technical training. There are “hard” and “soft” skills to auditing
E N D
Audit Principles NERC Auditor TrainingIntroduction to Audit Principles and Techniques
Please Remember… • This is not technical training. There are “hard” and “soft” skills to auditing • This course is focused on auditing skill building and is based on years of auditing knowledge. • ALL auditing follow the same basic principles. We ALL need to think like auditors. • Discussion is welcomed – especially on how to apply standard practices to NERC compliance audits • Training materials are under development, so your feedback is very much appreciated! • Trainers assume that you have read the CMEP
AGENDA • 10.00 Introductions/Course Overview All • 10.30 Module 1 - Conceptual Framework Russ Hissom • Dan Skaar • 11.15 Module 2 – Audit Process and • Planning Russ Hissom • 12.00 Lunch • 1:00 Module 3 – Managing Relations Carol Arneson • 1:30 Module 4 – Audit Documentation and • Evidence (and RSAW Breakout) Russ Hissom • Kevin Goolsby • 2:30 Module 5 – Workpaper Preparation and • Information Requests Carol Arneson • 3:00 Break • 3.45 Module 6 – Audit Testing and Testing Methodologies Russ Hissom • 4.45 Questions and Discussion • 5.00 Happy Hour!!
AGENDA • 8.00 Module 7 – Interview Techniques Carol Arneson • 9:00 Module 8 – Leveraging Project Management • Tools and Techniques for Audit Success Carol Arneson • 9:30 Break • 9:50 Module 9 – Report and Workpaper Review Russ Hissom • 10:20 Module 10 – Conflict Resolution and Escalation Protocols Carol Arneson • 10.45 Module 11– Audit Close Activities, Lessons • Learned and On-going Performance • Management Russ Hissom • Close Questions and Discussion – Course Evaluations
Compliance Program Assistance Leadership Team Virchow Krause Team Members Carol Arneson, PMP, MBA, Senior Manager in the Energy and Utilities Group, has worked in the utility industry since 1976 specializing in financial and operational needs. She has broad utility experience at two Fortune 500 utility companies where she managed financial, cost management, strategy and business planning, generation support processes, and various other business processes for over 20 years. Carol has managed numerous projects serving municipal and investor-owned utilities including contract compliance audits, energy management services contracting and performance audits. Russell Hissom, CPA, Partnerin the Energy and Utilities Group, specializes in serving the financial and operational needs of the utility industry. He has extensive experience with financial audits of utilities, management audits for utilities and State Public Utility Commissions, developing utility cost of service and rate design studies, analyzing the input and performance of parties under jointly owned electric generation contracts, assisting with accounting issues under FAS 133/149 and performing operational reviews.
Goals for this Course 1. 2. 3.
Very Brief History of the Compliance Landscape How Did We Get Here?
Don’t underestimate the power that lack of compliance of rules and regulations has on any industry • History has shown us that compliance (or lack of compliance) has the power to do many things: • Financial collapse large listed companies • Reduce market capitalization by billions of dollars for alleged, egregious violations • Bankrupt companies • Closure of plants
Do You Remember…….. • Enron (SEC compliance) • Worldcom (SEC compliance) • Ameranth (SEC compliance) • Bear Sterns, Lehman, AIG, Wachovia, Merrill Lynch, etc. etc. (SEC compliance) • Southwest Airlines (FAA compliance) • Northeast blackout of 2003 (NERC voluntary compliance) • Southern Florida blackout of 2008 (NERC mandatory compliance)
Crisis Leads to Regulation • Blackouts • Energy Policy Act of 2005 (section 215 of the Federal Power Act) mandated standards in USA with financial penalties (separate agreements within Canadian jurisdictions) • Financial Collapse and Fraud • COSO (internal controls) • Sarbanes-Oxley (internal controls, governance, civil penalties on corporate officers)
Role of the Auditor • Auditor’s responsibilities are much more important today than the past given the impacts from non compliance • Can you imagine if you, as an auditor, missed a major finding and had you made the finding you could have prevented an incident on the bulk power system? What about Enron, what if the auditor, early in the scandal discovered and reported the irregularities, perhaps things would be very different! • ALL auditors’ competency and training must be at a very high level to assure their duties to the industry and maintain the public trust. • Auditors’ work must stand up to public scrutiny and legal challenges!
Module 1 Conceptual Framework for Auditors
Learning Objectives • Understand what an “audit” is and is not • Develop the confidence to perform a competent audit • Understand the basic steps in an audit • Who are the audit standard setting bodies • What kinds of audits are there? • What does the technical guidance tell us to do?
What is an Audit? • An audit is an evaluation of a person, organization, system, process, project or product • It is not an investigation Audits may not presume a potential violation; investigation presume a potential violation exists. Skills are similar in the conduct of an investigation. Audits are performed to ascertain the validity and reliability of information, and may include an assessment of a system's internal compliance environment. The goal of an audit is to express an opinion whether some one or some entity meets a “standard” or does not meet a “standard” based upon a systematic review and testing of records. Due to practical constraints, an audit seeks to provide only reasonable assurance that the registrant is compliant with the applicable Reliability Standards.
Necessary Skills • Attention to detail • Good understanding of audit risks • Ability to work with people and experts • Subject matter expertise • Deep knowledge of reliability standards applicable to entities being audited • Knowledge of government auditing standards that apply to performance audits • Task management skills • Clear and concise communications • Ability to follow a standardized program • Good planning skills • Team player • Willingness to identify issues and be proactive in bringing them to attention
Major Steps Involved • A typical performance audit project involves the following steps: • Establish and communicate the scope and objectives for the audit. • Develop an understanding of the organization under review. This includes objectives of the audit, measurements, and key requirements. Review pertinent documents and interviews. • Identify control procedures used to ensure each key activity type is properly controlled, monitored and documented. Upfront, an internal compliance survey should be completed by the Registered Entity. • Develop and execute a risk-based sampling and testing approach to determine whether the most important activities are operating as intended. • Report findings and areas in compliance. • Complete audit closing tasks, review staff and start to prepare for your next audit.
Tips for Success • Be the “ultimate” professional • Expect to be “monitored”– lead by example • Use empathy – remember what it’s like to sit on the other side of the table. Be compassionate, but firm • Remember you’re there to complete a job – not solve the auditee’s problems • Be proactive • Know the project work planbeforeyou begin • Communicate with your audit leader • Thoroughly document all testing and findings with quality evidence
Audit Types • Financial Audit • A financial audit is an independent assessment of the fairness by which a company's financial statements are presented by its management • Authoritative standard bodies include GAO, Canadian versions as well • Compliance Audit • A compliance audit is an independent assessment of the compliance by an entity with various laws or regulatory requirements • Authoritative bodies include GAO (e.g. chapter 7) • NERC compliance audits • Management Audit • A management audit is an independent assessment of the efficiency in various operating areas by an entity • SAS 70 audit, “agreed upon procedures”
Value of Audits • Audits are not just checking if things happened or if compliance requirements were met – they can be used to provide great value to the registrant • Informal recommendations for process improvements or how to meet compliance requirements are a natural by-product of an audit (orally during the exit interview) • Audits serve the public and industry interests; there is a reliance on auditor’s work to identify compliance and non compliance; it’s the responsibility of the entity to comply and take necessary action to be compliance with standards
Auditing Concepts & Techniques • What is GAGAS? • In the United States – this is the standard for government performance audits – Generally Accepted Government Auditing Standards – GAGAS – aka the “yellow book” • Standards maintained by the Government Accountability Office (GAO) • GAGAS standards incorporate other standard bodies work • Requires auditors to serve the public interest and honor the public trust • Auditors must perform all duties with integrity, be independent and honest and candid with the entity being audited • Auditors should always exercise professional judgment and skepticism
GAO Chapter 2 Ethical Principles • Ethical principles guide the work of auditors • The public interest • Integrity • Objectivity • Proper use of information • Professional behavior • Please take five minutes and read Chapter 2. Any thoughts?
GAO Chapter 3-General Standards • Independence • Free of conflicts • Appearance and in fact • Professional Judgment • Knowledge, skills, experiences, • Reasonable care • Professional Skepticism • Due diligence • Competency • Blend of education and experience • MUST have skills to perform audit
Auditing Concepts & Techniques • Review GAO Chapter 3 • What are the key applicable parts of Chapter 3 in the conduct of our audits?
Professional Skepticism • The ability to approach any situation with a skeptical view towards conclusions reached without examining all factual data and using that data to verify and support your conclusion as an audit • ...show me!
Due Diligence Performance Improvement Intent Production & Reserve Increases Per Person Market Share Retail Return Market Growth Increase % Production Increase per Person France Germany Benelux Others Total Net Income $mm Reserves/10 per Person % ROI Production per Person MBoe/Person Plan 1993 1994 1995 1996 2Q95 3Q95 4Q95 1Q96 Net Income Market Share Change Strategic Targets/Initiatives % of Capital Spent Timeliness Quality Rating Comments 1 2 $M % 3 4 monthly monthly Change in Market Penetration Capital Expenditures Natural Gas % Refining Retail Increase in Gross National Product Refining Gross Operating Margin 2Q95 3Q95 4Q95 1Q96 2Q95 3Q95 4Q95 1Q96 Plan % % Actual 2Q95 3Q95 4Q95 1Q96 2Q95 3Q95 4Q95 1Q96 The act of researching all available data to support a conclusion or position about an activity or outcome
Due Professional Care • “Due Professional Care in the Performance of Work” • What is Reasonable Assurance? • Auditor must plan and perform audit to obtain appropriate evidence so that audit risk is limited to a low level appropriate for expressing an opinion on the assertion tested (making a compliance determination) • Absolute assurance may be not possible because of the nature of audit evidence. Point of distinction between an audit and investigation: an investigation may require absolute assurance for prosecution of a violation; therefore, “stacking of evidence” on a potential violation may be appropriate during an audit after a potential violation is discovered . • Materiality: is it material? Not all things are the same! • Management of the registrant is responsible for assuring compliance to Reliability Standards
Auditing Concepts & Techniques • GAGAS Continuing Education Requirements (per Government Auditing Standards) • Applies to external and internal auditors who perform GAGAS audits • Standards require 80 hours every 2 years of continuing education – 24 hours in subjects directly related to the governmental environment or governmental auditing • Remaining hours should be in topics that directly enhance the auditor’s professional proficiency to perform audits • At least 20 of the 24 hour requirement should be done in a single calendar year • Auditors who do not supervise audits or who charge less than 20% of the annual time to audits need 24 hours every 2 years • Assume these requirements apply to NERC—stay sharp, be a life long learner in this profession!
Auditing Concepts & Techniques • Review GAO Chapter 7-Field Work Standards for Performance Audits • What are the key applicable parts of Chapter 7 in the conduct of our audits? • Audit Evidence • Audit Risk • Audit Planning • Internal compliance environment • Sufficiency of audit evidence
Why do Auditor’s Fail? • SEC and others have reviewed audits and auditors and have determined several reasons why auditors fail: • Failure to obtain sufficient evidence to support conclusions. • Failure to maintain independence • Failure to follow-up on unusual events (exercising professional skepticism)
Independence Determine if the audit team is independent • “In all matters relating to the audit work, the audit organization and the individual auditor, whether government or public, should be free both in fact and appearance from personal, external, and organizational impairments to independence” • If it feels like you’re not independent, you’re not • If someone asks you “are you sure you’re independent?” - You’re probably not!
Other Independence Matters • Free of conflicts • NERC conflict of interest policy • Impairment of independence • Receiving gifts • Favors • If you think it is an impairment, it probably is an impairment • Must be independent “in appearance” and “in fact”
Auditing Concepts & TechniquesAudit Evidence • Audit Procedures for Obtaining Audit Evidence • Inspection of records or documents • Inspection of tangible assets • Inquiry • Confirmation • Recalculation • Re-performance • Analytical procedures
Auditing Concepts & Techniques Audit Risk and Materiality in Conducting an Audit Inherent Risk (IR) – the risk linked to the activity itself assuming there are no related controls • Example: Registrant performs activities linked to the bulk power system operations and planning. Registrant has no documentation and not trained staff to perform requirements under a Reliability Standard • Others? • ¹A material misstatement under Reliability Standards would be a requirement under a standard is not being met and it’s a material impact to the bulk power system or has the potential to materially impact the reliability of the bulk power system.
Auditing Concepts & Techniques Audit Risk and Materiality in Conducting an Audit Control Risk (CR) - the risk that controls will not prevent, detect and correct errors • Example: Registrant has documentation and trained staff, but no evidence of adequate supervision or review. • Others?
Auditing Concepts & Techniques Audit Risk and Materiality in Conducting an Audit Detection Risk (DR) – risk that auditor will not detect a material misstatement – function of audit procedure and its application by the auditor • Example: Regional Entity sampling of a requirement did not include enough samples. Result was a material number of samples which did not meet requirements (e.g.. non-compliant) were not detected due to insufficient sampling method or sample size. • Others?
Auditing Concepts & Techniques Understanding the Entity and Its Environment and Assessing the Risks of Material Misstatement (Compliance Environment) • Auditor should obtain an understanding of the entity’s internal compliance environment using the NERC internal compliance survey as the framework work; the framework includes: • Control environment (compliance) • Risk assessment • Information and communication systems • Control activities • Monitoring Auditor should send an internal compliance survey out before the audit to ascertain the relative risk; this is normally done in other audits which use a risk based approach
Elements of A Good Internal Compliance Program • Requires self assessments and/or self auditing • Encourages self-reporting • Directs aggressive, timely corrective actions • Provides documentation quickly • Demonstrates knowledge of the requirements of each applicable Reliability Standard • Maintains organization chart for internal compliance including senior management roles • Demonstrates independence from operations – a direct chain of command to senior management/CEO • Establishes internal compliance training program • Disciplinary procedures for deviation from compliance • Controls to prevent reoccurrence of violation • Whistleblower policies • Strong transparency of process and event facts • A strong internal compliance program is an important mitigating factor in any enforcement action as well as helping assessing risk (refer to handout for example of survey)
Basic Auditing Concepts • The three types of audit risk are: ______________________________ ______________________________ ______________________________ • Key principles of Chapter 2 of GAO:
Basic Auditing Concepts • Which is better audit evidence? • You read the policies and procedures manual of the registrant regarding their procedures in place to comply with PRC-005-01___? • The Compliance Manager of the registrant brags in an interview that the registrant’s procedures are so sound and controls so strong that “we can’t have an instance of non-compliance in this area and I’ll bet you lunch on it” ____?
Auditing Concepts & Techniques • Questions? • Follow-up items
Module 2 Audit Process and Planning
Preparing & Monitoring Your Audit • What is the “ideal” time allocation on an audit? Audit Planning – 10% - 20% Activities • Determine applicable Reliability Standards • Prepare budget and staffing • Information request development and issuance • Scheduling with Registrant • Travel arrangements • Determining materiality and areas of risk • Determining sample sizes • Follow-up on findings • Preliminary Registrant meetings • Review past history – past reports, alleged violations, enforcement actions, mitigation plans, etc.
Audit Planning and Supervision • Planning the project – the Standard Planning must be documented • Professional judgment should be used • Work plans should be established • Needs of potential users of the reports should be considered (regulators, the registrant, industry, public interests) • Auditor should understand what they are auditing • Controls around the area to be audited should be documented and understood • Procedures used should be specifically designed to test compliance and to detect non-compliance • Criteria needed to evaluate findings should be documented in the planning stage
Audit Planning and Supervision • Planning the project – the Standard Planning must be documented • Previous audits and findings should be used to focus the work plan • Data needed should be identified and requested through an information request from its source • Use of other auditor work or specialists should be considered and impact of that evaluated • Staffing should be sufficient to get the job done • Management should be contacted about audit planning – work plan and audit strategy should NOT be discussed in great detail • Most communication with management should be in writing • Professional judgment is key
Audit Planning and Supervision • Planning the audit – the Standard Planning should be documented Staff should be assigned that: • Know the work they are doing • Know the subject matter • Have the appropriate communication skills • Meet the appropriate continuing education requirements Staffing should be assigned • Staff should have the proper skills or “collective” knowledge base for the job • Assign enough staff to get the job done • Provide for on-the-job staff training • Bring in specialists when needed
Audit Planning and Supervision • Planning the audit Pre-audit procedures Organize data requests (e.g. via audit letter at least 60 days in advance) Cross reference information to authoritative documents (Approved Reliability Standards) Sufficient time to review documentation prior to field (site) work
