A mericas A erospace Q uality G roup AAQG Project Management

1. Americas Aerospace Quality GroupAAQG Project Management AAQG Project #61 (IAQG REQ-009) IAQG 9101:2009 Revision - Quality Management System Assessment Reported by: Tony Marino Date of Report: 2009-March-11

2. What happened and where is 9101? • Released in October 2008 for ballot • AAQG writing team members did not agree it was ready. • Consulted with AAQG leadership to release or move for sector rejection – decision for sector rejection. • Primary Issues • Writing was a balanced writing addressing both Conformity and Effectiveness. • Writing structure – too confusing. • Contained guidance material. • Differences of opinion on what could be changed during and after ballot. • Document revisions have been made and process of finalization for ballot.

3. IAQG REQ-009 Working Group Members Hans Luijt – Team Lead ASD-EASE - Netherlands Stuart Anthony Rolls-Royce UK Eddy Benattar IAI - Israel Alain Bonnel SAFRAN - France Conny Flink Volvo Aero - Sweden Michel Jacquaniello Zodiac - France Marie-Laurence Petit Airbus - France Brian Burroughs* LRQA – UK Michel Clément* Bureau Veritas - France Larry Beck Lockheed Martin – US (TX) Tom Gallagher Hamilton Sundstrand (UTC)-US Tony Marino The Boeing Company – US (WA) Robert Keys* DNV – US (TX) Paul Dionne* Intertek Masahiro Kawamoto Mitsubishi Heavy Industries – Japan

4. 9101: Revision Objectives • Ensure alignment with IAQG strategy (on-time, on-quality performance, Strengthen of Other Party certification) • Support the process approach for quality management systems as described in the 9100-series standards • Incorporate ISO 17021 changes and ICOP resolutions • Define requirements for all activities of audit program • Define adequate audit report that provides useful information to stakeholders on Organization Processes Performance And Effectiveness ( PEAR ) and conformity (Objective Evidence Record) to 9100 series . • Define audit records (objective evidence,…) to support ICOP Scheme

5. AAQG P61 - IAQG 9101:2008 Revision - Quality Management System Assessment Tony MarinoJack Flecher 2006-Oct-05 )6/20/08 • IAQG Gate 0 Approved October 5, 2006 - Seville Spain (REQ-009) • Current 9101 does not focus 9100 assessments on process based management • Too much emphasis on conformity of assessment – not auditing what is important i.e. performance, continuous improvement, customer satisfaction • Linkage to certification: certificate too much of a ‘drivers license, instead of a tool to prevent ‘accidents’. • Scoring is complex, time consuming, subjective and purpose of is unclear. • Strong relationship of assessment documents (9101, 9111 and 9121). • Revised 9101 • Develop position and decision to obsolete 9111 and 9121 • Aerospace Experienced Auditor training and certification/ recertification recommendations • Upon release of ISO9001:2000, assessment tool and methods for 9100 was not updated to assess the process approach. • New 9100 release in April 2009 will require update of 9101 • Publication of 9101 • Auditor efficiency increase • Greater fidelity of information in audit reports • Update 9101 to be more than an assessment (checklist) document. • Publish assessment methods and tools that will provide more meaningful information to the supplier, client and stakeholders • Consolidate 9101, 9111 and 9121 into one document • Project Milestone

6. P61 - IAQG 9101:2008 Revision - Quality Management System Assessment Tony Marino Y 2008-Sept, 11 2009-March, 11 Kristy Heffernan • Project Description • Restructure 9101 with the next revision of 9100 to combine the 9101/9111/9121 standards to reflect a process based auditing approach of 9100, 9110 and 9120 standards. • Auditor training and competency requirements development. • Benefit of project • Alignment of assessment tools and methods to process based management QMS. • Shift audit focus to process, product, system performance and customer satisfaction. • Reduced variation of training and training expectations. • AAQG SWOT and sector rejection of ballot document Nov 23, 2008. • Training specification and competency requirements development Nov 17-20, 2008 – Paris • Team F2F to address issues and actions. Huntington Beach CA. Jan. 2009. • Proposed 2nd ballot draft complete Feb 22, 2009. • Team comments submitted updated draft. Feb 24, 2009 • Completed comment incorporation – March 10, 2009 • AAQG Briefing F2F Portland. March 10-12, 2009 • IAQG Release of reballot. March 11, 2009 • AAQG Release of 2nd ballot. March 20, 2009 • AAQG Ballot cycle. March 20-April 20, 2009 • EAQG Ballot cycle. March 20-May 20, 2009 • APQG Ballot cycle. March 20-May 20, 2009 • Target Release July 2009 None – Sector rejection by the AAQG and member rejection by EAQG – Germany, triggered a revision to the document and 2nd ballot process. Schedule for 2nd ballot complete.

7. Process Approach Audit Records Tools Evidence of Conformance Process-Based Auditing Evidence of Effectiveness Inputs Output Core, Support & Management Processes Effectiveness Focused Development Model for aProcess-Based 9101

8. 9101: Major Changes Review • Introduction in the audit report of: • an “Process Effectiveness Assessment Record” form (PEAR) which : • State if for the audited process has been determined that • planned activities and results are realized • planned activities are realized but planned results are not realized • planned activities and results are not realized • Give objective evidence, observed conditions, data, and comments to support effectiveness determination • a process based “Audit Completeness Record” • an “Objective Evidence Record” capturing quality manual and procedures with objective evidence of conformity

9. 9101: What has not changed • Determining conformance to the standard. • Nonconformance discovered during the audit are to be documented and must be closed before certification. • Based on information collected during the assessment, the auditor will have to draw conclusions on conformance of the organization quality management system. Note: Conclusions on effectiveness are new

10. 9101: Progress Status • Issue of a coordination draft : 30 June 2008(Limit date for comments : 7 July 2008) • Review of the comments(8-11 July in Amsterdam) • 450 comments received, 233 accepted, 15 partially accepted • 126 comments reviewed and disposition from ballot rejection (January 9-12, 2009 in Huntington Beach 9101 Formal Ballot • Publication of the 9101 Formal ballot : Sept./Oct 2008 • Disposition on comments : end Nov. 2008 – Rejected • Comments addressed – January 2009 • Preparing for ballot release – March 2009 9101 Release Target Release – July, 2009

11. 9101 Status Update • Ballot version published 26 September 2008 • Results of ballot: • APQAG : passed, with comments • AAQG: rejected, with comments • EAQG: • France: passed, as is • Germany: rejected, with comments • Italy: passed, as is • UK: passed, as is • Switzerland: abstention, no experts • Sweden (VAC): passed, with comments • 9101 team conclusion: address the comments and go for 2nd ballot

12. Feedback summary • 126 detailed comments received • Major comments: • Checklist / record for objective evidence is missing (US, Germany, Sweden) • Structure of the document needs improvement (AAQG) • During OP assessment NO attention should be given to customer specific QMS requirements (Germany) • PEAR form and approach is too complex, e.g.‘all processes’ • 9101 team proposals: • To add record for objective evidence • To restructure part of the standard: • All methodology / approach issues will be combined in one chapter • Recommended practices will be in a separate document (ARP, TR?) • Use of PEAR form limited to Product Realization Processes only

13. Dispositions • 126 comments received • Team meeting in Huntington Beach 10-12 February, 2009 • Disposition statistics: • Accepted: 59 • Partially accepted: 17 • Rejected: 39 • Deferred: 8 • Noted: 3

15. 9101 Release Schedule Maximum 30 month Implementation from Publication 02/09 07/09 03/09 04/09 01/09 IAQG Phoenix 9100:2009 Publication 9101 Ballot start New 9101 Publication • Notes: • All changes in 2nd version are related to comments • No new requirements, except use of Objective Evidence Record as checklist • Request: to reduce 2nd ballot in Europe to 2 months

16. QuestionsComments/Feedback to:anthony.p.marino@boeing.com

17. Back Up

18. AS9101 Process Audit Flow From Pre-audit Planning 3.0 Complete appendix # 6 Customer Quality Data (not applicable for this audit) 1.0 Conduct Process and Business Research 2.0 Conduct Planning Interview Stage 1 7.0 Develop micro audit plan 4.0 Data Collection 5.0 Complete appendix # 1 Identify clauses and processes 6.0 Complete appendix # 5 Stage 1 Audit 8/20/08 – 9/3/08 Due 9/3/08 Due 9/3/08 Stage 2 8.0 Conduct Interviews & follow Audit Plan 9.0 Complete appendix # 4 Summary 10.0 Update appendix # 1 11.0 Complete appendix # 2 Write NCRs Conclusions & Reporting 12.0 Complete appendix # 7 EAR Form 13.0 Convene team for Consolidation / roll up (at lead discretion) 14.0 Complete appendix # 3 Audit Report Audit Lead Auditors

19. Beta Test Results – 1st Party View • Review and analysis of AS9101 Coordination Draft • Conclusion made by the auditors was that the approach is good. Our auditors are trained in the process approach and use the ISO 9001 Auditing Practices Group guidance documents. • All elements of a process audit were captured and defined but lacked sequencing of the activities. • Process flow was created. • Audit team formed and training on the process was conducted.

20. AAQG Project #61 (IAQG REQ-009)Scope Amend 9101 Assessment standard in order to enhance its clarity and compatibility with process based management QMS (ISO 9001) and IAQG 9100 Standard update to be published mid 2009

21. AAQG Project #61 (IAQG REQ-009)Purpose of Revision • Focus a methodological approach of quality systems audits • Enhance clarity of standard to resolve issues on how to use the standard • Keep a standard report format • Consider all stakeholder needs • Ensure that the audit approach is more process oriented • Extend the scope of the audit process not only to determine conformity but • to include the assessment of the organization’s ability to meet customer, regulatory • and the organization’s own requirements

22. Meeting 13-15 November 2007 ToulouseSummary of discussions/decisions • Scoring will no longer be required under the new 9101 • Audit Process Matrix list with future 9101 topics agreed • Inputs for new 9101 determined: • Audit Process Matrix List • ISO TC 176 APG material • Stakeholder inputs • Automotive IATF material for for auditing ISO/TS 16949 • Requirements flow down chapter from SCMH • Referenced documents determined: • ISO/IEC 17021 Part 1 and 2 (if available) • ISO 19011 • IAQG 9104 • IAQG 9100/9110/9120 • Relation with new 9101 structure and referenced documents ‘mapped’ • Detailed writing rules for drafting new 9101 were agreed • Writing tasks assigned to sub-teams • Schedule revised, now aligned with 9100 schedule

23. Meeting 12-13 February 2008OrlandoSummary of discussions/decisions • Sub-team report out • Assembled working draft • Working Draft Review • Some requirements had direct relationship to AS9104 rewrite • Developed list of input for Berlin OPMT meeting • Team size is becoming a factor. • Agreed after Naples working session, team would be scoped down increase velocity of development. • Output of smaller team would come back to original team.