1 / 48

2007 SSC Survey Analysis Sub-Team Summary October 23, 2007

2007 SSC Survey Analysis Sub-Team Summary October 23, 2007 Team: Susan Frailey (Lead), Arne Logan, Jacky Gualandri, John Haddock, EJ Kegerreis, John Stecklow and Dina Heasley SSC 2007 Survey Results In 2007 EVERY person registered in eAuditNet was invited to participate in the survey.

niveditha
Télécharger la présentation

2007 SSC Survey Analysis Sub-Team Summary October 23, 2007

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 2007 SSC Survey AnalysisSub-Team SummaryOctober 23, 2007 Team: Susan Frailey (Lead), Arne Logan, Jacky Gualandri, John Haddock,EJ Kegerreis, John Stecklow and Dina Heasley

  2. SSC 2007 Survey Results In 2007 EVERY person registered in eAuditNet was invited to participate in the survey. In 2003 & 2005, invitations were sent only to eAuditNet Administrators.

  3. SSC 2007 Survey Results • Results shown are from the 1,085 respondents who provided their identity. • As with previous years, anonymous results are not reported for analysis purposes.

  4. Survey Addressed the following facets: • Audit Sites • Redundant Audits • Flow Down • Task Group • Audit Consistency • PRI • Participation and Awareness • General Comments

  5. Date Reported Reported by Health 2007 Supplier Survey Results Health Criteria Facet & Question No: Recommendations: Survey questions: Actions: Assigned to: Status Conclusions: 10/23/07 Susan Frailey N/A Facet: Audit Sites Questions: 6, 7, 8, 9 & 10 The 25 mile rule should be reconsidered based more on the Quality organisation and system in addition to distance. (NOTE: This pertains to approximately 18% of the respondents.) There is no rating of this facet since these are “information gathering” questions. 6. NEW QUESTION: Does your company hold multiple Nadcap approvals at different locations which fall outside of the 25-mile rule (as defined in NIP 7-01), and are not classed as satellite sites? 7. NEW QUESTION: How many different locations does your company have Nadcap approvals for? Please list the number below. 8. NEW QUESTION: If you have multiple Nadcap Approvals, are these operating under the same approved quality system as your main site? 9. NEW QUESTION: Is the quality system at the remote site controlled by a central quality department? 10. NEW QUESTION: How many miles/kilometers separate the central/main facility from the other locations? Please list for each location • Supplier Support Committee to make proposal for revision of the 25 mile rule. • SSC and NMC working group to finalize an acceptable approach to multiple site operation. • SSC • NMC/SSC • In-work • In-work • Majority of respondents are single site 82% • 97% of respondents have 5 sites or less. • 195 responses indicated that where they had multiple sites, they were outside of the 25 mile rule. • Of those that operate multiple sites 81% control Quality from a central function but 40% of those sites have their own Quality System.

  6. Date Reported Reported by Health 2007 Supplier Survey Results Health Criteria Facet & Question No: Recommendations: Survey questions: Actions: Assigned to: Status Conclusions: 10/23/07 Susan Frailey Facet: Redundant Audits RED Questions: 12, 13 & 14 • All Primes to review their internal systems to ensure they are not performing redundant audits • Create an eAuditNet feature for suppliers to log in prime visits, the reason for visit and number of visit days. • Create an eAuditNet feature for primes to input in general terms their findings without revealing the source of the findings, in order to validate Nadcap process. 12. REVISED QUESTION: Are Nadcap Prime User Members performing special process/commodity audits which are already covered within the scope of the Nadcap audit? 13. NEW QUESTION: You indicated that Nadcap Prime User Members are performing special process/product audits which are already covered within the scope of the Nadcap audit. Is it a result of any of the following? Please select all that apply (13a) 14. REVISED QUESTION: In 2005 and 2006, which Nadcap Prime User Members performed special process/product audits for which you were Nadcap accredited? Please list in the box below. OPTIONAL (or if you would like a response back), please list the date(s) of the audit(s) and name(s) of the auditor(s). • Primes to investigate why they are performing redundant audits and report back. • Create eAuditNet feature for suppliers • Create eAuditNet feature for primes • Primes • PRI • PRI • Pending acceptance of action • Pending acceptance of action • Pending acceptance of action • Question 12 is that which the health is measured. Questions 13 & 14 are follow-up. • Primes are still conducting visits that suppliers (63%) believe are redundant to Nadcap. Little change from 2005.

  7. Date Reported Reported by Health 2007 Supplier Survey Results Health Criteria Facet & Question No: Recommendations: Survey questions: Actions: Assigned to: Status Conclusions: 10/23/07 Susan Frailey Facet: Flow Down YELLOW Questions: 15 & 16 • Create a way for suppliers to electronically identify, who is not providing the specifications or flow down information and what is being or specifically not being provided. • Use contract flow templates as a recommended practice of obtaining required information (HT, CP) 15. Do you find flow down of specifications, their revisions, and other requirements from Nadcap Prime User Members and their Sub-Tiers a problem? 16. NEW QUESTION: How often does your company experience a problem obtaining flow down of material, temper condition, drawings, and other requirements from the Nadcap Prime User Members and their Sub-Tiers? • NMC to review the current flow-down practices to ensure that suppliers are receiving the required information. • NMC • Pending acceptance of action • Health measured for both questions as they are independent questions • (Q15) Flow of specifications a problem: Slight improvement from 2005. 43% of suppliers indicated a flow down problem (this is a 24% improvement from 2005). • (Q16) Flow down of required information a problem: on average 59% of suppliers have problems obtaining required information to perform the process.

  8. Date Reported Reported by Health 2007 Supplier Survey Results Health Criteria 10/23/07 Susan Frailey Facet & Question No: Facet: Task Group GREEN Questions: 19, 20, 25 & 28 Recommendations: Survey questions: 19 & 20: 1) Forward to the responsible groups the specifics/issues identified in the survey responses for action; 2) SSC better define for the supplier the SSC process of getting issues from the suppliers to the TG and the response to the Suppliers. Example: TG Supplier Voting Member (SVM), we have NMC Supplier Voting members who are responsible for the supplier TG voice; 3) SSC needs to educate suppliers of other data sources of clarifications… (e.g. eAuditNet) 19. NEW QUESTION: Do the Nadcap checklist questions need further clarification? If Yes, please identify the checklists that require additional clarification. 20. NEW QUESTION: If you answered Yes, please identify the checklists that require clarification. 25. NEW QUESTION: Is standardization among the Task Groups a problem (ex. merit criteria, failure criteria, classification of findings, etc.)? Please check all that apply. 28. NEW QUESTION: Do the Nadcap Prime User Members place sufficient priority on the globalization of standards? Actions: Assigned to: Status • Review and respond to identified issues • Define process of getting supplier issues to TG and TG responses to suppliers • Educate suppliers of available data sources of check sheet clarifications • Task Groups • SSC • SSC • Pending acceptance of action • Pending acceptance of action • Pending acceptance of action Conclusions: • Health is measured from questions 19 & 25. Question 20 is a follow-on to 19 and question 28 is insufficient to measure health. • Questions 19 & 20: 73% agree that the checklist require no further clarification. • 3) Question 25: 82% agree with Merit Criteria, 85% agree with Failure Criteria and 79% agree with the finding classifications. • 4) Question 28: 61% indicated sufficient priority was given to “Global Standardization”, but without knowing the how the term “Globalization of Standards” was interpreted, action cannot be taken from the data.

  9. Date Reported Reported by Health 2007 Supplier Survey Results Health Criteria Facet & Question No: Recommendations: Survey questions: Actions: Assigned to: Status Conclusions: 10/23/07 Susan Frailey Facet: Audit Consistency YELLOW Questions: 21, 22, 23 & 24 • Maintain efforts on auditor training to develop common understanding from auditor to auditor. • Provide auditors with training for areas which may not be their specialty, especially within those commodities where there are a wide range of processes. • Provide more focused training on how to audit. 21. If your company has had more than one (1) Nadcap audit within a special process/product, please rate the Nadcap auditors for consistency in their interpretation of the checklists. 22. If auditor to auditor consistency is a bigger problem in one special process/product than another, please describe below. 23. NEW QUESTION: In what areas do you find the Nadcap auditors inconsistent? Please check all that apply 24. NEW QUESTION: Following your audit(s), do you find consistency in how the Staff Engineer and the Nadcap auditor interpret your finding(s)? • Task groups to focus on improving auditor consistency during training, including the interpretation of the checklist questions. • Task groups to consider how auditors can improve their knowledge in the wider subject content of their area. • Staff Engineers and Auditor Training to develop a more in-depth auditor selection and training. • Provide specific training on “how to conduct an effective audit”. • Task Groups • Task Groups • PRI Staff Engineers / Auditor Training • PRI • In-work • In-work • In-work • Pending acceptance of action. • There is a 17% improvement in auditor consistency over 2005 result. • 30% of respondents feel that there is inconsistency. • Two key areas of concern are variations between auditors when reviewing the same process and the auditors interpretation of the checklist. • Concerns raised about auditors auditing skills. • 74% of respondents felt Staff Engineer and Auditor were consistent with their interpretation.

  10. Date Reported Reported by Health 2007 Supplier Survey Results Health Criteria Facet & Question No: Recommendations: Survey questions: Actions: Assigned to: Status Conclusions: 10/23/07 Susan Frailey Facet: PRI GREEN Questions: 26, 27 & 31 • Continue with efforts to translate check sheets into key languages. • Continue to endeavor to recruit auditors within key countries with appropriate language skills. • Improve flexibility in re-scheduling auto-scheduled audits. 26. NEW QUESTION: Have you experienced any problems with Nadcap related to language barriers? If so, please select all that apply. 27. If you have experienced a problem in this area, please list any suggestions on how PRI can improve. 31. REVISED QUESTION: In 2004, auto-scheduling for reaccreditation audits was implemented for most special processes/products. If your company has had an audit auto-scheduled, please indicate if this enhancement is beneficial. • Globalization Sub-Team continues to work the language issues. • PRI to continue recruiting auditors with language skills to alleviate language barriers. • PRI to improve upon the re-scheduling flexibility of auto-scheduled audits. • Globalization Sub-Team • PRI / Auditor Hiring • PRI / eAuditNet Administrators & Scheduling • In-process • In-process • In-work • 72% have no problems with language. Of those that did have a problem during an audit, 54% were in the USA or UK. • Still a strong desire among some to see checklists and audits carried out in their local language. • 60% agreed that auto-scheduled audits are beneficial. • 9% felt the limited flexibility was a concern with audit scheduled during plant shut-down or scheduled with not enough lead time.

  11. Health Criteria Date Reported Reported by Health 2007 Supplier Survey Results 10/23/07 Susan Frailey Facet & Question No: Facet: Participation & Awareness YELLOW Question: 30, 32, 33 & 34 Recommendations: Survey questions: Questions 30 & 32: re-word questions so action can be taken from results. Question 33: 1) Add link in eAuditNet to SCC section of PRI website to provide for greater exposure to the SSC and the benefits. 2) Simplify the navigation within the PRI website….too difficult to locate information….needs be no more than 2 clicks away Question 34: 3) Of those who want to be involved and gave the OK to contact them, need to be contacted; 4) For those not aware of opportunities need to come up with a way of making opportunities better known; 5) Of those not interested and gave the OK to contact them, need to be contacted to find out their reasons; 6) Next survey provide option of giving a reason why no interest. 30. NEW QUESTION: How would you rate the effectiveness of Nadcap WebEx meetings and/or trainings? 32. REVISED QUESTION: Do you plan to attend a Nadcap meeting in the next twelve (12) months? Please select all that apply…a) Yes, I will attend for the following benefits…b) No, I will not attend for the following reasons 33. REVISED QUESTION: How well do you feel the Supplier Support Committee (SSC) represents your interests and concerns? 34. NEW QUESTION: The Nadcap process needs supplier involvement in all areas of the process. Are you aware that you can volunteer for Task Group, Supplier Support Committee (SSC) and Nadcap Management Council (NMC) activities and teams? Actions: Assigned to: Status • Enhance eAuditNet with link to SSC information • Simplify navigation in PRI website to enhance supplier awareness • Create more awareness of SSC opportunities • Determine the level of interest of suppliers regarding their involvement in SSC. • PRI • PRI • SSC • SSC • Pending acceptance of action • Pending acceptance of action • Pending acceptance of action • Pending acceptance of action Conclusions: • Health measured from Questions 30, 33, & 34. Question 32 is an informational question. • Question 30: Effectiveness of WebEx - 46% had taken advantage of WebEx and 72% of those indicated it was effective. • Question 32: Informational question about reasons for either attending Nadcap meetings or not. • Question 33: SSC representation - 57% responded they are aware of the SSC and 79% of those feel the SSC represents them Very Well or Well. However, 43% of total were not aware of the SSC. • Questions 34: Supplier involvement - 36% not aware of involvement opportunities; 17% want to be involved; 8% indicated they are currently involved; 39% had no interest and no option to provide reason why.

  12. Date Reported Reported by Health 2007 Supplier Survey Results Health Criteria Facet & Question No: Recommendations: Survey questions: Actions: Assigned to: Status Conclusions: 10/23/07 Susan Frailey Facet: General Comments GREEN Questions: 11, 18 & 29 • PRI Staff, NMC, SSC, et al, should continue to work to ensure all suppliers’ best interests are being addressed. • Notify companies that have responded to SSC Survey of the results/action taken. 11. If your company has been accredited for one (1) year or more, and your company measures business and/or quality trends, have you seen improvement in this area(s) related to your Nadcap accreditation(s)? Select all that apply. 18. REVISED QUESTION: Prior to your audit, did you take advantage of pre-audit preparation tools? If yes, which of the following did you find helpful? Please check all that apply. 29. How well do you think the Nadcap process takes the interests of the suppliers into consideration? • SSC/PRI to consider how all responding companies will view the Survey Results / Actions Taken. • SSC / PRI • Pending acceptance of action. • 95% of respondents have been accredited for one year or more and measure business and quality trends. • 76% of respondents both reported an increase in quality and 45% noted a business increase. This is a 25% increase in business over 2005 data. • 97% of respondents utilized pre-audit preparation tools of some sort. • 74% of respondents felt their best interests were taken in to consideration. This is a 7% increase from 2005. • 26% of the respondents still feel their best interests are not taken into consideration. This is down from 31% in 2005 (Result is a 16% positive trend).

  13. RECOMMENDATIONS TO IMPROVE NEXT SURVEY

  14. RECOMMENDATIONS TO IMPROVE NEXT SURVEY (cont’d.)

  15. Health Criteria Not Applicable Legend

  16. Health Criteria Legend

  17. SSC 2007 Survey Results

  18. SSC 2007 Survey Results Responses by sector. 2007 2005 Asia 9 2% Asia 90 8% Europe 70 20% Americas 688 64% Europe 306 28% Americas689 64% Americas 275 77% Data not recorded in 2003.

  19. SSC 2007 Survey Results Response by Task Group: Results from 2005 & 2003 surveys that reviewed similar data are included as an embedded document.

  20. SSC 2007 Survey Results NEW QUESTION: Does your company hold multiple Nadcap approvals at different locations which fall outside of the 25-mile rule (as defined in NIP 7-01), and are not classed as satellite sites?

  21. SSC 2007 Survey Results NEW QUESTION: How many different locations does your company have Nadcap approvals for? Please list the number below. 4.5 is the average Complete responses are enclosed on the document below.

  22. SSC 2007 Survey Results NEW QUESTION:If you have multiple Nadcap Approvals, are these operating under the same approved quality system as your main site? Responses are from the 18% who answered Yes to previous question. 117 responses 79 responses

  23. SSC 2007 Survey Results NEW QUESTION:Is the quality system at the remote site controlled by a central quality department ? 159 responses Responses are from the 18% who answered “Yes” to the first satellite question. 37 responses

  24. SSC 2007 Survey Results NEW QUESTION: How many miles/kilometers separate the central/main facility from the other locations? Please list for each location. Responses are from the 196 survey participants who indicated that they have a facility that falls outside of the satellite audit rule. Average is 441 miles Complete responses are on the document below.

  25. SSC 2007 Survey Results REVISED QUESTION: Are Nadcap Prime User Members performing special process/commodity audits which are already covered within the scope of the Nadcap audit? Results from 2005 & 2003 surveys that reviewed similar data are included as an embedded document.

  26. SSC 2007 Survey Results NEW QUESTION: You indicated that Nadcap Prime User Members are performing special process/product audits which are already covered within the scope of the Nadcap audit. Is it a result of any of the following? Please select all that apply. Responses are from the 63% who responded Yes to previous question

  27. SSC 2007 Survey Results REVISED QUESTION:In 2005 and 2006, which Nadcap Prime User Members performed special process/product audits for which you were Nadcap accredited? Companies listed 50+ times in order: Results from 2005 survey that reviewed similar data is included as an embedded document. This question was not asked on the 2003 survey.

  28. SSC 2007 Survey Results Do you find flow down of specifications, their revisions, and other requirements from Nadcap Prime User Members and their Sub-Tiers a problem? In 2003, 5% selected “Other”

  29. SSC 2007 Survey Results NEW QUESTION: How often does your company experience a problem obtaining flowdown of material, temper condition, drawings, and other requirements from the Nadcap Prime User Members and their Sub-Tiers? Sub-Tier Primes

  30. SSC 2007 Survey Results NEW QUESTION: Do the Nadcap checklist questions need further clarification? If Yes, please identify the checklists that require additional clarification. Checklists that were listed and number of times: AQS 8 CP 69 CT 9 COMP 9 FLU 1 HT 47 MTL 36 NDT 43 NM 6 SEALS 4 SE 4 WLD 14 Additional comments and specific documents listed on document below.

  31. SSC 2007 Survey Results NEW QUESTION:Is standardization among the Task Groups a problem (ex. merit criteria, failure criteria, classification of findings, etc.)? Please check all that apply.

  32. SSC 2007 Survey Results NEW QUESTION: Do the Nadcap Prime User Members place sufficient priority on the globalization of standards?

  33. SSC 2007 Survey Results If your company has had more than one (1) Nadcap audit within a special process/product, please rate the Nadcap auditors for consistency in their interpretation of the checklists.If auditor to auditor consistency is a bigger problem in one special process/product than another, please describe below. 162 Comments were made. Task Groups mentioned: CP 36 HT 30 NDT 24 WLD 10 NM 6 MTL 5 CT 3 SE 2 Other comments are general to the process and included on the document below.

  34. SSC 2007 Survey Results NEW QUESTION: In what areas do you find the Nadcap auditors inconsistent? Please check all that apply.

  35. SSC 2007 Survey Results NEW QUESTION: Following your audit(s), do you find consistency in how the Staff Engineer and the Nadcap auditor interpret your finding(s)?

  36. SSC 2007 Survey Results NEW QUESTION: Have you experienced any problems with Nadcap related to language barriers? If so, please select all that apply.If you have experienced a problem in this area, please list any suggestions on how PRI can improve. 130 Comments/ Suggestions were made; hire auditors that speak language of supplier, translate checklists in other languages, allow responses to be submitted in any language, etc. All comments are included on the embedded document. See next slide for country breakdown

  37. SSC 2007 Survey Results Language Question – Additional Data: Countries of respondents who indicated they had a problem with language during a Nadcap audit:

  38. SSC 2007 Survey Results REVISED QUESTION: In 2004, auto-scheduling for reaccreditation audits was implemented for most special processes/products. If your company has had an audit auto-scheduled, please indicate if this enhancement is beneficial. 75 Comments made; no opinion, audits are scheduled too far in advance, more flexibility is needed, etc. All comments are included on the document below. 2005 Results are included, data not collected in 2003.

  39. SSC 2007 Survey Results NEW QUESTION: How would you rate the effectiveness of Nadcap WebEx meetings and/or trainings?

  40. SSC 2007 Survey Results REVISED QUESTION: Do you plan to attend a Nadcap meeting in the next twelve (12) months? Please select all that apply. Yes, I will attend for the following benefits… Results from 2005 & 2003 surveys that reviewed similar data are included as an embedded document.

  41. SSC 2007 Survey Results REVISED QUESTION: Do you plan to attend a Nadcap meeting in the next twelve (12) months? Please select all that apply. No, I will not attend for the following reasons...

  42. SSC 2007 Survey Results REVISED QUESTION: How well do you feel the Supplier Support Committee (SSC) represents your interests and concerns? Results from 2005 & 2003 surveys that reviewed similar data are included as an embedded document.

  43. SSC 2007 Survey Results NEW QUESTION: The Nadcap process needs supplier involvement in all areas of the process. Are you aware that you can volunteer for Task Group, Supplier Support Committee (SSC) and Nadcap Management Council (NMC) activities and teams?

  44. SSC 2007 Survey Results If your company has been accredited for one (1) year or more, and your company measures business and/or quality trends, have you seen improvement in this area(s) related to your Nadcap accreditation(s)? Select all that apply.

  45. SSC 2007 Survey Results REVISED QUESTION: Prior to your audit, did you take advantage of pre-audit preparation tools? If yes, which of the following did you find helpful? Please check all that apply. Results from 2005 & 2003 surveys that reviewed similar data is included as an embedded document.

  46. SSC 2007 Survey Results How well do you think the Nadcap process takes the interests of the suppliers into consideration? Data not recorded in 2003.

  47. The slide 25 can be misleading because respondents can give multiple answers.  If we assume 10 people responded and 6 said there was no problem but the other 4 answered in the negative 3 times. Problem1    Problem2    Problem3    no problem 4                4                    4                6                        total 10 respondents 40%            40%                40%            60%                response per question / 10  total does not add up to 100% if the number of responses is used as the number respondents (i.e. response per question/18) 22%            22%                22%            33% These give a different slant on the result and add up to 100% so it is likely that this is the way the calculation in the chart was made.

More Related