NSWC Corona-MS Interval DJ June 2002

1. Dr. Dennis Jackson 909-273-4492 DSN 933-4492 JacksonDH@Corona.Navy.Mil NSWC Corona-MS Interval DJ June 2002 1

2. CALIBRATION INTERVAL ANALYSIS: CURRENT AND FUTURE Dr. Dennis Jackson MS30A1 June 2002

3. Overview • Current Calibration Interval Methods • Interval Analysis Results • New Approaches to Calibration Interval Estimation

4. Current Methods:What Is a Calibration? • Compare the measurement values from a UUT with the measurement values from a calibrator. • Deviation = UUT Measurement – Calibrator Measurement • A UUT is considered in tolerance if: • Lower Tolerance < Deviation < Upper Tolerance • Measurement Reliability is the probability of being in tolerance. • A Calibration Interval is the amount of time between calibrations that will meet a measurement reliability target (keeps the UUT in tolerance).

5. 100 90 80 70 60 50 40 30 20 10 0 Test Equipment Reliabilityvs. Calibration Interval Measurement Reliability (%) 0 6 12 18 24 30 36 42 48 Calibration Interval (Months) Current Methods:Calibration Interval Determination 72% EOP Reliability for GPTE 85% EOP Reliability for Safety-of-Flight and Mission Critical

6. No Further Review Engineering Interval Est. 1 2 3 5 4 Integrated Interval Est? Yes Gather Relevant Data QA Division Review METRL Statistical Interval Est. No Policy Review TR-6 Current Methods: Stages of the Calibration Interval Process

7. Interval Analysis Results:NAVSEA Interval Changes (FY 2002 through April 2002)

8. Interval Analysis Results:Annual Calibration Cost Avoidance (Based on changes made in FY 2002 Through April 2002)

9. New Approaches to Calibration Interval Estimation • Near Term - Binomial Calibration Interval Estimation Methods • More accurate interval estimates • Alternative reliability models • Visual analysis methods • Long Term - Variables Data Calibration Interval Estimation Methods • Fixes data problems • More information on measurement characteristics • Less data required • MEASURE 2 capability with automated data

10. Traditional Reliability Methods Assumptions: You know when the failure occurs. R = 1.0 at time 0. Data: Failure Times. Exponential Model: R = exp(-t)

11. Tolerance Testing Data • Characteristics: • The failure occurs during an interval. • R < 1.0 at time 0. Note: The points on this graph are observed in tolerance proportions.

12. Using Traditional Methods On Tolerance Testing Data • Problem: • The estimates don’t match the data because the intercept must go through 1.0.

13. Reliability Methods For Tolerance Testing Data Assumptions: The failure occurs during an interval. R < 1.0 at time 0. Data: Success/Failure (Binomial) Intercept Exponential Model R = Ro exp(-t) = exp(0+ 1t)

14. Current Status of Near Term Efforts • 2002 MSC Paper: “Calibration Intervals – New Models and Techniques” • Binomial Analysis, New Models, Reliability Intercepts, Initial Variables Methods • Binomial Calibration Interval Analysis System

15. Benefits of Binomial Calibration Interval Estimation Methods The use of Binomial estimation methods provides more accurate calibration interval estimates based on current statistical estimation theory. Binomial estimation methods allow for alternative measurement reliability models, including intercept and multivariable models. Better graphical tools provide more understanding of test equipment behavior.

16. Long Term Approach: Variables Calibration Data

17. Calibration Intervals Based on Variables Data • Compute a Drift Trend. • Compute a Variability Trend using residuals from the drift trend. • Obtain a Reliability Curve using the drift and variability trends. • Determine the Calibration Interval from the reliability curve. • Predict the Measurement Uncertainty using the drift and variability trends.

18. Drift Trend Analysis E(d) = B0 + B1 t (Weighted Linear Regression on d)

19. Variability Trend Analysis E(res2) = C0 + C1 t (Linear Regression on res2)

20. A Basis for Increasing Variability Generally, a single serial number does not show increasing variability

21. A Basis for Increasing Variability However, several serial numbers could have slightly different slopes and intercepts:

22. A Basis for Increasing Variability The overall effect is one of increasing variability for the population

23. Reliability Curve Analysis

24. Determining Calibration Intervals From Variables Data

25. Current Statusof Long Term Efforts • 2002 MSC Paper: “Calibration Intervals – New Models and Techniques” • Binomial Analysis, New Models, Reliability Intercepts, Initial Variables Methods • 2003 MSC Paper: “Calibration Intervals and Measurement Uncertainty Based on Variables Data” • NPSL, SCE • Variables Analysis Excel Tool • Estimates Trends, Calibration Intervals, Measurement Uncertainty • MEASURE 2 • Automated/Electronic data

26. Benefits of UsingVariables Data MEASURE data is often suspect In-Tolerance data is difficult to verify (success/failure) Engineering review required for nearly all calibration interval determinations Variables data is more trustworthy This could significantly increase the number of interval analyses Variables data provides much more information Requires fewer calibrations to accurately determine a calibration interval than In-Tolerance data Development of automated/electronic data recording could reduce calibration time.

27. Summary Calibration intervals minimize the amount of calibration effort required to keep test equipment adequately in tolerance. Recent adjustments to calibration intervals will result in significant cost avoidance. Near-term improvements using Binomial methods will provide better visual analysis and more accurate estimation techniques. Long-term improvements using variables data methods will: Fix data problems Provide faster analyses with less data Possibly reduce administrative part of calibration time