Download Presentation
## Flight Test and Statistics

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -

**Flight Test and Statistics**PRESENTED BY Richard Duprey Director, FAA Certification Programs National Test Pilot School Mojave, California**Flight Test and Statistics“If you want to be absolutely**certain you are right, you can’t say you know anything.”**Flight Test and Statistics Overview**• Background on National Test Pilot School • Coverage of Statistics • Scope - six hours of academics • Detail • Use of statistics in flight test • Types of questions we try to answer**NTPS Background**• Private non-profit • Grants Master Science • Only civilian school of its kind • SETP equivalent to USAF and Navy Test Pilot Schools • Offers variety of courses (Fixed Wing and Helicopters) • Professional - 1 year • Introductory • Performance and Flying Qualities Testing • Systems Testing • Operational Test and Evaluation • NVG • FAA Test Pilot / FTE initial and recurrent training**Data Analysis**0 z**Data Analysis - Hour 1**• Types of Errors • Types of Data • Elementary Probability • Classical Probability • Experimental Probability • Axioms • Examples**Introduction**• Flight testing involves data collection • time to climb • fuel flow for range estimates • qualitative flying qualities ratings • INS drift rate • Landing and Take-off data • Weapon effectiveness • All of these experimental observations have inaccuracies • Understanding these errors, their sources, and developing methods to minimize their effect is crucial to good flight testing**Types of Errors**• There are two very different types of errors • systemic errors and random errors • Systemic errors • repeatable errors • caused by flawed measuring process • ex: measuring with an 11 inch ruler or airspeed indicator corrections • Random errors • not repeatable and usually small • caused by unobserved changes in the experimental situation • errors by observer - reading airspeed indicator • unpredictable variations - small voltage fluctuations causing fuel counter errors • can’t be eliminated but typically distributed about a well defined distribution**Types of Data**• There are four types of numerical data: • NOMINAL DATA • numerical in name only - say an aircraft configuration • 1 = gear down, 2 = gear up, 3 = slats extended • normal arithmetic processes not applicable • 3 >1 or 3-1=2 are not valid relationships • ORDINAL DATA • contains information about rank order only • #1 = C-150, #2 = B-1, #3 = F-15 • in terms of max speed: 3>1 is valid, but not 3-1=2**Types of Data**• There are four types of numerical data (continued) • INTERVAL DATA • contains rank and difference information - ex: temperature in degrees Fahrenheit • 30, 45, 60 at different times, 15 deg. difference • zero point arbitrary, so 60o F is not twice 30oF • RATIO DATA • all arithmetic processes apply • most flight test data falls into this category • Can say that a 1000 pound per hour fuel flow is 4 times greater than 250 PPH**Probability and Flight Test**• Quantitative analysis of random errors of measurement in flight testing must rely on probability theory • Goal • Student to understand what technique is appropriate and limitations on the results**Elementary Probability**• The probability of event A occurring is the fraction of the total times that we expect A to occur - • Where: - P(A) is the probability of A occurring • - na is the number of times we expect A to occur • - N is the total number of attempts or trials**Elementary Probability**• From this definition, P(A) must always be between 0 and 1 • if A always happens, na = N and P(A) = 1 • if A never happens, na= 0 and P(A) = 0 • In order to determine P(A) we can take two different approaches • make predictions based on foreknowledge (“a priori”) • conduct experiments (“a posteriori”)**Classical (‘a priori’) Probability**• If it is true that • every single trial leads to one of a finite number of outcomes • and, every possible outcome is equally likely • Then, • na is the number of ways that A can happen • N is the total number of possible outcomes • For example: • six-sided die implies six possible outcomes: N = 6 • if A is getting a 6 on one roll, na = 1 • P(A) = 1/6 = 0.1667**Second Example**• What is the probability of getting two heads when we toss two fair coins? • There are four possible outcomes (N = 4) • (H,H) (H,T) (T,H) (T,T) • na = 1 since only one of the possible outcomes results in two heads (H,H) • Thus P(A) = 1/4 = 0.25**Classical (‘a priori’) Probability**• Approach instructive • Generally not applicable to flight test where: • Possible outcomes infinite • Each possible outcome not equally likely • Leads us to second approach**Experimental (‘a posteriori’) Probability**• Experimental probability is defined as • Where - nA obs is the number of times we observe A Versus . number of times we expect A to occur - Nobs is the number of trials**Experimental Example**• If the probability of getting heads on a single toss of a coin is determined experimentally, we might get 1.0 Porb (heads) 0.5 0 norb 1000 100 10 1**Probability Axioms**• Probability Theory can be used to describe relationships between events**Probability Axioms**• Three probability axioms are easily justified as opposed to proven • P(not A) = 1 - P(A) • Probability of something happening has to be one • P(A or B) = P(A) + P(B) • P(H or T) = 0.5 + 0.5 =1 for a single coin • P(A and B) = P(A) x P(B) • P(T and T) = 0.5 x 0.5 = 0.25 for two coins • same answer we got when examining all possible outcomes • The last two axioms require that • each outcome is independent • A occurring doesn’t affect probability of A or B occurring • each outcome is mutually exclusive • Only one can occur in a single trial**Example**• Problem: • Based on test data, 95% of the time an F-4 will successfully make an approach-end barrier engagement on an icy runway • what is the probability that at least one of a flight of four F-4’s will miss? • Solution: • P (1 or more miss) = 1 - P(all engage) • Probability that at least one will miss is the complement of the probability that all will engage • P (all engage) = P(1st success) × P(2nd ) × P(3rd) × P(4th) = 0.95 × 0.95 × 0.95 × 0.95 = 0.954 = 0.81 Thus, • P (1 or more miss) = 1 - 0.81 = 0.19**Example**• Problem: • What is the probability of getting 7 or 11 on a single roll of a pair of dice? • Solution: • Since getting 7 or 11 are independent, mutually exclusive events, we can say • P (7 or 11) = P (7) + P (11) • N = 62 = 36 • n7 = 6 • (6, 1) (1, 6) (5, 2) (2, 5) (4, 3) (3, 4) • n11 = 2 • (6, 5) (5, 6) • Thus, • P (7) = 6/36, P (11) = 2/36 • P (7 or 11) = 6/36 + 2/36 = 0.222**Data Analysis - Hour 2**• Populations and Samples • Measures of Central Tendency • Dispersion • Probability Distributions • Discrete • Continuous • Cumulative**Population & Samples**• A population is all possible observations • Many populations are infinite • A pair of dice can be rolled indefinitely • Population of F-117 weapons deliveries is all the possible drops it could make in its lifetime • Some populations are limited • Votes by registered Republicans • A sample is any subset of a population • For example • 100 rolls of a pair of dice • Bomb scores for 100 weapon delivery sorties**Population Constructs**• Constructing a population • Must impose assumptions • Homogenous • Independent • Random**Sample Requirements**• Homogeneous • the data must come from one population only • DC-10 take-off data shouldn’t be used with MD-11 • Independent • selecting one data point must not affect subsequent probabilities • selecting and removing a heart from a deck of cards changes the probability of drawing another heart • DC-10 landing 75 feet past touchdown aim point on one landing doesn’t change probability that next landing will miss by same distance (or any distance) • Random • equal probability of selecting any member of population • using a member of a population with a bias would be non-random • F-16 with boresight error would cause a bias in downrange miss distance**Measures of Central Tendency**• Given homogenous, independent, random sample, need to describe the contents of that sample • Measure steel rod diameter with a micrometer - would get several different answers • Tighten the micrometer • Dust particles on the rod • Reading scale on micrometer • What to do with answers that are different?**Measures of Central Tendency**• There are three common measures of central tendency: • Mean (arithmetic average) - most commonly used • Mode • most common value in the sample • there may be more than one mode • Median • middle value • for an even-numbered sample, average the two middle values • Dangers ........**Dispersion**• Just reporting the mean as the answer can be very misleading • Consider the following two samples, both with a mean of 100 (and same median as well) • Sample 1: 99.9, 100, 100.1 • Sample 2: 0.1, 100, 199.9 • We also need to report how much the data generally differs from the mean value**Deviation**• We define deviation as the difference between the ith data point and the mean: • Averaging the deviations does not help:**Mean Deviation**• Since there as many deviations above and below the mean, we could average the absolute values of deviations:**Standard Deviation**• While the mean deviation can be used, the standard deviation s is a more common measure of dispersion: • versus • The square of the standard deviation, s2, is called the variance**Notation**• Normally, we use Greek letters to denote statistics for populations: m for population mean s2 for population variance • And we use Roman letters for sample statistics: for sample mean s2 for sample variance**Sample Standard Deviation**• One other difference exists between s and s • The sample standard deviation has the sum of the squares divided by N - 1 versus N • Mathematically, this is due to a loss of one degree of freedom • The effect is to increase the standard deviation slightly • Difference decreases as sample gets larger**Flight Test Example - PA28 Takeoff Distance**• Two data points eliminated - wrong configuration, improper technique • Data adjusted for standard weight (2150 lbs.), runway slope (GPS), temperature, pressure, airspeed/altimeter corrections • Technique, rotate at 65, liftoff at 70, maintain 75 until 50 feet AGL**Probability Distributions**• Statistical applications requires understanding of the characteristics of the data obtained • Probability distributions gives us such understanding**Probability Distributions**• To understand probability distributions, consider the problem of tossing 2 coins • Let n represent the number of heads for a single toss of both coins • Then the probabilities of getting n = 0, 1, or 2 can be calculated: • for n = 0, P(0) = 0.25 • for n = 1, P(1) = 0.5 • for n = 2, P(2) = 0.25**Discrete Distributions**• We can present the data as a bar graph**Empirical Distributions**• In flight test, we are concerned with empirical distributions versus theoretical in the coin example • If we collect data on landing errors:**Continuous Distributions**• If we get more and more data, and make the intervals smaller, our histogram approaches a continuous curve: Continuous Probability Distribution of Touchdown Miss Distance • Can’t be interpreted same way as the previous discrete distribution**Continuous Distributions**• Height of curve above a point is not the probability of “x” having that point value • Any one point on the x-axis represents a non-zero point on the curve • But the probability associated with that single point must be zero, since there are an infinite number of points on the x-axis • We can meaningfully talk only about the probability of being between two points a and b on the x-axis**Probability as Area Under Curve**• The probability of getting a result between a and b is rep-resented by the area under the probability distribution curve between a and b f (x) P(a £ x £ b) x**Cumulative Probability Distribution**• A cumulative probability distribution gives the probability that x is less than or equal to some value, a • Relative probability of aircraft landing miss distances could be displayed in the following cumulative distribution 1.0 0.95 f (x) 0.5 x xT**Data Analysis - Hour 3**• Special Probability Distributions: • Binomial • Normal • Student’s t • Chi squared**Binomial Distribution**• The binomial is a discrete distribution • It tells us the probability of getting n successes in N trials given the probability (p) of a single success • Limiting cases • if n = N, then obviously P(N) = pN • if n = 0, then P(0) = (1 -p)N • or, letting q = 1 - p, P(0) = qN • For 0 < n < N, the possible number of combinations of success and failure gives**Binomial Distribution -flight test ex.**• Two flight control systems are equally desirable • What is probability that 6 out of 8 pilots would prefer system A over B? • If A and B are truly equally good, probability of pilot picking A over B is 0.5 (P=q =0.5) • Probability of 6 pilots picking A over B is: = 0.109 • There is only a 11% probability that this would happen. If it did, it would mean that your initial assumptions about the two flight control systems was in error**Binomial Flt. Test Example**• If p = q = 0.5, then for N = 8, the binomial distribution would be and from the figure, P(2) is about 11%**Normal Distribution**• The normal distribution is a continuous probability distribution based on the binomial • SINGLE MOST IMPORTANT DISTRIBUTION IN FLIGHT TEST ANALYSIS • Any deviation from a mean value is assumed to be composed of multiples of elemental errors evenly distributed • The mathematical derivation is left as an exercise**Normal Distribution**• Graphically, it can be seen that x = m gives the maximum value and x = m ± s are the two points of inflection on the curve f (x) x m m+s m-s