Leong Lee, Ph.D., Associate Professor, Computer Science

1. Investigating and Comparing Spatial Accuracy and Precision of GPS-Enabled Devices in Middle Tennessee Leong Lee, Ph.D., Associate Professor, Computer Science Matthew Jones, Ph.D., Associate Professor, Mathematics & Statistics Gregory S. Ridenour, Ph.D., Professor, Geosciences Maurice P. Testa, Student, Geosciences Michael J. Wilson, Director, GIS Center Austin Peay State University, Tennessee, USA

2. Introduction • GPS-enabled mobile devices are extremely popular • How accurate are these devices? • Define accuracy and precision: • Average Euclidean Error (AEE) • Root Mean Square Error (RMSE) • Central Error (CE) • Field data collection for 12 popular GPS-enabled mobiles devices, in two geographical regions in Middle Tennessee • Various mobile devices performed differently: • varied in geographical regions, AEE, RMSE, and CE values & ranking

3. GPS-enabled Mobile Devices • The Global Positioning System (GPS) has been incorporated into every aspect of civilian life • GPS-enabled mobile handsets (cellphones): 960 million units in 2014 [1] • Dynamic technologies • a combination of satellite based, cell tower based, wi-fi based, and/or other local positioning system signals • Limitless research and application potential • e.g. traffic control, mobile networks, real-time data management, location mining, disaster relief / damage assessment and geo-informatics • How exactly are accuracy and precision defined?

4. Accuracy and Precision Accuracy is a quantification of how close measurements are to the “target” or “true” value [2,3] Precision is a quantification of how close replicate measurements are to each other [2, 3] Both accuracy and precision are often measured using average Euclidean error (AEE) and root mean square error (RMSE). RMSEpartitions as a sum of an accuracy measurement (central error, CE) and a precision measurement.

5. Accuracy and precision

6. Average Euclidean Error (AEE) - Accuracy and Precision Calculation Pythagorean distances between n control points Δlat and Δlong are the differences between coordinates of the control points and the field measurement average

7. Root Mean Square Error (RMSE)- Accuracy and Precision Calculation - average x-coordinates of the ith set of field measurements - x-coordinate of the ith control point - average y-coordinates of the ith set of field measurements - y-coordinate of the ith control point Recommended by the Federal Geographic Data Committee

8. Root Mean Square Error (RMSE)- Accuracy and Precision Calculation • (RMSEr)2= sum of accuracy measurement CEand a precision measurement • Accuracy (and incorporated precision) was computed using one of two algorithms, depending on if (RMSEx) was approximately the same as (RMSEy) • When RMSEx = RMSEy, Accuracyr = 1.7308(RMSEr­) • When 0.6 < RMSEx / RMSEy < 5/3, Accuracyr ≈ 1.22385(RMSEx + RMSEy)

9. Central Error (CE)- Accuracy Calculation Delineate between the notions of accuracy and precision when measurements are not centered at the true locations

10. Dixon’s Q Test • Determine if potential improvement could be made in accuracy and precision by rejecting statistical outliers • Q = gap / range • rangeis the difference between extreme (maximum and minimum) values of replicate measurements • gapis the difference between a suspected outlier (maximum or minimum) and its closest neighbor within the dataset • Two Qstatistics: one for latitude, and one for longitude • If a Q statistic exceeded Qtable (a value from a table whose values are a function of sample size and confidence level), the outlying value was eliminated

11. Field Measurements Collection • Four student volunteers collected the spatial data • Five latitude and five longitude measurements were taken at each of fifteen locations, with several devices • For each location and device, the four or five latitudes were averaged and the four or five longitudes were averaged (after Dixon’s Q test) • The fifteen locations were taken across two regions with different landscapes: • Land Between the Lakes, no cell phone reception • Downtown Nashville, tall buildings (might obstruct satellite signals) • Field data were compared with control points • electronic United States Geological Survey base map • AEE, RMSE, CE were computed

12. Online Spatial Accuracy Calculator A website was developed to facilitate future calculation of field data for its AEE, RMSE, and CEvalues

13. Algorithm of the Spatial Accuracy Calculator Algorithm 1: AEE, RMSE, and CE calculation (Spatial Accuracy Calculator) begin user to enter the number of control points, save to n; for i = 1 to n do user to enter the number of spatial coordinates (field measurements) for control point i, save the number to m; for j = 1 to m do user to enter spatial coordinate (latitude x and longitude y) for control point i, measurement j, save coordinator xij, yij to 2D associative array; spatial coordinates can be entered in Decimal Degrees (DD) format or Degrees-Minutes-Seconds (DMS) format; if user chooses to enter in DMS format convert xij, yij to DD format; end-if; end-for; user to enter spatial coordinate for control point i, save to xci, yci; if user chooses to enter in DMS format convert xci, yci to DD format; end-if; end-for; Please refer to paper

14. Algorithm of the Spatial Accuracy Calculator for i = 1 to n do retrieve the number of measurements for control point i from 2D associative array, save to m; retrieve all x and y coordinates of m measurements from 2D associative array; calculate average of all x coordinates of m measurements, save to xdi; calculate average of all y coordinates of m measurements, save to ydi; ∆xi = xci - xdi; ∆yi = yci - ydi; SSi = (∆xi)2 + (∆yi)2; save (SSi)½SSi, ∆xi, ∆yi, (∆xi)2, (∆yi)2 to 1D associative array; end-for; retrieve all values from 1D associative array; AEE = average of all (SS)½ values; CE = ((average of all ∆x values)2 + ((average of all ∆y values)2) ½; RMSEx = (average all (∆xi)2 values)½; RMSEy = (average all (∆yi)2 values)½; RMSEr = (average all SS values)½; if RMSEx = RMSEy Accuracyr = 1.7308 * RMSEr; else Accuracyr = 1.22385 * (RMSEx + RMSEy); end-if; RMSE = Accuracyr; return final output AEE, RMSE, CE; end Please refer to paper

15. Algorithm of the Spatial Accuracy Calculator Time complexity of the algorithm is polynomial with respect to the number of control points, and the maximum number of field measurements for each control point.

16. Results Table 1.Average of Rankings by Root Mean Square Error (RMSE), Central Error (CE), and Average Euclidean Error (AEE) for the Land Between the Lakes (LBL) area.

17. Results Table 2. Average of Rankings by Root Mean Square Error (RMSE), Central Error (CE), and Average Euclidean Error (AEE) for the Downtown Nashville (Nashville) area.

18. Results • Spatial accuracy and precision based on physical distance can be easily deduced • The spatial accuracy and precision (based on AEE, RMSE, and CE) are not the same for different devices • differences in values can be significant • rankings based on AEE, RMSE, and CE for different devicesare not the same • ranking analysis and physical distance analysis = future research

19. Results • AEE, RMSE, and CE comparison between the two geographical areas • significantly different (LBL vs Nashville) • No cell phone reception in LBL area ? = future research • values of AEE, RMSE, and CE in LBL area are uniformly larger than values in the Nashville area, why less accurate? • The ranking of the different devices for these two areas are not the same, e.g. Apple Ipad’s ranking was 1 in LBL area for all three categories (AEE, RMSE and CE), butits ranking was 12 in the Nashville area for all three categories • Average rankings of the devices between the two areas are not the same

20. Conclusion Spatial accuracy and precision for GPS-enabled mobile devices were defined: AEE, RMSE, and CE Field data collected (different devices, 2 regions) LBL area, which has no cell phone reception, and Nashville, in Middle Tennessee The values of AEE, RMSE and CE of these devices in these two areas were ranked and compared Online Spatial Accuracy Calculator / algorithm AEE, RMSE and CEvalues / rankings were not the same for different devices, in the two geographical regions

21. Conclusion Acknowledgments. This work was supported by DHS/Southeast Region Research Initiative (SERRI) under funded project number 4000112222 (Disaster Mitigation & Recovery Kit). Field data of spatial coordinates were collected by Maurice Testa, Patrick Robbins, Eric Whitaker, and James Martin, students in the Department of Geosciences, Austin Peay State University (APSU), Tennessee under the supervision of Michael J. Wilson, Director of the Geographic Information Systems Center at APSU. The figure used in this paper was created by Tabitha S. Y. Lee, student of Clarksville Academy, Tennessee. The online Spatial Accuracy Calculator was programmed by Brian N. Rivers, student in the Department of Computer Science, APSU. • Identify future research topics, such as • analysis of data and ranking based on accuracy and precision in physical distance • analysis and comparison of ranking of devices between different geographical areas

22. References 1. GPS and Mobile Handsets. LBS Research Series (2010) 2. JCGM 200:2008 International vocabulary of metrology — Basic and general concepts and associated terms (VIM) (2008) 3. Taylor, J.R.: The Grid: An Introduction to Error Analysis: The Study of Uncertainties in Physical Measurements. University Science Books (1999) 4. Zandbergen, P.A.: Positional Accuracy of Spatial Data: Non-Normal Distributions and a Critique of the National Standard for Spatial Data Accuracy. Transactions in GIS 12:1, 103--130 (2008) 5. Dean, R.B., Dixon, W.J.: Simplified Statistics for Small Numbers of Observations. Analytical Chemistry 23:4, 636--638 (1951) 6. Spatial Accuracy and Precision: http://www.leeleong.com/spatial/ 7. Geospatial Positioning Accuracy Standards Part 3: National Standard for Spatial Data Accuracy. Subcommittee for Base Cartographic Data, Federal Geographic Data Committee, FGDC-STD-007.3-1998 (1998) 8. Li X.R., Zhao, Z.: Measures of Performance for Evaluation of Estimators and Filters. In: Proc. SPIE 4473, Signal and Data Processing of Small Targets 2001, 530 (2001)

23. ::: Thank You ::: Leong Lee, Ph.D., Associate Professor, Computer Science Matthew Jones, Ph.D., Associate Professor, Mathematics & Statistics Gregory S. Ridenour, Ph.D., Professor, Geosciences Maurice P. Testa, Student, Geosciences Michael J. Wilson, Director, GIS Center Austin Peay State University, Tennessee, USA