Standards Working Group, NASA STD-0006

1. Standards Working Group, NASA STD-0006 Bob Beil NESC Systems Engineering Office June 12, 2007

2. Agenda • Background • Team Listing • Status • Current Activities • Backup This briefing is for status only and does not represent complete engineering data analysis

3. Information Overload! SSP, ISS PRACAdatabases Exploration…. Robotic Missiondatabases JPL, ARC,Stennis etc GSFCSOARSdatabase We are drowning in data, but starving for knowledge! This briefing is for status only and does not represent complete engineering data analysis

4. NESC led Data Mining & Trending Working Group (DMTWG) • NESC tasked to form DMTWG to assist the Agency in the formulation and implementation of a plan to strengthen trending of NASA programs and projects, and to ensure appropriate visibility of data mining and trending within the Agency • Function as an advisory group to the NASA Office of the Chief Engineer and the EMB. • Assist the Office of the Chief Engineer in the development, coordination, and implementation of best practices for data mining and trending of technical data. This briefing is for status only and does not represent complete engineering data analysis

5. NESC Data Mining/Trending Activities To Date • NESC received data mining/trending task • Conducted initial DMTWG workshop • Recommendations and lessons learned (including identification of need for taxonomy/data dictionary) • Initiated Recurring Anomalies effort • Completed SSP recurring anomalies effort • ISS report to be presented to NRB June ‘07 • NESC funded taxonomy working group produced a proposed taxonomy standard • Proposed Standard formed backbone of CxP requirements document (CxP 70068) • Response team formed to perform requested data mining activities within the Agency This briefing is for status only and does not represent complete engineering data analysis

6. Team Listing Team Listing Bob Beil Systems Engineering Office (SEO) - Co-Lead Vickie Parsons NESC SEO – Co-Lead Tina Panontin Ames Research Center Roxana Wales Ames Research Center Michael Rackley Goddard Space Flight Center James Milne Goddard Space Flight Center Tim Barth Kennedy Space Center John McPherson Marshall Space Flight Center Jayne Dutra Jet Propulsion Laboratory Larry Shaw Johnson Space Center This briefing is for status only and does not represent complete engineering data analysis

7. Background cont., why taxonomy • There is a frequently quoted statistic that more than 80% of information housed in corporate repositories is unstructured data, for NASA it is at least equivalent • The reality is that until computers can consistently and accurately recognize concepts, using a taxonomy as a framework for categorizing documents will aid in navigation and retrieval. • Natural language searching and keyword searching yield high retrieval but can miss essential pieces of content that do not contain the specific terms that are being searched. • A taxonomy can be counted on to improve search precision and facilitate discovery when drilling down into a subject hierarchy This briefing is for status only and does not represent complete engineering data analysis

8. Background, Taxonomy • The Office of Chief Engineer requested the NESC lead a small group to develop a proposal for a common taxonomy to be used by all NASA projects in the classifying of nonconformance's, anomalies and problems • The intent was to determine what information is required to be collected and maintained in order to facilitate future trending and root cause analysis • Taxonomy: The science of classification Today, our electronic technology allows us the opportunity to present information from multiple viewpoints maximizing the probability of discovery of relevant information by the end user • A taxonomy provides an important piece of the puzzle to performing efficient data mining and trending This briefing is for status only and does not represent complete engineering data analysis

9. Background cont. • NASA personnel with experience in both human spaceflight and robotic missions were recruited • Expertise in knowledge management systems, anomaly resolution, trending, current problem reporting systems and taxonomy development • Developed use cases across various missions • Using use cases, developed taxonomy(ies) for a Problem Reporting System • Product provides suggestions for inclusion in a problem reporting system and may serve as a partial guide to system developers; the intent was not to design a problem reporting system This briefing is for status only and does not represent complete engineering data analysis

10. Status • In Review This briefing is for status only and does not represent complete engineering data analysis

11. Resolution Matrix Summary 73 comments • Concur • 47 • Reject • 24, have discussed all but 3 with authors • Require further consideration • 2, forward work This briefing is for status only and does not represent complete engineering data analysis

12. Current Activities • CxP PRACA, CxP 70068 (Constellation Program Problem Reporting, Analysis, and Corrective Action (PRACA) Requirements) • Proposed Standard provide backbone for this document • Noted weakness of Standard was software inclusion • Working to incorporate lessons learned into proposed Standard • NESC funded effort planned to benchmark the data dictionary/taxonomy This briefing is for status only and does not represent complete engineering data analysis

13. Back-up This briefing is for status only and does not represent complete engineering data analysis

14. Report Summary • Data elements marked with an asterisk in the share column represent the minimum set of data elements that all projects should make available through a common user interface to organizations such as the NESC, tasked with performing trending across projects • This set of data elements should provide enough information to facilitate the root cause and trend analysis required of the individual projects by NPR 7120.5C • Given differences between human spaceflight and robotic missions life cycles and post launch activities, some data elements may not be applicable for both. This briefing is for status only and does not represent complete engineering data analysis

15. Recommended Data Elements and Taxonomies for Problem Reporting This briefing is for status only and does not represent complete engineering data analysis

16. Pick List Where pick lists or taxonomies are provided for individual data elements, these are meant to be a starting point and not all inclusive This briefing is for status only and does not represent complete engineering data analysis

17. Example List This briefing is for status only and does not represent complete engineering data analysis

18. Taxonomy This briefing is for status only and does not represent complete engineering data analysis

19. Report Summary cont. • Critical need for projects to capture information on aberrant events in order to determine the causes and prevent future occurrences • Relevant data needs to be translated into shared data elements and provided to a common source so trending across projects may be accomplished by independent organizations • Report includes suggested definitions for key terms • Report also includes a white paper on the characteristics of a good taxonomy • Additional recommendations were provided for consideration when developing an actual system This briefing is for status only and does not represent complete engineering data analysis

20. A B C A D C D B A C B B B D D D A D C B B D B A C B B D A D C B B D D A B C A D C D B A C B B B D D D A D C B B D B A C B B D A D C B B D D Background, Why Data Mining? • Data mining to find patterns • Analysis to identify correlations or clusters • Search for explanations “The nontrivial extraction of implicit, previously unknown, and potentially useful information from data.” DD DDDD D D D D D B B B B B B B B B BBB AC AC AC AC AC AC Trending: the search for patterns and subtle relationships in data to infer relative data behavior. This briefing is for status only and does not represent complete engineering data analysis

21. NESC Data Mining Activities • Data Mining Trending Working Group (DMTWG) meeting monthly • Comprised of representatives from all centers as well as NTSB, FAA, DHS, & INPO • Workshop scheduled for June 20/21 to discuss successes in data mining • Purchased and Installing SAS (statistical analysis software) on a LaRC server to make available for DMTWG • ARC developing custom data mining tool (clustering based) to aid search for recurring anomalies • Successful data mining and trending involves a variety of methods and tools as well as subject matter expertise This briefing is for status only and does not represent complete engineering data analysis