Unified Search and Information Integration for the ISS Mission Control Center

1. Unified Search and Information Integration for the ISS Mission Control Center Richard M. Keller, Ph.D. Group Lead for Information Sharing and Integration Intelligent Systems Division NASA Ames Research Center August 16, 2011

2. Outline • Motivation: Search for Mission Control • XSearch System: Functionality & Architecture • Data Integration Subsystem • Link Generation Subsystem • Interface • Wrap-up • Status • User Acceptance • Related Work • Lessons Learned • Summary

3. Primary Customer ISS Flight Controllers • Monitor ISS systems • Troubleshoot • Perform routine maintenance procedures • Initiate ISS system changes via telemetry commands • Document status and actions taken • Coordinate and execute complex operations tasks (e.g. docking, boosting) International Space Station (ISS) Mission Control Center

4. Why is SEARCH fundamental to International Space Station operations? • 24x7 Space Station operations staff generates huge volume of operations workflow records in multiple databases • Flight controllers must access large number of engineering & operations documents and manuals • Flight controllers search with very high frequency while on console; search is essential to accomplishing almost every type of operations activity • Finding information quickly is critical for • effective decision making • mission safety and assurance

5. Search in the Mission Control Center

6. What are the problems? • Information fragmented across many systems • Each search interface is different • Searching is labor intensive, time-consuming, error prone • Interrelated information across systems is not linked and therefore difficult to access

7. Why is interlinked information important? • Obtaining a search result is just the first step in information gathering • The next step is to explore key related information needed to interpret the result • Must understand the context and relationships among data to make effective decisions Linking Open Data cloud diagram, by Richard Cyganiak and Anja Jentzsch. http://lod-cloud.net/

8. Example: Mission Control Center Search Search the Mission Action Request Tracking System(a.k.a. the “Chits” System) for: “ALS heater” Chit Procedure Anomaly Report Part Flight Rule Interlinked information provides context for decision-making

9. Outline • Motivation: Search for Mission Control • XSearch System: Functionality & Architecture • Data Integration Subsystem • Link Generation Subsystem • Interface • Wrap-up • Status • User Acceptance • Related Work • Lessons Learned • Summary

10. XSearch Functionality • Consolidates search • searches multiple databases & displays integrated results • Standardizes & improves search • uniform search interface • improved search functionality • full text search, conjunctive search, acronyms, synonyms • Interlinks & contextualizes search results

11. XSearch Components • Data Integration System: • integrates and warehouses ISS information to be searched on a single server • Information Web: • interlinks the warehoused information • Link Generation Engine: • dynamically scans new and revised information to detect links and insert into web of data • Search Interface: • searches information stored in the warehouse and displays both results and linked data

12. XSearch Architecture Data Integration Subsystem Link Generation Subsystem XSearch Web Interface

13. Outline • Motivation: Search for Mission Control • XSearch System: Functionality & Architecture • Data Integration Subsystem • Link Generation Subsystem • Interface • Wrap-up • Status • User Acceptance • Related Work • Lessons Learned • Summary

14. Data Integration Subsystem web services Data Warehouse Data sources Replicated sources data bases periodic replication extract and normalize XSearch server • Data warehousing approach • “Core” data fields extracted, normalized & stored • Highly configurable approach accommodates changes in data sources web pages documents

15. Normalization Example TARGET: Anomaly Database SOURCE: XSearch Warehouse Mapping Rules • tblAnomaly • ItemID • RequestTitle • WhenModified • StatusText • ControlNumber • OriginatorNetID • RequestOrg • RequestDiscipline • Activity • Core Fields • RecordType • RecordID • Title* • DisplayDate • Content* • ContentType • Status • DisplayName • OriginatorName • Org • Discipline • Activity • tblAnomalyBody • DocumentPart • DocumentSect • ErrDescription • ErrImpact • ErrWorkaround • ErrResolution • ItemID • tblAnomalyAttachment • ATTKEY • UserFileName *Full-text indexed

16. Mapping Tablefor Anomaly Database Target Datatable Target Field Source Database Mapping Rule • source field • constant • SQL statement

17. Mapping Tablefor Anomaly Database WhenModified  DisplayDate a field in Anomaly DB in Warehouse Fill DisplayDate in the warehouse record with the WhenModified field from the Anomaly database

18. Mapping Tablefor Anomaly Database ‘ANOMALY’  RecordType a string constant in Warehouse Fill RecordType in the warehouse record with the string ‘ANOMALY’

19. Mapping Tablefor Anomaly Database multiple fields  Content in Anomaly DB in Warehouse Fill Content in the warehouse record with multiple fields concatenated from the Anomaly database {CAT select '<div class="ANOMALY_Body" id="ANOMALY_' + cast(documentpart as nvarchar(max)) + '_' + cast(documentsect as nvarchar(max)) + '">' + case when len(cast(isnull(ErrDescription,'') as nvarchar(max))) < 1 then '' else '<div class="ANOMALY_Body_Column" id="ANOMALY_' + cast(documentpart as nvarchar(max)) + '_' + cast(documentsect as nvarchar(max)) + '_ErrDescription">' + cast(isnull(ErrDescription,'') as nvarchar(max)) + '</div>' end + case when len(cast(isnull(ErrImpact,'') as nvarchar(max))) < 1 then '' else '<div class="ANOMALY_Body_Column" id="ANOMALY_' + cast(documentpart as nvarchar(max)) + '_' + cast(documentsect as nvarchar(max)) + '_ErrImpact">' + cast(isnull(ErrImpact,'') as nvarchar(max)) + '</div>' end + case when len(cast(isnull(ErrWorkaround,'') as nvarchar(max))) < 1 then '' else '<div class="ANOMALY_Body_Column" id="ANOMALY_' + cast(documentpart as nvarchar(max)) + '_' + cast(documentsect as nvarchar(max)) + '_ErrWorkaround">' + cast(isnull(ErrWorkaround,'') as nvarchar(max)) + '</div>' end + case when len(cast(isnull(ErrResolution,'') as nvarchar(max))) < 1 then '' else '<div class="ANOMALY_Body_Column" id="ANOMALY_' + …

20. Outline • Motivation: Search for Mission Control • XSearch System: Functionality & Architecture • Data Integration Subsystem • Link Generation Subsystem • Interface • Wrap-up • Status • User Acceptance • Related Work • Lessons Learned • Summary

21. Link Generation Subsystem Data Warehouse Information Web links link generation • links generated by analyzing textual information in warehouse records and documents • links and warehouse records are displayed in interface XSearch Interface

22. Link Generation Methods Method 1 (forexplicit cross-references):XSearch analyzes document/record text to identify citations to other documents Method 2 (forimplicit cross-references):XSearch performs cross-document text analysis to determine similarities among documents • Pairs of documents in the corpus are compared for similarity – over 10 billion comparisons!

23. What do flight control documents/records look like? • Combination of structured and free text fields authored by Space Station flight controllers while on duty • Free text fields cross-reference other documents/records using established naming and numbering conventions • NO markup used to identify cross-references in text • Difficult to spot cross-references visually

24. Sanitized Excerpt from an ISS “Flight Note” “The purpose of this note is to outline the hazards to the EVA crew associated with the EVA repair of the SWFPF, for 22A EVA 7. Dynamic events may affect structural integrity of the damaged component – this hazard cause will be controlled by implementation of chit 5452, “Update to Docked Loads Constraints for SWFPF during XPF rotations”, currently open for review. Analysis has shown the Solar Array… …Crew-induced loads on the WIF extender are generally controlled by Flight Rule 22A_C2-D, “Electrical Load Constraints [HC] [RI]”; loads constraints for EVA Articulating Portable Foot Restraint (APFR) and Worksite Interface (WIF) Extender are defined in table 22A_C2-86-F. APFR/WIF loads constraints for EVA 4 have been updated in CHIT 5480. Note that for contingency use of the WIF Extender for EV work on P6, the only approved attitude control is SRM QFMT NO DTP as detailed in EFN F012457A. VRCS loads during LoAC recovery are acceptable as detailed in AR 001767. This hazard control must continue to be observed, to prevent structural failure of the WIF Extender.”

25. ...This chit revises the policy established in Chit 004361 Chit 004374 AR 00447 EFN F013094 EFN F0121902 ...temperature instability was observed shortly after the primary ALS heater failure was detected... similar terms Cross-reference Detection: Mining for links Analyze each search result to determine three types of cross-references hidden in the text CITED-BY x-reference Time Chit 004361 ...This chit implements a Resolution to the primary ALS heater failure described in Anomaly #447...Based on information described in FN F0121902,temperature instability began on GMT 125. CITES x-reference SIMILAR-TO x-reference Search for “ALS heater”

26. Method 1: Explicit Cross-reference Detection via Pattern Matching • Step 1: Cross-reference Candidate Identification Find the cross-references in the text • Step 2: Cross-reference Validation Make sure the cross-referenced records exist in the specified database

27. Step 1: Cross-reference Candidate Identification • For regularized naming schemes: Use specialized BNF Grammars to recognize cross-reference citation patterns • ‘Anomaly Report 001245’, ‘Anomaly 1245’, ‘AR #1245’ • ‘Flight Note F013457B’, ‘EFN F013457B’, ‘F013457B’ • For irregular naming schemes: Use exhaustive match against database of defined record identifiers: • ‘SPS cable assembly’

28. Anomaly Reports XRef Detection Grammar • ARpattern ::= AnomalyTrigger {Digit,1,6} • AnomalyTrigger ::= AnomalyReport | AR • AnomalyReport ::= Anomaly Report{‘s’,0,1} • Anomaly = “Anomaly” | “anomaly” • Report = “Report” | “report” • AR = “AR” | “ar” BNF Grammar AND {Digi,1,6} OR • Example Citations: • ‘Anomaly Report 001245’ • ‘Anomaly 1245’ • ‘AR #1245’ OR AND “ar” “AR” OR ? OR “report” “Anomaly” “Report” “anomaly”

29. Step 2: Cross-reference Validation • Check that cross-referenced record actually existsto avoid creating dead-end links • Query appropriate database to confirm

30. Method 2: Implicit Cross-Reference Detection viaSimilarity Analysis TF-IDF Vector Space Method • Model each record as a point in a multi-dimensional vector space • Each dimension represents a word (found within some record) • Each rec is described by a vector: <w1, w2, w3, w4, …. wn> • Dimensions normalized by frequency of occurrence • Similarity between two points is measured by the cosine of the angle between their two vectors.

31. Outline • Motivation: Search for Mission Control • XSearch System: Functionality & Architecture • Data Integration Subsystem • Link Generation Subsystem • Interface • Wrap-up • Status • User Acceptance • Related Work • Lessons Learned • Summary

32. XSearch Interface Search Query Pane Search Results Pane Details Pane

33. XSearch Interface: Query Pane Search Term(s) “ALS heater” Search Field Selection Time Period Selection Information Sources to be Searched

34. XSearch Interface: Results Pane Individual Search Result Common set of fields displayed across all data sources Record Type Identifying Information (Line 1) Cross-References Summary (Line 2)

35. XSearch Interface: Details Pane Identifying Information Record Preview Cross-References Detail Tabs

36. Outline • Motivation: Search for Mission Control • XSearch System: Functionality & Architecture • Data Integration Subsystem • Link Generation Subsystem • Interface • Wrap-up • Status • User Acceptance • Related Work • Lessons Learned • Summary

37. Status • Deployed in Space Station Mission Control Center since July 2008 • Integrates 15 Mission Control information sources containing >1M records, total • In use daily • User base: 2200 Space Station & Space Shuttle personnel

38. User Acceptance • Informal evaluation performed using search log files and user interviews • Documented cases of value during problem solving • Strongest & most enthusiastic adoption by those w/cross-cutting flight control responsibilities: • Group leads, management personnel, multi-functional consolidated flight control positions • Preference for legacy tool-specific search capabilities over XSearch in the absence of cross-cutting analysis needs • Training environment challenging

39. Related Systems • Search: • Google Search Appliance, Lucene/Solr, Autonomy Search • Enterprise integration: - IBM’s InfoSphere, Informatica’sPowerCenter • Bibliographic Citation Analysis: • CiteSeer, Google Scholar, Scopus • Linked data: • Link Discovery Systems, Named Entity Resolution, DB Record Linkage XSearch uniquely combines capabilities, while addressing specific Space Station Mission Control Center needs: • Fine-grained access control • Negligible indexing latency

40. Lessons Learned • Architect for ease of system reconfigurability • Mitigate impact of ‘data politics’ w/negotiated data service agreements • Ensure adequate systems health monitoring • Understand user needs thoroughly • Guard against “requirements creep”

41. Summary • XSearch integrates Space Station Mission Control data from multiple sources and enables cross-source search • Text mining & information retrieval techniques are used to create linked “document networks” connecting related mission control documents • Interconnected “document networks” provide Flight Controllers with rapid access to related documents, thus decreasing incident response times and improving overall situational awareness

42. Acknowledgments: XSearch Team Ames Research Center Johnson Space Center Design /Development : Intelligent Systems Division Dan Berrios MohanaGurram Rich Keller* Chris Knight Steve Lin May Windrem Shawn Wolfe Infrastructure / OperationsSupport:MCC Web ToolsTeam James Hart Tim Hall* Jason Toschlog Brian Ulman Jenny Yang Ames Research Center Johnson Space Center Customers & Requirements Brian Kelly* ISS/STS Flight Controllers ISS Engineering ISS Program Staff ISS

43. Questions? ? References R.M. Keller and C.D. Knight, "XSearch: A Unified Search and Cross-Reference Detection Engine for the ISS Mission Control Center", IEEE Aerospace Conference, Big Sky, MT, March 2011 R.M. Keller, S.R. Wolfe, M.N.Windrem, and D.C. Berrios, "XSearch: A System for Searching and Interrelating NASA Mission Operations Data", AIAA Space Ops 2008 Conference, Heidelberg, Germany, May 2008. .

44. Backup

45. Authorization and Access Control • Only users belonging to a flight controller group can access portal where XSearch resides • XSearch is a read-only interface • Most sources were read-accessible by all flight controllers • Where required, flight controller groups utilized as a basis for access control (console logs, flight notes) • Exceptions for flight director and “private” notes • Multi-group users have union of permissions

46. Why not buy the Google Search Appliance? • We analyzed pros & cons of GSA • GSA cannot provide contextual cross-references • No commercial search product provides this capability • GSA can handle basic search and cross-source integration • But GSA is not magic and requires configuration (firewalls, security) and ongoing maintenance (index, database reconfiguration) • GSA solution does not meet NASA needs in some key areas: • access control – privacy and restricted information • latency of indexing • interface customizability • indexing methods are ‘black box’