NIEM 3.0 Data Analytics App

1. NIEM 3.0 Data Analytics App Dr. Brand Niemann Director and Senior Data Scientist Semantic Community http://semanticommunity.info/ AOL Government Blogger http://gov.aol.com/bloggers/brand-niemann/ May 5, 2013 http://semanticommunity.info/NIEM_and_Thetus

2. Background • Previous Stories and Analytics • NIEM Linked in Discussions • DoD Officially Adopts NIEM • NIEM 3.0 Alpha Review Request • Thetus Savannah Knowledge Base & Platform • CSDS: Cloud, SOA, Semantic, & Data Science Conference and Pilots

3. Previous Stories and Analytics http://semanticommunity.info/NIEM#Slide_1_Title_Slide

4. NIEM Linked in Discussions URL

5. DoD Officially Adopts NIEM http://semanticommunity.info/NIEM_and_Thetus#DoD_officially_adopts_NIEM

6. NIEM 3.0 Public Review https://www.niem.gov/technical/coming-soon/Pages/Public-Review.aspx

7. NIEM 3.0 Public Review Steps • STEP 1 Download Alpha 2 • STEP 2 Determine your Review Type and Access Materials • Non-Technical Review MY NOTE: Why is this a non-technical review? • Reviewing the model in Excel allows you to see the elements and types in NIEM 3.0 without having to understand XML schemas. To begin a non-technical review: • Locate the Microsoft Excel (.xlsx) file in the Alpha 2 package. MY NOTE: There are three Excel Files. • Open the Excel file. The file is a spreadsheet with several tabs. The first tab contains the content of NIEM Core. The other tabs each contain an individual NIEM domain. All domains are included. • STEP 3 Conduct your Review • We’re looking for your review and feedback on all things 3.0! If you have never taken part in a model review and/or not sure what to look for, a few suggested items to consider when reviewing NIEM 3.0 Alpha 2 are: • Definitions: Do you see a definition that appears to be incorrect? Do any definitions appear to conflict with each other? Are definitions distinct and distinguishable (or at least different)? • Grammar/spelling/punctuation: Do you see errors in the text of the model? • Technical Architecture: You might notice some things look different in NIEM 3.0, such as augmentations. NIEM 3.0 has over a dozen technical improvements to architecture and structure. Because we are currently updating the NIEM Naming and Design Rules (NDR) for 3.0, we don’t expect that reviewers will be able to comment on all changes during Alpha 2. • Harmonization: Do data element exists only once? Are they semantically precise and clear? Are they reusable and scalable across domains and the NIEM community? • Code Lists: Are code lists up-to-date and relevant? • Other: Are there content or structural items or issues that need to be addressed? • STEP 4 Submit your Feedback on NIEM.gov

8. NIEM 3.0 Alpha 2 Public Review https://www.niem.gov/Pages/contact.aspx

9. niem-3.0-alpha2 File Inventory

10. niem-3.0-alpha2.xlsx: NIEM Core My Note: Why these yellow lines and Web links? http://semanticommunity.info/@api/deki/files/23929/niem-3.0-alpha2.xlsx

11. niem-3.0-alpha2.xlsx: Tabs • Knowledge Base • Consolidation: Number 3-14. • NIEM Core • Biometrics • CBRN • CYFS • Emergency Management • Immigration • Intel • Infrastructure Protection • International Trade • JXDM • Maritime • Screening • Type Description • Elts Sorted by Type • Elts Sorted by Name • Attributes • Namespaces • Class Type

12. niem-3.0-alpha2.xlsx: Consolidation http://semanticommunity.info/@api/deki/files/23929/niem-3.0-alpha2.xlsx

13. niem-3.0-alpha2.xlsx: Spotfire Knowledge Base Navigation and Metadata Distribution of NIEM Types Distribution of Types in Knowledge Base 99 Excel Tabs 112 XSDs Consolidation Data Set Inventory Data Set https://silverspotfire.tibco.com/us/library#/users/bniemann/Public?niem-3.0-alpha2-Spotfire.dxp

14. NIEM and Thetus Knowledge Base Mashup Federated Search Well-defined URLs Semantic Index (see next slide) http://semanticommunity.info/NIEM_and_Thetus

15. NIEM and Thetus Knowledge Base: Semantic Index http://semanticommunity.info/@api/deki/files/23929/niem-3.0-alpha2.xlsx

16. niem-3.0-alpha2-codes.xlsx: Spotfire https://silverspotfire.tibco.com/us/library#/users/bniemann/Public?niem-3.0-alpha2-Spotfire.dxp

17. Thetus Savannah: Knowledge Base & Platform http://semanticommunity.info/NIEM_and_Thetus#Savanna_Solution_White_Paper

18. Thetus Savannah: All Source Analysis Capabilities • Mindmap, an analysis process modeling interface and visualization. • Search interface with connectors to external data sources such as CIDNE, as well as the local application repository. • My Stuff for organizing analysis assets, grouping them thematically and tracking sources. • Link chart tool for creating link charts that can be the inputs to SNA analytics or used as a network visualization. • Geospatial tool for creating high-quality maps and visualizing data with geo coordinates. • Document viewer and annotation/markup for managing and synthesizing unstructured text. • Virtual document to allow analysts to track content and capture insights. • Production tool to quickly assemble high quality output. • Structured data visualization for quick line, bar, scatter, pie, and bubble chart visualizations. http://semanticommunity.info/NIEM_and_Thetus#ALL_SOURCE_ANALYSIS_CAPABILITIES

19. Conclusions and Recommendations • This NIEM 3.0 Alpha Review Pilot Demonstrates: • Cloud Tools – MindTouch and Spotfire • SOA Services – Integrate Multiple Data Sources • Semantic Search – Facets and Context • Data Science – Extract meaning from data, create data products, and tell a story that can be easily understood by non-practitioners. • Source: Wikipedia • Thetus Savannah Is An Example of Mature Platform That Does These Things Well for the Intelligence Community. • Which Is the Subject of Our Conference

20. CSDS: Cloud, SOA, Semantic, & Data Science Conference and Pilots http://www.afei.org/events/3A03/Pages/default.aspx