Microformat-Enabling DoD Information Sources

1. Microformat-Enabling DoD Information Sources Mary Ann Malloy, PhD mmalloy@mitre.org NCW 2009 Conference 17 June 2009

2. Disclaimers • The views, opinions, and conclusions expressed here are those of the presenter and should not be construed as an official position of MITRE or the United States Department of Defense (DoD). • All information presented here is UNCLASSIFIED, technically accurate, contains no critical military technology and is not subject to export controls.

3. Golden nuggets • Microformats are lightweight, non-invasive metadata to easily mark up web content easily. • Microformat injection is an incremental tweak to legacy practices based on proven technologies with the potential to pay back large information-sharing dividends. • Microformats with mashups can help expose legacy information for broader reuse. • Commercial standards origins suit them for DoD – NGO sharing • Microformats and mashups can be used in military contexts to expedite the cost-effective delivery of information & capability to 21st century warfighters & analysts.

4. Info sharing 9/11 Federal Inter- Agency Info Sharing Lessons Learned from Iraq and Afghanistan San Diego Wildfires State, Civil, Local and NGOs DOD and IC Information Sharing Initiatives Hurricane Katrina DOJ / DHS Experience in Fed, State, Local, Tribal Interoperability Asian Tsunami Foreign Allies, Partners GWOT The new interoperability landscape Operationally derived information sharing requirements Implement Lessons Learned DOD and IC Information Sharing Initiatives to achieve Operationally significant results DOJ / DSH Experience in Fed, State, Local, Tribal Interoperability

5. Digital natives cannot even remember a time before the internet and ubiquitous access to media of all kinds…now only 18% of U.S. homes have no internet access, down from 29% in 2006. * Every tiny individual act in a cyber-community creates expanding ripples of data…the average user can generate 250 gigabytes of information every two months! – from The Diverse and Exploding Digital Universe, International Data Corporation, 2008 * see http://www.webpronews.com/topnews/2008/05/14/18-of-us-households-have-no-internet-access

6. Key Air Force Joint/Fed Coalition Strategic Server Army Navy Marines Critical Interface Legacy information sharing approach Supplies Force Readiness Planning Monitoring Intelligence Infra-structure

7. JWIDs JMAPS Message Parser JAMPS Message Preparation Web-enabled Info Services Technology Insertion Timeline • RDF; OWL • Business Processes XML-MTF Ontology 2010 • Object-Oriented SW • C++ PL • Query Language • Shadow Metadata objects • Semantic mediation • Dynamic Brokerage • & Workflow • Agents 2005 • XML-JBI • xDAP • Table-driven SW • C PL • Direct connection to AUTODIN 2000 XML-MTF Schema 1995 • Schema Specification • Constraint language • Metadata capture XML-MTF 1990 • XML Mapping Specification • XSLT presentation Msg Autogen 1985 • JAMPS 1980 OED Exercises

8. DoD’s Net-Centric Vision

9. What are microformats? CLAIM Microformats can act as a “lingua franca” for facilitating machine-to-machine exchange of DoD source information containing widely agreed concepts, both within and outside DoD user communities. A set of simple, open data formats built upon existing and widely adopted standards Markup used by content providers to add semantic value to web content and offerings

10. www.microformats.org“central” • A place for supporting the “microformats process;” e.g., • Research and analysis of real-world examples • Documentation of existing formats (both spec and draft) • Documentation of rejected formats • Code examples • Educational materials (e.g., books, cheatsheets) • “Examples in the wild” • Brainstorming to motivate above; e.g., • mailing lists • Irc • blog • wiki

11. Methodology • Develop and analyze appropriate mappings from DoD-specific metadata elements into microformat classes • Develop microformat injectors using XSLT transformations to implement the mappings. • Convert sample XML documents into microformat-enhanced HTML documents. • Browse the HTML documents in a microformat-aware browser (i.e., Mozilla Firefox). • Observe and analyze automated mashup capabilities enabled by the presence of embedded microformats.

12. Microformats relevant to this effort NOTES: [Items in square braces] are commercial standards from which the corresponding microformats were derived. vCard is a file format standard for electronic business cards. hAtom : represents semantic information relevant to syndicated content such as weblog posts [Atom] hCard : represents semantic information relevant to identifying people, companies, organizations and places [vCard] hCalendar : represents semantic information relevant to events in time [iCalendar] Adr : represents semantic information relevant to addresses [adr property of vCard] Geo : represents semantic information relevant to geographic locations using WGS84 geographic coordinates (latitude, longitude) [geo property in vCard]

13. Artifacts • DoD Discovery Metadata Standard (DDMS) • A common set of descriptive metadata elements to be associated with each data asset to be made visible via the DoD Enterprise Discovery Service • like a library index to DoD information assets in shared space

14. Tools etc Technorati MIT Timeline flickr delicious YouTube Google Maps Yahoo Calendar • Microsoft Visual Studio 2008 : a powerful integrated development environment • used here for XSLT editing and debugging • Mozilla Firefox™ 3.0 : a publicly-available, microformat-aware web browser, whose functionality can be extended by plug-ins; e.g.,: • Operator : Provides microformat parsing and mashup capabilities. Supports new microformat-based functionalities into the toolbar. • Simile Timeline : built via Operator’s javascript interface to display event timelines based on semantic information exposed by hCalendar

15. Develop mappings NOTE This was the most time-consuming part of the work: one staff-week to accomplish the DDMS mappings for this one artifact. • Identify mappings from semantic elements in the DoD source artifact to counterparts in the microformat specification. • Minimum implementation constraint: all mandatory microformat elements must be mapped. • There is incomplete semantic coverage due to fundamental differences between DoD and civilian semantics. • There is a lack of automated support to do the semantic mappings.

16. Develop injectors NOTE Injector encoding was accomplished in about two working days. • XSLT transformations rewrite the structure of an input XML document according to a set of user-defined rules, resulting in a new transformed document. • Additional utility functions are sometimes needed (e.g., translate a coded value into a text phrase) • Our “injectors” are XSLT transformations that automate the insertion of microformats into the subject documents based on a disciplined use of the class attribute that won’t impact the web content. • The HTML wrapper used to make the injected microformats non-disruptivealso make them browsable by Firefox.

17. Capabilities gained “for free” Operator Toolbar NOTE In a military context, all the above are required for targeting and situational awareness. • Viewing key locations on a map display • Searching for related imagery and video products • Searching for related, bookmarked, reference information • Searching for blogs with related content • Searching other tag spaces for related content • Viewing key events in a visual timeline or calendar display.

18. microformats in web documents microformat aware applications + = cool mashups   Mashup employments

19. Challenges • Lack of automated support to do semantic mappings • Incomplete semantic coverage due to fundamental differences between DoD and NGO domains risks loss of meaning and/or misinterpretations • Semantic counterparts to microformats may be “fragmented” throughout legacy source documents • The concept of tag spaces needs further development within DoD to fully leverage the Rel-Tag microformat • This associates a keyword with a contextual domain & further enhances semantics. • Regarding [microformat-aware] mashups: • Due to security reasons, it’s difficult for DoD to leverage commercial tools using real data. • Lack of testing methodologies is worrisome for life-critical applications.

20. Future directions • NOTE • Microformat counterparts to DDMS can be injected directly into other DoD information assets, making them discoverable by any microformat-aware search engine (vice maintaining separate DDMS “metacards” for the DoD Enterprise Discovery Service). • Apply the injection approach to additional DoD exchange formats • Propose DoD-facing classes to the microformat standard to “close the semantic” gap to NGO partners; e.g., • Mission • Vehicles • Tracks • Overlays • Targets • Consider standing up www.microformats.gov • e.g., DoD security information may have no relevant counterpart in the NGO flavor of the standard, but is necessary for DoD missions.

21. A possible way ahead Map microformats to more warfighter content 1 Stand up www.microformats.gov 5 Reuse / develop microformat-aware apps relevant to warfighters 3 Propose DoD-specific microformats 4 Inject microformats * 2 * NOTE: Step 2 must be automated to be feasible on the scale DoD needs

22. Acknowledgements Access our final report at: www.maryannmalloy.com/AUTHOR/sources/Microformats-SemTech2009.docx This work was funded by an FY08 MITRE Program Development / Special Initiative (PD/SI) grant 20PRD010-1J. The presenter gratefully acknowledges Dr. Roger Costello, a MITRE thought leader in the microformats community, who introduced our project to this technology. The presenter also acknowledges her co-investigator, Mr. Charles E. Altizer.

23. QUESTIONS??