Linguistic Analysis for Subject Identification

1. L.A.S.I. Linguistic Analysis for Subject Identification Presented by Red Team: Scott Minter, Dustin Patrick, Aluan Haddad, Richard Owens, Brittany Johnson and Erik Rogers

2. September 26th, 2012 Index Introduction NCSOSE 1 Dr. Patrick Hester 2 Problem Identification 4 AID Significance Why it matters 5 Criteria Abstraction Synergy 6 Public Sector Work 7 The Private Sector 8 Beyond Strategic Assessment Applicability 9 Current Process Flow Diagram 10 Mission Statement 11 Project Vision 12 Goals 13 Organizational Hierarchy 14 Problem Specifics Affected Areas 15 Time 16 Problem Specifics (cont.) Documentation Gathering 17 Data Analysis18 Scientific Process 19 The Potential Solution Characteristics 20 How would the process change? 21 Possible Issues 24 The Competition Overview 25 Syntactic v. Semantic 26 Carrot227 Word Stat 28 ReadMe 29 A Unique Solution What the Competition Lacks 30 Conclusion Final Thoughts 31 Works Cited 32

3. September 26th, 2012 Introduction: NCSOSE • National Center for Systems of Systems Engineering • Provides key services in strategic analysis to corporations and government agencies • Discover the Underlying Causes of Issues 1 NCSOSE website

4. September 26th, 2012 Introduction: Dr. Patrick Hester • Ph. D. from Vanderbilt University, 2007 • Major: Risk and Reliability Engineering and Management • B.S. from Webb Institute, 2001 • Major: Naval Architecture and Marine Engineering “My research interests include multi-objective decision making under uncertainty, probabilistic and non-probabilistic uncertainty analysis, critical infrastructure protection, and decision making using modeling and simulation. “ 2 - Dr. Hester 2Patrick Hester Website

5. September 26th, 2012 Introduction: Dr. Patrick Hester cont. • While working for the Department of Homeland Security he was involved in research for resource allocation. • This research led to the development of the Enterprise AID methodology. • Showing the importance of problem identification and assessment

6. September 26th, 2012 Introduction: Problem Identification “Failing to key in on the right problem means even the best solution is destined to fail.” – Dr. Hester 4Parsing Tool for Linguistic Analysis

7. September 26th, 2012 AID Significance: Why it matters The AID Methodology is a flexible, transparent, universal systems assessment approach which: • Analyzes reports and documents critically objectively • De-shrouds issues from an author’s frame of reference • Identifies relevance of explicitly stated problems • Determines the feasibility of proposed solutions

8. September 26th, 2012 AID Significance: Criteria Abstraction Synergy “none directly measureable but every one justifiable as an attribute that merits the title of MOE and hence might promote the assessment, improvement, or design of innumerable enterprises.” 3 -Hester & Meyers • The AID process provides a basis for assessing qualitative well as quantitative goals such as • Team Chemistry • Positive Attitude • Customer Satisfaction1 • Qualitative concerns are highly significant • Qualitative concerns are what people care about • They provide important context • Integrate an agency’s Mission and Vision, directly into the specific problem area 3Enterprise AID

9. September 26th, 2012 AID Significance: Public Sector Work NCSOSE’s public sector work improves Gov. Agency efficiency and helps it maintain focus • Currently engaged in a 3 year, $1.6 million effort to improve efficiency for The Department of Homeland Security4 • Cost and necessity assessments for The Federal Emergency Management Agency3 • Public sector projects matter to all of us 4Parsing Tool for Linguistic Analysis 3Enterprise AID

10. September 26th, 2012 AID Significance: The Private Sector • The AID methodology is generally applicable • Malleable yet not generic • Robust yet adaptable • Beneficial to a wide array of specific issues, focus areas, and problem sets • Corporate planning and management • Academic organization and strategy

11. September 26th, 2012 Beyond Strategic Assessment: Applicability of LASI An intelligent, heuristic algorithm to analyze written work to aid in reaching an unbiased, intelligent conclusion would be invaluable. • Broad Applicability • Strategic Analysis • Education6 • Translation Engines • Web Search Algorithms7 • General Management Decisions8 5 “A Framework for the Computerized Assessment of University Student Essays.” 6 “Using an Intelligent Agent to Enhance Search Engine Performance." 7“A fuzzy multi-criteria decision-making method for facility site selection”

12. Customer Contact Client Goes Elsewhere Will NCSOSE Be Useful? September 26th, 2012 Current Workflow Process: Process Diagram Situational Awareness Meeting Manual Search for: Strategy/Strategic Plans Mission/Vision/Goals Client Input/SA Meeting Problem Statement Independent Research Required Does the customer provide documentation? yes Documentation Gathering Process Strategic Plans Vision/Mission/Goals Org. Hierarchy yes no no

13. September 26th, 2012 Current Workflow Process:Mission Statement • Qualitative overview of the company solving the problem • One or two sentences at most • Must be considered to find the scope of the problem

14. September 26th, 2012 Current Workflow Process: Project Vision • Qualitative outline of the problem to be solved • Several sentences that outline the problem in question • Very important for Situational Awareness Meeting

15. September 26th, 2012 Current Workflow Process: Goals • Quantifiable Statements outlining a specific task • Several are included in each document • The most useful thing to look for in a strategic document

16. September 26th, 2012 Current Workflow Process: Organizational Hierarchy Dept. of Homeland Security FEMA TSA S&T Directorate Individual Disaster Relief Public Disaster Relief Hurricane Relief Other 8 “DHS Component Websites”

17. September 26th, 2012 Problem Specifics: Affected Areas • Time • Documentation Gathering • Data Analysis • Scientific Process

18. September 26th, 2012 Problem Specifics: Time • Only 2 Ph. D.’s, Dr. Hester and his counter-part, currently work on this process. • Process is very time consuming. • Can take up to 12 hours to go through problem statement process.

19. September 26th, 2012 Problem Specifics: Documentation Gathering • Documents will have “jargon” specific to a client’s organization, which must be defined and understood ahead of time. • The client may not provide enough documentation, which requires additional research. • If a lack of documentation requires extensive research, only one of the researchers will perform the entire problem statement process.

20. September 26th, 2012 Problem Specifics: Data Analysis • Large quantities of documents must be mentally digested, assessed, and interrelated. • Must determine aspects achievable given specific monetary and temporal restrictions. • There is difficulty keeping the problem space and solution space separate and without bias.

21. September 26th, 2012 Problem Specifics: Scientific Process • Identify and weight individual problem aspects in terms of stated scope, goals, and resources. • At this time, there is no defined “critique criteria” used to determine the problem statement. • There is only some scientific defense to convince the client the problem statement is accurate, and not have to start over.

22. September 26th, 2012 The Potential Solution: Characteristics • Fully Automated • Run Locally • Web Querying Capabilities • Unbiased • Provides a Weighted Breakdown

23. September 26th, 2012 The Potential Solution: How would the process change? • Decreased time for researching documents If the customer does not provide documentation, the program will be able to search for it. Independent Research Required Does the customer provide documentation? no

24. September 26th, 2012 The Potential Solution: How would the process change? • Decreased time for reading and assessment Reading and analyzing documents currently takes about 6-8 hours and the total time can take up to 12 hours. 9 9 Hester, Patrick Manual Search for: Strategy/Strategic Plans Mission/Vision/Goals Client Input/SA Meeting Problem Statement

25. September 26th, 2012 The Potential Solution:How would the process change? • Provide justification for the analysts’ reasoning Statistical and visual proof is needed when presenting the final problem statement. Present findings to customer Situational Awareness Meeting Problem Statement

26. September 26th, 2012 The Potential Solution:Possible Issues • Incomplete Documents • Computation Time • Memory Usage • Acronyms

27. September 26th, 2012 The Competition: Overview • Document parsing and linguistic analysis are both popular subjects that have much use. Used in: • In the field of Linguistics • In the field of Computer Science • By the Government • As an aid to productive business for companies

28. September 26th, 2012 The Competition: Syntactic v. Semantic Syntactic Semantic • Logical grammar • Statistical Data • Alphabetical Frequencies • Word Counts • Parts of Speech • Word Dependencies • Relating syntactic structures to language-independent meanings • Extracting meaning and conceptional arguments • Summarization

29. September 26th, 2012 The Competition: Carrot2 • Type: Semantic Analysis Tool • Input: • Documents • Search Engine Results • Output: • List of associations of words between different sources • Two different visual representations of the data • Note:This is what is currently used. 10Carrot2

30. September 26th, 2012 The Competition:WordStat • Type: Syntactic Analysis Tool • Input: • Transcripts • Websites • Documents • Books • Taxonomy lists • Output: • User interface with plots and graphs • Database • Website • Document 11WordStat

31. September 26th, 2012 The Competition: ReadMe • Type:Syntactic Analysis Tool • Input: • Populations of Social Science Documents • User-input categories • Output: • Unbiased estimations for topics in the input categories • Opinions of thousands of people about a given topic 12ReadMe:Software for Automated Content Analysis

32. September 26th, 2012 A Unique Solution: What the Competition Lacks • Single theme finding capabilities over a span of multiple documents • Finding themes based on user provided key terms • Web querying capabilities to find organizationally related documents

33. September 26th, 2012 Conclusion: Final Thoughts • The ability to determine a problem is necessary. • The current process is inefficient and resource intensive. • L.A.S.I will “sniff” out the problem searching for a single theme across a plethora of documents. • Competitors can be outshined by additional help from data mining. • Automating the document analysis process will ensure the problem statement will be correct.

34. September 26th, 2012 Works Cited 1 "National Centers for System of Systems Engineering." National Centers for System of Systems Engineering. Web. 23 Sept. 2012. <http://www.ncsose.org/>. 2 “Patrick Hester" Old Dominion University. N.p., n.d. Web. 24 Sept. 2012 <http://www.odu.edu/directory/people/p/pthester>. 3 Hester, P.T., Meyers, T. (2012). Enterprise AID 4 Patrick Hester, Parsing Tool for Linguistic Analysis 5 Duwairi, Rehab M. "A Framework for the Computerized Assessment of University Student Essays." Computers in Human Behavior 22.3 (2006): 381-88. Web. 6 Jansen, James. "Using an Intelligent Agent to Enhance Search Engine Performance." First Monday 2.3 (1997): Web. 7 “A fuzzy multi-criteria decision-making method for facility site selection” GIN-SHUH LIANG, MAO-JIUN J. WANG International Journal of Production Research Vol. 29, Is. 11, 1991 8 “DHS Component Websites.” Homeland Security. N.p., n.d. Web. 26 Sept. 2012. <http://www.dhs.gov/>. 9 Hester,Patrick. Personal Interview. 12 Sept. 2012. 10 Stanislaw Osinski, Dawid Weiss. 13 August, 2012 . Carrot 2. 9/25/2012 <http://project.carrot2.org>. 11”WordStat” Provalis Research. Web. 24 Sept. 2012. <http://provalisresearch.com/products/content-analysis-software/>. 12 “ReadMe: Software for Automated Content Analysis” Web. 24 Sept. 2012. <http://gking.harvard.edu/node/4520/rbuild_documentation/readme.pdf>