Writing a Validator for TMATS

1. Writing a Validator for TMATS Bryan Kelly 83rd FWS/D04 Tyndall AFB, FL

2. Internet Sites • www.bkelly.ws/irig_106 • www.bkelly.ws/irig

3. Purpose • Common validator needed • Proprietary vendors • May not be responsive • Do not advise users of defects • Secure systems have software restrictions • Limited software budgets • Open Source effective solution

4. Audience • Programmers • Intermediate Level • C++ knowledge presumed • Working knowledge of TMATS • Users • Simple Interface, Simple Results • So Far

5. Open Source • Eliminates malware • Capabilities and defects available to all • Easier to introduce into secure areas • Can be customized • End result is usually better code

6. Platform • Windows 7 • Visual Studio • C++ • Core code is in classes independent from GUI • Easy to port, hopefully

7. Programmer’s Advisory #pragma message statements put this information in the Output dialog during build 1>*************************************** 1> 1>This project will not build until you understand how to use Additional Include Directories 1>You may reference a Code Project article found here. 1>http://www.codeproject.com/Tips/588022/Using-Additional-Include-Directories 1> 1>This project is discussed in my Bulletin Board found here: 1>http://www.bkelly.ws/irig_106/ 1>And in my web pages found here: 1>http://www.bkelly.ws/irig 1> 1>***************************************

8. Validate in Three Phases • Read the definitions • Definitions are easily editable • Application loads them into an STD::MAP • The map makes them randomly accessible • Read the Data • Stored into STD::MAP • Validation • Iterate through data • Compare each item with appropriate definition

9. Definitions • The definitions are directly from the standard. • TMATS data is validated with the definitions. • Application Requirements • Easily editable • Easily readable by application • Easy to parse • Storage • CSV (Comma Separated Value) file • Created via Excel

12. Definitions • Group: This is from group G • Separators: \ - : ; • Identifier: DSI and DST • Enumerators: n, ( x, m, d, s, r, p ) • Payload: text and RF\TAP\etc

13. Payload Categories • Exact List • RHCP\LHCP\LIN • Type • text • decimal • Integer • binary • Case is critical

14. Upper & Lower Case • Some Payload are Mixed • P-d\SYNC3 (example) • Payload can contain a positive integer • Payload can be exact: NS (Not specified), or, • Some Complex • NRZ-L\NRZ-M\NRZ-S\BIO-L\BIO-M\BIO-S\RNRZ-L\OTHER • ? How to convey this to the validator • Use Case • Upper case denotes exact text • Lower case denotes type of field • Definition payload: NS\dec

15. STD::MAP • Standard Template Library • Available to all compliant C++ compilers • Easy to use • Stores all the definitions • Provides random access • Requires a key • Selecting Key is critical

16. Constructing the Key • Key = • Group • Identifier • Difficulties • Identifier not consistent, it moves • Duplicate Identifiers • Major problem

17. Duplicates • G\DSI\N:<decimal>; key = GDSI • G\DSI-n:text; key = GDSI • V-x\VN:text; Identifier moved • T-x\AP:text; key = TAP • T-x\AP\POC1:text; key = TAP • T-x\AP\POC2:text; • T-x\AP\POC3:text;

18. Expand Identifier Definition • Identifier consists of each field that is preceded by a backslash, and is before the colon. • All identifier fields are concatenated to build the key.

19. Key Construction • First character is Group • Iterate • Find backslash • Concatenate backslash • Concatenate next token • Repeat to semicolon • TSCON • T\SCO\N

20. Goal of Definition Phase • Read the definitions • Parse into strings • Combine strings into a record • Construct a key • Store the record in map using key

21. Helper Files and Classes C_Get_CString_Tokens.h, .cpp C_Get_Cstring_Tokens_And_Separators.h, .cpp C_Helper_Class.h, .cpp C_Log_Writer.h, .cpp Gnu_Notice.h

22. Worker Classes • C_Tmats_Definitions.h, .cpp reads defintion file, contains def. map • C_Tmats_Data Reads the TMATS data, contains data map • C_TMATS_Basic_Grammar_Checker Checks the grammar. More work to do.

23. Definition Class const unsigned int TOKEN_DEFINITION_GROUP = 0; const unsigned int TOKEN_DEFINITION_02 = TOKEN_DEFINITION_GROUP + 1; … const unsigned int NUMBER_OF_DEFINITION_TOKENS = TOKEN_DEFINITION_24_FORMAT + 1; typedefstruct { CString token[ NUMBER_OF_DEFINITION_TOKENS ]; unsigned intmax_payload_length; } td_tmats_definition;

24. Definition Map std::map < CString, td_tmats_definition > m_tmats_definition_map; std::map < CString, td_tmats_definition > ::iterator m_tmats_definition_map_iterator; CString declares the format of the map key. td_tmats_definition declares the type of data stored in the map. In this case, an array of CString, one per token of the definitions. This map is the heart of the definition class. Everything serves one of two purposes: 1. Acquiring and formatting the data to put records in the map. 2. Fetching selected records from the map for validation.

25. Definition Review • Definition File • Definition parsing • Building a key • Storing the data

26. Data Preview • Similarities to Definitions • Parse into tokens, then into a record • Build key for retrieval • Build key for definition • Save record • Differences • Parser must save separators

27. Data Parsing • No control over the format • Unlike the definitions • Definitions use CSV, commas are separators, discarded • Data separators: \ - : ; • All fields in data are significant, no discards • Parser is different.

28. Get_Cstring_Tokens_And_Separators • Data parser is • C_Get_CString_Tokens_And_Separators • Uses a simple FSM to parse the data • Constructs a definition key • Data key is simple counting integer • Read from data map in same order as in file

29. TMATS_Basic_Grammar_Checker while( mp_C_Tmats_Data->Get_Next_Record( &one_data_record, &data_key ) ) { definition_key= one_data_record.token[ TOKEN_DATA_26_DEFINITION_KEY ]; definition_status= mp_C_Tmats_Definitions->Get_One_Definition( definition_key, &one_definition_record); data_status = Compare_Definition_And_Data( &one_definition_record, &one_data_record); }

30. FSM States enum ENUM_STATE { GROUP, // Always begins here DETERMINE_NEXT_STATE, // Separators, Sets next state COUNTER, // one of the enumerators: -n, -x, etc IDENTIFIER, PAYLOAD, SEMICOLON, EMPTY, // after the colon all tokens must be empty EXIT };

31. Payload Validation • Non-payload fields easy to validate • Payload varies widely • Parse the definition of the payload • Dedicated parser • Examine payload

32. Payload Types • text • int • dec • ANAL\CASS\HDDR\PARA\SSR\MD\N\OTHR • INC\DEC\int differentiated from INC\DEC\INT • NO\dec

33. End of TMATS Marker • There is none defined • How to differentiate between the end of TMATS data and a bad TMATS record • One bad record? Two? Three? Possibly consumes good data from Chapter 10 section. • Suggest a specific identifier for end of data • Create group to mark end. • E:END;

34. Questions? • www.bkelly.ws/irig_106 • Bulletin board • Questions, discussions • www.bkelly.ws/irig • Paper, presentation • Visual Studio solution • Definition file