OneSAF Test and Distribution Process

OneSAF Test and Distribution Process Stephen Lopez-Couto Session 7.5

Discussion Points • Introduction • Major and Minor Releases • The Content of a Release • Planning for a Major Release • PM (Internal) Testing • Assessment Testing • Miscellaneous Release Preparation Tasks • Conclusion

Introduction • Prior to any official distribution of OneSAF a lot of preparation, coordination and testing occurs • Major “X.0” releases • Process begins approx. ten months prior to the release • Minor “X.Y” releases • Process begins approx six months prior to the release • This presentation will discuss all of the tasks that lead up to packaged versions of OneSAF being shipped to users

Major and Minor Releases • Major Releases • Generally occur once per year • Based on a full fiscal year’s worth of work • Are subject to the most thorough T&E process • Test is managed by TPO OneSAF • At a user site • Release decision falls on TPO OneSAF • Minor Releases • Generally occur twice per year • PM Determines when OneSAF is going to provide a useful new capability or set of bug fixes • Release test managed by PM • Has TPO and user participation • Release decision falls on PM OneSAF

A Year’s Releases • Version 1.x of OneSAF set and followed this release formula: • 1.0: The major release for the year • Very thorough testing and the most new capabilities • In this case it was the first major OneSAF release, so it is a slightly different case than in future major releases • 1.1: Completed shortly (3-4 months) after the Major release • Includes mostly non critical bug fixes found during Major release testing • 1.5: Completed approximately 2/3 through the year • Contains a sampling of the features that will be fully available in the next Major release

Determination of Version Content • Two primary sources • PM OneSAF Developed • Bi-annual requirements boards meet and determine the items that should be developed during the year • Those items are developed based on availability of primary program (Army) funding • Customer Funded • A customer pays the PM to perform development work • Co-Developers • Build/Modify OneSAF and handover the code to the PM for integration

Determination of Version Content 1.1 1.0 1.5 October November December January February March April May June July August September October November December Build 1 Build 2 Build 3 Build 4 Build 5 Build 6 • The year’s development and integration tasks are distributed into the system build schedule

Release Process Flow Chart System Integration and Test Analyze PTRs, CRs, CoDeveloper Handovers Produce List of RIB items, capabilities Use List to Select Tests from test categories Identify specific tests [List of Tests to execute] Complete Integration, System Testing, Regression Testing PM/TPO Witness Test, creates PTRs Begin Integration ECCB Approval of Final Capabilities And PTRs Start PRB Reviews PTRs, sets severity ECCB reviews [All PTRs completed for this release] [Other PTRs (not Sev 1)] [Sev 1 PTR, to be fixed for this release] Fix PTR Sev 2 PTR Urgent, provide patch prior to next release, Sev 2 PTR for next release are noted, remaining PTRs are prioritized at future RIB meetings. Begin Release Documentation (VDD, Installation Inst.) Documentation Review Create Media for testing Test Media and installation Create Media for distribution End

Major Release Planning • TPO and PM OneSAF select an approximate date for the major release • First major task is to plan for the pre-release software assessment • Logistical planning • Location • Attendees • Hardware, Software, Network • C2 systems • Assessment content • Threads • New and regression • Domain Events • Three face to face meetings conducted during the year in preparation

Major Release Planning • Assessment site selection • TPO queries various user sites to ascertain interest in hosting • Site must satisfy a number of requirements • Equipment • Accessibility • Availability • Space • TPO and PM personnel travel to perform site surveys at interested locations • TPO selects best site for that year’s assessment

Major Release Planning • Initial / Mid / Final Planning Conferences (IPC / MPC / FPC) • IPC (10 months out) • Finalize the location for the assessment • Discuss the general approach that will be taken for the assessment • Examine the previous event and look for changes • Determine success criteria • Discuss in general the content of the assessment • Development is still in the early stages

Major Release Planning • Initial / Mid / Final Planning Conferences • MPC (6 months out) • Discuss the technical details of the assessment • Machine configurations and placement • Network • C2 • Finalize the schedule and content • Begin the development of the test threads and larger scenarios • Agree on each domain’s involvement • Assign specific thread and scenarios • FPC (3 months out) • Held onsite at the assessment facility • Finalize all technical details • Review thread and scenario development • Cover any remaining details

Major Release Planning • One of the most troublesome areas in selecting a site is the availability of tactical C2 systems • Some have little to no equipment • Others have incompatible versions • PEO STRI Digital Integration Lab has assisted in the past • This is not sustainable and will not be the approach for the 2.0 assessment

PM Testing • Occurs informally to prepare for both Major and Minor releases • This is more stringent than standard development testing • Status is tracked and reported to the PM regularly • Formally prior to a minor release • Has TPO and user involvement • PM Uses the results of the testing as an input to his decision as to whether the software should be released

PM Testing • Approach • A&I determines a set of threads and vignettes that will best test the baseline applying known resource constraints • It is already impossible to test all threads for each release • Thread tests will continue to grow as new capabilities are added • Threads and vignettes primarily test model and tool capabilities • Weekly Wars (large scale distributed tests) • Run over two days • OneSAF is supposed to remain executing overnight • This is the primary test of system stability and performance

Assessment Testing • Similar in structure to PM tests, but… • TPO/Users determine the threads that will be run • PM generally does not review these • Users create the vignettes and large scale distributed exercise

Miscellaneous Release Preparation Tasks • Regardless of the state of the software or outcome of the assessments there are other tasks that must be completed before OneSAF can be distributed • Security Accreditation • Disc Layouts • Installer Creation • Documentation creation / cleanup • Packaging preparation • Media Creation • All of these tasks are tracked in regularly occurring distribution meetings

Security Accreditation • Before any Army information technology system is fielded it is supposed to complete a security accreditation and receive an Authority to Operate (ATO) • Certifies that the system is generally secure and all major vulnerabilities are known and protected • Lasts for three years or until the system undergoes a major change • OneSAF completed the DoD Information Technology Security Certification and Accreditation Process (DITSCAP) and received an ATO for version 1.0 • If within the three year timeframe the PM must present all system changes to the Designated Approving Authority (DAA) to determine if a new accreditation is required under the “major change” clause • At this time it is most likely not required for OneSAF 2.0

Disc Layouts • OneSAF contains lots of executable code, data and documentation • 1.5 release encompasses 17CDs or 6 DVDs • Determining the best layout of the content onto optical media helps with • Lowering media requirements • Aiding user understanding of content/installation • Terrain Databases are by far the largest consumers of space • 8/17 CDs, 3/6 DVDs • Future terrains will far surpass current requirements • We may need to come up with unique distribution approaches to handle

Installer Creation • OneSAF 1.0 and 1.1 had very difficult installation processes • The PM received numerous support requests just about installation • The old approach was very generalized and required limited work by the development team for any single release • 1.5 has a new “unified” installer that greatly simplifies the process for the end user • More like commercial software installations • Much more complex for the development team • Each release requires rework of the installer to handle changes in: • OneSAF code and data • Java and other 3rd party software

Documentation Creation and Cleanup • Prior to each release a detailed Version Description Document (VDD) is created • Includes input from all task orders and Government • Installation Manuals updated • Maintenance and User’s Manuals are presently extracted from the onesaf.net website • Requires work to ensure that all data is placed onto discs properly and links function as designed

Packaging and Media Creation • OneSAF is released in commercial grade packaging to ensure that media remains organized and accessible to end users • PM provides the PEO STRI graphics office data and design ideas for disc labels and case graphics • Software Distribution and Test (SWDT) Team • Purchases and prepares media cases and any printed documentation • Coordinates with media duplication vendor to have discs created • Executes final media testing • Test a random sample of discs after handover from duplication vendor

Correcting Distribution Problems • OneSAF 1.1 users experienced numerous problems with the DVD version • Multiple actions to correct this have been implemented • New duplication vendor selected • PM No longer provides “Gold discs” • Everything is kept on hard drives • More thorough final disc verification will occur

Conclusion • Getting a version of OneSAF out the door is not a quick or simple process • The content and quality of the software is of most importance • but without these events occurring the user’s (You) would never get to use it

Questions Stephen Lopez-Couto Stephen.lopezcouto@us.army.mil

OneSAF Test and Distribution Process