CDC PM SUMMIT Operational Analysis

1. CDC PM SUMMITOperational Analysis

2. Janelle B. Hill • Lead Consultant of Federal Concierge LLC, www.federalconcierge.com • Forum Director, Federal CPIC Forum www.cpicforum.org • 10-years+ experience in Life Cycle Solutions & Professional Services to the Federal Government Sector • 3 Masters in CIO studies, Information Assurance, and Information Management, proceeding towards Doctorate work in CPIC / Information Management • Certified in the Governance of Enterprise IT (CGEIT) • Co-Author of the CPIC Guidebook and the Draft CDC Operational Analysis Guide. Federal Concierge LLC All Rights Reserved

3. What is an Operational Analysis? • Operational Analysis is an evaluation process that formally assesses how well an investment is meeting strategic and business objectives, financial goals, customer needs, system performance goals, and how well it is performing within baseline cost and performance goals. • An operational analysis is, by nature, less structured than performance reporting methods applied to developmental projects and should trigger considerations of how the investment's objectives could be better met, how costs could be reduced, and whether the organization should continue performing a particular function. • A system for this collection of performance data is the key to effective and efficient operational analysis. Federal Concierge LLC All Rights Reserved

4. Operational Analysis • Operational analysis involves • The collection of information concerning a capital asset’s performance • Comparison of this performance with an established baseline • Asset performance measures should include • How well the asset supports customers and stakeholders • How well the asset is managed by the agency • The outputs of this process are recommendations to agency resource managers as to the asset’s continued use, modification, improvement, or termination Federal Concierge LLC All Rights Reserved

5. Where Does Operational Analysis Fit into the EPLC Equation? • Must be performed on all Operations & Maintenance Investments, or on the O&M components of an investment in Mixed Life Cycle. • Commences somewhere between 6 months to 18 months after your Post – Implementation Review (PIR) Federal Concierge LLC All Rights Reserved

7. What Requires Me To Perform Operational Analysis? • OMB Capital Programming Guide requires and prescribes Operational Analysis. • Also Leverage • Post Implementation Review (PIR) • Program Assessment Rating Tool (PART) • Federal Enterprise Architecture (FEA) Performance Reference Model (PRM) • GAO Cost Assessment Guide • GAO Audits or “High Risk” Information Federal Concierge LLC All Rights Reserved

8. Effective Monitoring Using Operational Analysis • Are you managing your O&M investment to cost & schedule with a WBS? • Are you able to predict even a modest partial-point variance or deviation from your cost and schedule? • Do you have an OMB approved baseline? • Are you regularly monitoring your alternatives to ensure that steady state on this investment is a better management idea than performing the business another way? • Is running this investment in O&M the best, most cost-effective or most mission-essential way to leverage the agency-level and department-level inventory of information technology assets? Federal Concierge LLC All Rights Reserved

9. New CDC Operational Analysis Guide • Release scheduled for February 2009 • Has been a team effort by the Compass Systems Consulting team and the CDC CPIC Office to gather, review, analyze and select best practices and a best-of-breed approach. • Seven agency Operational Analysis methodologies and documents were reviewed for best practices including NOAA, USDA, VA, VHA, DHS, Dept. of Commerce, and HHS. • Best-of-breed approach has been blended with CDC processes, the CDC Operational Analysis template and the EPLC framework. • The Guide walks the reader through the process for ensuring and documenting compliance as well as ensuring quality, effective reporting of performance to cost and schedule targets. • Appendices in the Guide provide helpful examples of performance measures and goals as well as the CDC Operational Analysis template/ form. Federal Concierge LLC All Rights Reserved

10. Cost & Schedule Performance • You should • Periodically be measuring the performance of operational assets against the baseline established during the planning or full acquisition phase (i.e., operational analysis), and • Be properly operating and maintaining the asset to maximize its useful life Federal Concierge LLC All Rights Reserved

11. Cost & Schedule Performance • Operational analysis may identify the need to redesign or modify an asset by • Identifying previously undetected faults in design, construction, or installation/integration • Highlighting whether actual operation and maintenance costs vary significantly from budgeted costs, or • Documenting that the asset is failing to meet program requirements Federal Concierge LLC All Rights Reserved

12. Who Performs the Operational Analysis? • The PPM – Project/ Program Manager is responsible for determining whether or not the system or investment meets expectations • For example Section ID of the OMB Circular 300 describes the strategic performance goals and Section IIIB provides operations and maintenance cost estimates for operating and maintenance • These elements form the baseline for accomplishing an OA Federal Concierge LLC All Rights Reserved

13. Tools and Techniques • There are numerous analytical models and tools that are useful for conducting Operational Analysis (all of which should be a part of your EPLC deliverables and/ or your repository for your project). • Examples: • Cost benefit analysis—do benefits outweigh cost? • Trend analysis—investment performance over time • Bench marking—measuring performance against other systems or standards • Lesson Learned—what can be learned from other systems? • Performance against technology strategy—is it inline with CDC’s technology strategy? • Performance against environmental strategy—is it in line with CDC’s environmental management strategy? • Vendor company’s commitment to the product (for off-the-shelf investments)—concern about long term support? • Risk analyses and security assessments—define system vulnerabilities, strengths and weaknesses • Previous OMB 300 reports—provides past technical and cost performance assessments Federal Concierge LLC All Rights Reserved

14. Estimating O&M Costs • Use actual data where it exists • Use planned O&M data derived during system design phase • Use similar system performance data • Consider technology implications (e.g., the use of certain technologies may become more economical in the future • Be aware of the new GAO Cost Guide that prescribes how we assess and estimate our costs, including our risk-adjusted costs. Federal Concierge LLC All Rights Reserved

15. And Your OA Output Includes Recommendations… • Provide your conclusions • Investment is ___ is not ___ meeting its performance goals • ___ Investment should remain in steady state or mixed life cycle for foreseeable future • ___ Investment team should commence planning for DME (Development, Modernization, & Enhancement) in order to ensure continued investment viability • ___ Operational Analysis results suggest that investment should be discussed at IT-Investment Review Board • ___ Operational Analysis results suggest that IT-Investment Review Board does not need to discuss this investment Federal Concierge LLC All Rights Reserved

16. Operational Analysis Results • The results of your Operational Analysis • will be presented to the IT Governing Body • can be presented with Project reviews • Are essential when conducting the system re-certification • Are essential when conducting the Re-accreditation review • Are an output of the EPLC stage gate review indicated between O&M and disposition. Federal Concierge LLC All Rights Reserved

17. Thank You For Attending If your office is interested in utilizing support when coordinating and performing your OA efforts, contact: Within CDC: Sandra McGill & Robin Eggleston Compass / Federal Concierge Presenter Contact Information: Janelle B. Hill, Lead Consultant 760-208-0488 mobile – Eastern Standard Time jbb@federalconcierge.com Federal Concierge LLC All Rights Reserved