1 / 74

Programming Languages and Lifecycle Cost

Programming Languages and Lifecycle Cost. 30 August 1997 Ada Joint Program Office Center for Computer Systems Engineering Joint Interoperability & Engineering Organization Defense Information Systems Agency. PURPOSE.

Sophia
Télécharger la présentation

Programming Languages and Lifecycle Cost

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Programming Languages and Lifecycle Cost 30 August 1997 Ada Joint Program Office Center for Computer Systems Engineering Joint Interoperability & Engineering Organization Defense Information Systems Agency

  2. PURPOSE Provide Quantitative Information on Lifecycle Cost Implications of Programming Language Use (Ada* vs C/C++) * Includes Ada 83 and Ada 95

  3. OUTLINE • Introduction • Cost Factors • Language Studies • Project Examples • Findings

  4. CAVEATS • Language debate is often highly subjective and inconclusive • Different perspectives produce different positions • Many have negative reaction to Ada mandate • Qualitative arguments do not resolve key issues • “Ada is too slow”; “No Ada programmers”; “No Ada tools” • “C code is cryptic”; “C is unsafe”; “C maintenance cost high” • Quantitative cost and productivity data difficult to obtain • Lifecycle cost data requires discipline to collect • Productivity data is often company proprietary • “Controlled experiment” implementing equivalent large, complex applications in multiple languages is exceedingly expensive • Technical language characteristics considered only indirectly • Discussed only with respect to impact on lifecycle cost

  5. Commercial Examples Dr. John McCormick (SUNY) Kurt Welker (USAF) Ulf Olsson (CelsiusTech) Dr. Stephen Zeigler (Rational) Information Sources Currie Colket (SPAWAR 332) Les Dupaix (STSC/Hill AFB) Michael Schrank (MITRE Corp.) AEGIS Community (Various) NASA, IEEE, ACM, etc. SOURCES • Study conducted by: • Michael Masters (NSWCDD) • James Readan (NSWCDD) • J. Hoppel (CSC/KGO) • W. Wright (CSC/KGO)

  6. BACKGROUND • C created in 1970s to support system software development • “Portable assembly language” - Dennis Ritchie (C co-developer) • Ada 83 designed to address issues related to complex systems • C++ added object-oriented features and some type checking to C • Ada 95 revision based on a decade of experience with Ada 83 • Real-time and distribution upgrades • Object-oriented features • “Programming in the large” upgrades • C in widespread use in commercial industry; C++ use increasing • Ada 83 used predominantly in complex/real-time system market

  7. 10000 MISSION CONTROL: GROUND STATION ATF PROJECTED SHUTTLE/OPERATIONAL SKYLAB 2 B-2 F-15E SHUTTLE/OFT APOLLO 17 B-1B APOLLO 7 MANNED SYSTEMS GEMINI 12 1000 SHUTTLE/OFT B-1A C-17 PROJECTED F-16 C/D GEMINI 3 P-3A AWACS MISSILE GALILEO S-3A APOLLO 11 E-2C 100 UNMANNED SYSTEMS P-3A F-15 PERSHING 11(ED) SKYLAB 2 GEMINI 2 APOLLO 7 F-111 TITAN 34D (IUS) F-111 VIKING MERCURY 3 C-5A PERSHING 11(AD) A-7D/E UNMANNED 10 TRIDENT C4 GEMINI 3 UNMANNED INTERPLANETARY VOYAGER TITAN IIIC MANNED A/C POSEIDON C3 PERSHING 1 MANNED SPACE PERSHING 1A MARINER VENUS MERCURY MANNED SPACE CONTROL TITAN SURVEYOR 1 1960 65 70 75 80 85 90 95 SOFTWARE GROWTH Program Size (KSLOC) Year Source: USAF Software Technology Support Center

  8. OUTLINE • Introduction • Cost Factors • Language Studies • Project Examples • Findings

  9. COST FACTORS • Commercial viability • Standardization and portability • Ada 83 and Ada 95 are ISO standards • C is ANSI standard; “It will probably be years before the C++ standard is finally adopted.” H. Schildt, ANSI C++ committee,1995 • Lifecycle Productivity • Training, programmer productivity, reuse, maintainability • Process and tools • High cost of Ada 83 compilers; Ada 95 GNAT is freeware • Added cost of C/C++ “process” to provide features inherent in Ada • Reliability and safety • Ada compiler/runtime validation • Ada technical features: type checking, run-time range checks, restricted pointer use, library consistency, determinism, etc

  10. 3GL (g.p.) M SLOC Ada 83 49.70 C 32.50 FORTRAN 19.55 CMS-2Y/M 18.55 Jovial 13.80 C++ 5.15 Pascal 3.91 PL/1 1.00 Other 3GLs 4.22 TOTAL 148.38 Generation M SLOC First 3.90 Second 26.30 Third (gen.purp.) 148.38 Third (special) 3.70 Fourth 5.00 Fifth 0.29 TOTAL 187.57 DOD WEAPON SYSTEMS Source: Institute for Defense Analyses survey, P-3054, Jan 95

  11. NON-DOD Ada EXAMPLES • Non-DOD Government Use • DOE AdaSage (200K SLOC) • FAA Display System Replacement (DSR) (2.3M SLOC) • NASA Space Station • Commercial Use • Boeing 777 (10M SLOC; 2-4M will be inherited/reused) • “DEC Design” tool • Motorola Cellular Phone • Nippon Communication Products (over 4M SLOC) • Rockwell Beech Starship (375K SLOC) • Rockwell Global Positioning System • GM Heavy Truck transmission computer • Volvo Automated Parts Delivery • Weirton Steel Automated Rolling Mill (500K SLOC) • Genesis Prompt Paymaster (250K SLOC) • Reuters NY Stock Exchange Trader program (250K SLOC)

  12. Air Traffic Control Australia Belgium Brazil Canada China Czech Republic Denmark Finland France Germany Greece Hong Kong Hungary India Ireland Kenya Netherlands New Zealand Pakistan Scotland Singapore South Africa Spain Sweden United Kingdom United States Vietnam COMMERCIAL Ada USE Banking and Financial Networks Reuters news service Swiss Postbank Electronic Funds Transfer system Commercial Aircraft Airbus 330 Airbus 340 Beechjet 400A (US business jet) Beech Starship I (US business turboprop) Beriev BE-200 (Russian forest fire patrol) Boeing 737-200, -400, -500, -600, -700, -800 Boeing 747-400 Boeing 757 Boeing 767 Boeing 777 Canadair Regional Jet Embraer CBA-123 and CBA-145 (Brazilian-made regional airliners) Fokker F-100 (Dutch DC-9-size airliner - American Airlines flies these) Ilyushin 96M (Russian jetliner) Saab 2000 Tupolev TU-204 (Russian jetliner) Communication and Navigational Satellites and Receivers INMARSAT - voice and data communications to ships and mobile communications Intelsat VII NSTAR (Nippon Telephone and Telegraph) PanAmSat (South American Intelsat-like consortium) United States Coast Guard Differential Global Positioning System (GPS) Rockwell Collins NavCore V GPS receiver ESA/Alcatel-SEL GPS receiver TDRSS Ground Terminals - NASA Scientific Satellites Cassini command subsystem ENVISAT-1 - European Space Agency (ESA), Earth observation satellite XMM - ESA EOS - NASA's Earth Observing System GOES RadarSat (Canada) UK Space Technology Research Vehicle, auxillary payload on Ariane4 Railway Transportation Cairo Metro Calcutta Metro Caracas Metro Channel Tunnel Conrail (major U.S. railway company) French High-Speed Rail (TGV) French National Railways Hong Kong Suburban Rail London Underground Paris Metro Paris Suburban Rail Television Industry Canal+ (French pay-per-view TV, remote cable box control software) Medical Industry JEOL Nuclear Magnetic Resonance Source: George Washington University EE & Comp. Science Dept.

  13. “What are your top three development tools?” Percentage of Tool Companies using PowerBuilder 44 Visual BASIC 42 COBOL 40 C++ 20 Access (DBMS) 18 Oracle (DBMS) 14 Borland Delphi 8 Source: Forrester Research Inc. survey responses from 50 Fortune 1000 companies LANGUAGE USE Changing Language Use Percent Change Language from 1993 to 1994 C / C++ -32 FORTRAN 49 Visual BASIC 60 COBOL 75 Smalltalk 200 Source: Dataquest survey of Microsoft Windows application programmers, Communications Week, Aug 1995 Source: IEEE Computer, March 1996

  14. Pascal Ada Modula C C++ Scheme Others Pascal Pascal 36% 40% FOUNDATION LANGUAGES WORLDWIDE USA Scheme Scheme Others 12% Others C++ 11% 12% 6% 10% C++ 8% C 11% C 9% Modula 5% Modula 8% Ada Ada 16% 17% Source: University of Michigan Computer Science Dept. survey

  15. Ada AS A FOUNDATION • University of Alabama • Auburn University • California State Long Beach • Clemson University • Gallaudet University • Georgia State University • University of Maryland • University of Missouri • University of New Orleans • State University of New York • Penn State University • University of Texas Austin • West Virginia University Number of institutions teaching Ada as a Computer Science foundation Year CS1 CS2/7 Total 1991 20 20 40 1992 30 27 57 1993 52 33 85 1994 63 37 100 1995 97 41 138 1996 110 36 146 1997 114 36 150 Source: George Washington University EE & CS Dept. survey

  16. STANDARDIZATION • Ada 83 is an ISO standard • Ada 95 is an ISO standard • C is an ANSI standard • C++ is not yet a standard • “It will probably be years before the C++ standard is finally adopted.” H. Schildt, ANSI C++ committee,1995 • Results of recent ISO canvas for C++ standard: • 10 in favor • 6 against • USA abstained • Britain lodged approximately 85 pages of comments

  17. Ada market appears stable or increasing Ada VIABILITY • Increasing acceptance in USA • 50M SLOC 3GL code base in DOD Weapon Systems • Increasing non-DOD and commercial use • Ada increasingly taught in universities • Ada 95 provides enhanced object oriented features • GNAT Ada95 freeware compiler likely to encourage use • JAVA WWW language has partial C++ syntax and Ada95 semantics • High degree of acceptance in Europe • NATO requires Ada; European vendors rarely request waivers • High degree of acceptance in Far East • Compiler sales greater than US and Europe combined

  18. OUTLINE • Introduction • Cost Factors • Language Studies • Project Examples • Findings

  19. LANGUAGE STUDIES • USAF Study, “Ada and C++: A Business Case Analysis,” 1991 • Sponsored by Lloyd K. Mosemann, SAF/AQK • USAF Software Technology Support Center (STSC) surveys • Les Dupaix of STSC tracked software business case info since 1990 • Monitored numerous government studies (over 100M SLOC) • Survey of 11 European aerospace and defense contractors, 1995 • Institute for Defense Analyses survey • MITRE Software Maintenance study • “Ada in the Maintenance Arena,” Schrank, Boyce & Davis, 1995 • Other sources: NASA, IEEE, ACM, AT&T, Ada Information Clearinghouse, etc. • AEGIS Experience

  20. AIR FORCE STUDY • “Ada and C++: A Business Case Analysis,” 1991 • Five independent studies sponsored by Lloyd Mosemann, SAF/AQK • Institute for Defense Analyses study of tools and training • Tools and courses are available • Software Engineering Institute used IBM / FAA ranking method • Bottom line scores (scale 0-100): C++ = 64, Ada = 79 • CTA Inc. analyzed cost factors from Reifer Consultants database • Productivity (KSLOC/SM): All = 183; C++ = 187; Ada = 210 • Error rate (Errors/KSLOC): All = 33; C++ = 31; Ada = 24 • TRWdetailed analysis of typical info systems & C3I projects • Info system ranking: C++ = 1324; Ada = 1631 (23% better) • C3I system ranking: C++ = 1401; Ada = 1738 (24% better) • Naval Postgraduate School addressed policy issues • Recommended continued investment in Ada

  21. BT Systems, Sweden CelsiusTech, Sweden CRI, Denmark Dassault Electronique, France FMV, Sweden Heise Software, Germany Note: Software products from the above companies represent an Ada code base of over 25M SLOC SATT Control, Sweden Sextant Avionique, France TERMA, Denmark Thompson CSF, France Volvo, Sweden STSC EUROPEAN SURVEY

  22. OUTLINE • Introduction • Cost Factors • Language Studies • Productivity • Program Integration • Reliability & Maintenance • Project Examples • Findings

  23. 600 500 400 300 Productivity (SLOC/MM) 200 100 0 Avionic/ Command Data Environ. Comm. AI Sonar & Control Processing Ada All Lanugages (including Ada) DOMAIN PRODUCTIVITY Source: USAF Software Technology Support Center Based on over 100M SLOC of military and commerical software

  24. 200 180 160 $ / S O L C 140 120 100 80 60 40 20 0 Avionic/Sonar Cmd & Control Data Processing Environment Telecomm. All Lanugages (including Ada) Ada DOMAIN COST Source: USAF Software Technology Support Center Based on over 100M SLOC of military and commerical software

  25. 250 627% Improvement Source: CelsiusTech, Sweden 12 200 S 10 L O 150 8 C SLOC / MM 118% Improvement / 6 100 H 4 o 50 u 2 r 0 0 Historical Ada (First Ship) Ada & Reuse Standard (all lang.) First Ada Project Ada Long Term Source: “Ada vs C++: A Business Case Analysis,” CTA Substudy, July 1991 PRODUCTIVITY

  26. NASA Ada REUSE % Reused Source: NASA/Goddard Space Flight Center, Oct 91

  27. 400 350 300 250 Productivity (SLOC/SM) * 200 150 100 50 0 2 8 32 128 Program Size (KSLOC) DISECONOMY OF SCALE • Barry Boehm:* • “The more members that are added to a team, the more . . . time is consumed in communication with other team members . . .” • “This decrease in [programmer] productivity on larger projects is called a diseconomy of scale in economic terms.” • Methods with different impacts on productivity/cost will have a relatively greater impact as program size increases * Source: Software Engineering Economics, Barry Boehm, Prentice-Hall (COCOMO model)

  28. PRODUCTIVITY IMPROVEMENT Ada productivity compared to other HOLs Source: USAF Software Technology Support Center Based on over 100M SLOC of military and commerical software

  29. OUTLINE • Introduction • Cost Factors • Language Studies • Productivity • Program Integration • Reliability & Maintenance • Project Examples • Findings

  30. PROGRAM ERRORS Source of Errors - %'s Communications of the ACM, Jan. '84 50% 30% 10% Requirements Definition Software Design Coding Testing Deployment $ 10 Relative Cost to Correct Errors - $1000's Source: AT&T Bell Labs Estimates $ 5 Software Design Coding Testing Deployment Requirements Definition

  31. 50 45 40 35 30 Percnet of Total Effort (%) 25 20 15 10 5 0 Design Code & Debug Non-Ada Ada Test Source: NASA/Goddard Space Flight Center, 1982-89 DEVELOPMENT PHASES

  32. DEVELOPMENT PHASES Source: USAF STSC European survey Percent of Total Effort (%)

  33. 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Analysis, Design, Code Historical Integration Ada (First Ship) Source: CelsiusTech, Sweden PROGRAM INTEGRATION • Errors found at integration are costly to fix • “Integration is the most costly phase” • Requires the most people and equipment • “Integration is the most repeated phase” • Required after major upgrades • Tools that reduce integration time also reduce cost

  34. CSC EXPERIENCE Program Lang. SLOC Design Code U/T I&T AEGIS CMS-2Y 31M* 38 12 13 37 ADM Ada 75K 39 19 22 20 Australian Sub Ada 100K 42 16 21 21 JCALS Ada 400K 38 21 21 20 ENWGS Ada 511K 16 40 26 18 AN/BSY-2 Ada 700K 42 16 21 21 IPS Ada 718K 16 23 24 37 JCALS C 7K 38 21 21 20 TGC C 43K 35 29 15 22 E-CALS C 91K 40 20 10 30 Note: design, code, unit test, and integration & test values are percent of total effort * Baselines 1 through J2.2, 1984-1995

  35. OUTLINE • Introduction • Cost Factors • Language Studies • Productivity • Program Integration • Reliability & Maintenance • Project Examples • Findings

  36. 35 Source: USAF study - “Ada vs C++: A Business Case Analysis,” CTA Substudy, July 1991 30 25 20 Error Rates Errors / KSLOC 15 10 5 0 Integration Formal Qual. Test Standard (all lang.) Average Ada RELIABILITY • New York, Chicago telephone system crashs • Crashs caused by C pointer problems • Baltimore Memorial Hospital X-ray machine • 6 deaths, 250 injuries • Defect caused by C pointer problem • Smog sensor • $1M+ fine by EPA • Internat’l Energy Commission • “150 features of C language are unsafe for nuclear use”

  37. AIRBUS BRAKES Source: Thompson, France Data with-held Number of Errors

  38. Maintenance cost advantage of Ada over other languages 40 35 30 25 20 Percent Advantage for Ada in Cost to Make Change 15 10 5 0 -5 -10 72 LOC 87 LOC 350 LOC 550 LOC 870 LOC 10,000 100,000 1,000,000 LOC LOC LOC 10% Change 30% Change Source: “Ada Technology: Current Status and Cost Impact,” IEEE Vol. 79 No. 1 MAINTENANCE

  39. MITRE STUDY • “Ada in the Maintenance Arena,” Schrank, Boyce & Davis, MITRE, 1995 • Used Lockheed Martin’s Programmed Review of Information for Costing and Evaluation (PRICE) Software (PRICE-S) model • Examined military airborne avionics software • Used Capers Jones’ “function point” data base • Compares equal functionality vice equal SLOC • Estimated 10-year maintenance cost for • Ada • C • “Other HOL” (Pascal & FORTRAN) • Assumed experienced programmers and good tools

  40. ANNUAL MAINTENANCE 270,000 LOC 225,000 LOC 135,000 LOC 1994 dollars X $1000 150,000 LOC 112,500 LOC 75,000 LOC Airborne avionics software Function Points Source: MITRE Study, Schrank, Boyce & Davis, 1995

  41. OUTLINE • Introduction • Cost Factors • Language Studies • Project Examples • Findings

  42. PROJECTS • Perceived Failures • Real-time course at the State University of New York (SUNY), Plattsburgh NY • Air Force Improved Many-On-Many (IMOM) reengineering and Idaho National Engineering Laboratory (INEL) studies • CelsiusTech, Swedish defense contractor (corvette class) • Rational Software Corporation experience with Verdix Ada Development Systems (VADS)

  43. PERCEIVED FAILURES SYSTEM Federal Aviation Administration Advanced AutomatIon System (AAS) - program was restructured and descoped in 1994 F16 Avionics - system worked but had cost and schedule overruns SUBACS - Failed to deliver “on schedule” P3C Update 4 - Cancelled while behind schedule ASSESSMENT Lincoln Lab report: Shifting requirements. “[T]he benefits of using Ada for the development and maintenance of large systems are well-known.” Development team unfamiliar with Ada and F16, had 18 mo. schedule, used new parallel processor w/shared memory, changed runtime and methodology, used rate monotonic scheduling. Well into the project the sponsor cut one year off the schedule; product delivered on original schedule. Mission need diminished due to declining Soviet threat. Technology transferred to allies.

  44. FAA AAS • $4B program to replace FAA air traffic control system hardware and software tower, terminal and enroute facilities; 2M SLOC Ada on distributed IBM RS/6000 • In 1994 the program was restructured: terminal part cancelled; tower part descoped; enroute part became Display Replacement System (DSR) • “The Role of Ada in the Restructuring of the Advanced Automation System Program” • Abstract of paper by Jonathan Dehn and Michael Glasgow, Loral Air Traffic Control • “Root problems in the AAS” • “Inadequate requirements baseline control” • “Extremely high availability requirements that led to high complexity in the implementation” • “Changing mission needs” • “[Ada] had little to do with the problems that caused the restructuring of the program.” • Note: beneficial aspects of Ada use not yet addressed in the draft paper • “Shortage of truly knowledgable Ada programmers. . .” • “Ada is frequently maligned for being a poor [execution] performer . . . We reject this notion.” • “Resource intensive in the development environment” • “Ada suffers from a severe image problem . . . viewed as a dead-end language”

  45. PROJECT “X” • Project “X” (also perceived by some as an Ada “failure”) • Company and project name withheld by agreement with source • Large, real-time, distributed system on a COTS UNIX paltform • Command and control system with a high reliability requirement • Average level of programmer experience • Approximately 1M SLOC of Ada • Approximately 110K SLOC lines of C code • Handled interface with a COTS network product • Project development completed • Currently in system test phase

  46. PROJECT “X” RESULTS • Productivity rates for both Ada and C (SLOC per staff month) • 300: Includes time spent writing code, unit and string testing • 200: Including design and CDR time • 160: Including integration time • 150: Including system test time • 11 defects observed per KSLOC, regardless of language • C defects took 4 times longer to fix than Ada defects • C fix rate may have been impacted by low-level nature of services (communication code) implemented in C

  47. SUNY REAL-TIME COURSE • Real-time Course at the State University of New York (SUNY), Plattsburgh NY • “A Model Railroad for Ada and Software Engineering”, Communications of the ACM, November 1992 • POC John W. McCormick, Ph.D. • Teams of 3-4 students design and build real-time model railroad control system • Students have background in Pascal or Modula-2 • 5,000-15,000 lines of code • 30-60 loosely coupled threads of control • 27 independently powered blocks of track • 6 reversing loops, 25 turnouts (switches) • 150 toggle switches would be needed to manually control layout

  48. SUNY RESULTS • Used C as the programming language for the first five years • No team ever completed its project, even when the instructor provided up to 60% of the project code • Five years ago the programming language was changed to Ada • Initially about 50% of the teams completed their projects without instructor support code • With instructor support code, about 75% of the teams now complete their project

  49. Dr. McCORMICK’S COMMENTS • Students produced working systems significantly faster with Ada • Students were able to focus on software engineering issues rather than language issues • Elimination of single keystroke errors (e.g., using = instead of ==) • High level of abstraction for multi-tasking • Higher level of abstraction for bit manipulation and data types (far less pointer manipulation) • Ada program library (team members know when a specification changes) • Exception handlers to separate normal processing from error handling • Modularity and information hiding through packages • Encapsulation through private and limited private types • Code reuse through generic units

More Related