Software Configuration Management

1. Software Configuration Management • Concepts in SCM, versioning, change control, and future trends • Jalote-2002, SEI-1991, Estublier-2000, Conradi-1998

2. Motivations • Sample Case 1: • Two change requests from customer, assigned to different people, overwriting each other work when saving • Sample Case 2: • Requests to fix a defect or explain implementation of the system in an old version w/o upgrading, need for old code • Sample Case 3: • Faulty software due to old versions of components used in build

3. Background • Software Configuration Management (SCM) is the discipline that enables us to keep evolving software systems under control, and contribute to satisfying quality and time constraints. • SCM emerged in early 70s soon after the “software crisis” (Software Engineering is not equal to Programming !) • Most of the research was done in 80s but not implemented in actual systems till 90s.

4. Short History • 80s • In house systems • Unix scripts over RCS (Revision Control System) and Make • Sun NSE (Network Software Environment), versioning and cooperation • Early 90s • First real SCM systems • File control -based with limited workspace support • ClearCase and Adele • Late 90s • Process support and reliability • Over $1 billion sales

5. Primary Definitions • Configuration is a set of “software objects” required to build a software system. • Baseline is a snap-shot of configuration status with a functional relation between items, e.g. : • Initial • Developmental • Test • Released • Release is a delivered (deliverable) baseline • Version is a configuration item state.

6. SCM Definition • SCM is a discipline to control and manage the evolving configuration of software systems. • IEEE standard 729-1983 for SCM: • Identification: structure and components • Control: changes to and release of products, consistency of components • Status Accounting: recording and reporting the status of components and change requests • Audit and Review: validating the completeness of product and consistency of components

7. SCM Extensions • User roles and operating environments introduce extra SCM functionality. • Manufacture • Managing the construction and building of the product in an effective and optimal way • Process management • Ensuring the carrying out of the organization’s procedures, policies, and lifecycle model • Team work • Controlling the work and interactions between multiple users on a product

8. User Roles and Views • Project manager • Developing within a certain time frame and meeting requirements • Status reporting and reviews • Configuration manager • Defining and following procedures for creating and changing items • Software Change Request (SCR), Configuration Control Board (CCB), task lists • Developer • Working efficiently (sharing, changing, building)

9. SCM Requirements

10. SCM Concepts

11. Repository • Library of configuration items (e.g. files), providing version control. • Objects are immutable while in repository. Need to be “checked-out” to change. • Changes automatically create a new version (and number) • Versions may not be saved completely but as “delta” • Multi-user management may exist to some degree

12. Distributed Components • Repository may be logically centralized but physically distributed in different platforms (e.g. Sherpa Design Management System, DMS) • Fault tolerance and format translation for user transparency.

13. Change Set • Change set is the set of logically related changes to configuration items. • Change set relates low level configuration changes to higher level change requests, in Rational UCM terminology, artifacts to activities. • All related file changes (e.g. for a bug-fix) are put in one set linking modified files, the reason, people involved, and additional information.

14. Workspace • The notion of a workspace is designed to prevent users from interfering with one another’s work. • Workspace is a private area where users can “change” a configuration item, e.g. a local directory. • It is this service that really convinced practitioners that SCM was thee to help. • Needs: • Resynchronization (e.g merging) • Concurrency control

15. System/Product Model • System modeling describes a software product, its structure and components, and how to build it. • Usually and historically only file-based • Build with makefiles. Not adequate enough ! • System modeling includes the notion of family to capture the history of the product, i.e. succession of versions of the components. • Version selection rules determine which versions of components have to be used for each version of the product.

16. Change Request • Software Change Request (SCR) is a documented request for any modification in software (correcting a defect, adding a feature, …) according to a certain procedure • Change tracking part of SCM traces any SCR to all modified configuration items. • Common SCR attributes are status (e.g. assigned and completed), people, dates, and comments.

17. CM Paradigms • No single CM system supports all CM concepts and requirements. • Certain pattern of CM concepts can be seen in existing systems, resulting in four (sometimes combined) paradigms: • Checkin/checkout • Composition • Long transaction • Change set

18. Checkin/Checkout • Repository support for versioning of components • Branching and merging • Concurrency control through locking • Users “check out” before accessing an existing version and “check in” to create a new one. • Local file system is the working area for checked out items • Mutually exclusive modification within a version branch • Checkin/checkout paradigm results in a version history in form of a graph (version graph)

19. Checkin/Checkout

20. Composition • Creating configurations and managing their history/family. • Configuration consists of • a system model (aggregation of components) • version selection rules (appropriate version of each component) 1.0 1.1 1.2 A B C 1.0 1.1 1.0 1.1 1.2

21. Long Transaction • Evolution of the whole software system as a series of atomic changes. • Start with a versioned configuration • End up with creation of a new version of configuration • Development path is the sequence of transactions. • Multiple transactions coordinated through concurrency control • Different from traditional DB transaction • Creating a new version rather than updating existing one • Persistence, i.e. long duration more than a login session • Paths can create branches.

22. Change Set • Management of logical changes to configurations • Change-oriented vs. version-oriented SCM • Logical change is a set of physical changes • Link between change requests and actual modifications in configuration items • No concurrency control, so combined with checkin/checkout A B C D + + Fix 1 Feature 4

23. Common SCM Tools • SCCS, RCS (Unix) • ClearCase, Summit, CMVC (Rational) • Visual SourceSafe (Microsoft) • CVS, Bugzilla (open source and free software) • Razor (Tower) • Bugbase (Archimedes Software) • Many other tools with CM capabilities (e.g. Requisite Pro and Rational Rose)

24. Characterization of CM Systems

25. Layered CM System • Levels of CM support CM Policy QA, CM Process, etc e.g. SCR and quality audits procedures CM Protocol Transactions, workspaces, etc e.g. checkin/checkout and change sets CM Primitives Tool primitives, OS operations, etc e.g. versioning and locking

26. Versioning • Product space • Software objects • Relationships • Representations • Version space • Versions • Deltas • Version graphs • Interplay with product space • Merge tools

27. Product Space • Software objects are the results of a development or maintenance activity. • Coarse or fine-grained • Composite objects (configurations) • Relationships connect software objects • Composition • Composite vs. atomic • Product is the root of composition hierarchy • Dependency • Master and dependent objects

28. Representations of Product Space • Product “foo” • Modules with dependencies • File system structure • Tree structure with typed objects and relationships • Dependency graph

29. Representations of Product Space

30. Version Space • Version v represents a state of an evolving item I • v = (ps, vs) • Revision (modified) or variant (alternative) • Difference between two versions is called a “delta” • Symmetric delta between two versions (state-based versioning) • Directed delta is a sequence of elementary change operations to create a version from another (change-based versioning)

31. One-Level Version Graphs • One level organization • Each version connected by one single type of relationship called “successor” • Supported in NSE and PCTE • Different shapes based on multiple sibling and successor

32. Multi-Level Version Graphs • Two-level organizations • Graph is composed of branches each with a sequence of revisions • At least two relationships • Successor (within a branch) • Offspring (between branches) • Supported in RCS • ClearCase introduces merging • Multidimensional variation • So many variants may explode the graph • Clusters of related versions can be used • Grids (n-dimensional coordinate system) are another solution. • Revisions can be represented in grids by adding a time dimension

33. Two-level Version Graph

34. Multidimensional Variants

35. Interplay of Version/Product Spaces • Multiple configuration items with multiple versions • AND/OR graphs provide a general model for integrating product and version space. • AND/OR edges come from AND/OR nodes, respectively. • Product can be represented by only AND nodes/edges. • Versioning is introduced to product space by adding OR nodes/edges. • Selection order • Product first • Version first • Intertwined

36. AND/OR Graphs

37. AND/OR Graphs • In intertwined graph, it is easier to add a new version for one of the items without having to create virtual versions for other items as in “version first” graph.

38. Merge Tools • Raw merging applies a change in a new context • Supported by SCCS • Two-way merging compares two alternative versions and combine them into a new one. • Displays differences to the user and allow the choice of appropriate one • No automatic resolution of differences • Three-way merge tool consults a common baseline when difference is detected • Conflicts can be resolved manually or automatically • Aide-de-Camp provides this method.

39. Semantic Level of Merging • Textual merging • Only for text files • Supported by almost all SCM systems • Only physical conflicts are detected (no valid C file !) • Syntactic merging • Needs knowledge of file structure • Can guarantee a syntactically correct output • Research prototypes • Semantic merging • Can find semantic conflicts and fix if possible • Not fully implemented yet

40. SCM Process • Planning and Setting up SCM • Configuration items and procedures • Performing SCM • Managing the state transition of configuration items • Controlled and uncontrolled environments • Checkin/checkout • Release • Change control • Monitoring and Audits

41. SCM Planning Steps • Identify configuration items • Choose the tools and environments • Define naming and numbering scheme, and director structure • Define change control procedure (and people) • Set up configuration control board (CCB) • Define status tracking methods • Define procedures for backup, release, archival, and audits

42. Software Change Control • Tools for creating and tracking software change request (SCR), examples: • ClearQuest • Bugzilla • SCR General Procedure • Accept • Assign • Check out • Perform • Check in

43. Change Control Integration • Change control needs to be integrated with repository and other SCM tools to trace all changes to the SCRs. • Rational Unified Change Management combines ClearQuest (activity-based) with the version control tool, ClearCase (artefact-based). • More intelligent tools can/should show and track the changes in higher levels of abstraction than files, e.g. functionality modules.

44. Status Monitoring and Audits • Projects must perform regular status checking (e.g. “under development”, “ready for release”) for all configuration items. • SCRs must also be checked regularly. • Projects can also perform CM audits o make sure procedures are being followed. • Audit report should generate a task list assigned to specific people.

45. State of Practice • Most useful features • Change control • Activity control • Workspace support • Global view • traceability • Most missing features • Process support • Concurrent and distributed engineering support • Scalability • Cross platform operation

46. Current Research • Versioning • Why are two objects versions of each other? • Data/product model • Use of more advanced data models rather than file systems • Relation with version model (on top, below, part of) • Composition • Configuration description languages • Building • Unified makefiles • Language dependent systems (smart build)

47. Current Research • Workspace • Complex objects available at a given location in a given format (e.g. editor file or DBMS schema) • Cooperative and remote work • Define, at high level, the cooperation strategies, organization and procedures • Merging objects (more than text files) • Web support • Process support • Integration of activity-based and item-based approaches • High level procedures and models