1 / 41

ISO/IEC 29119 Part 1, Part 2, Part 3, Part 4 Testauksen prosessit, dokumentointi ja menetelmät

ISO/IEC 29119 Part 1, Part 2, Part 3, Part 4 Testauksen prosessit, dokumentointi ja menetelmät. Heikki Uusitalo FiSMA r.y. Content. ISO/IEC 29119 Testing standard Part 1 Concepts and Definitions Part 2 Testing Process Part 3 Test Documentation Part 4 Test Techniques.

macy
Télécharger la présentation

ISO/IEC 29119 Part 1, Part 2, Part 3, Part 4 Testauksen prosessit, dokumentointi ja menetelmät

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ISO/IEC 29119 Part 1, Part 2, Part 3, Part 4 Testauksen prosessit, dokumentointi ja menetelmät Heikki Uusitalo FiSMA r.y.

  2. Content • ISO/IEC 29119 Testing standard • Part 1 Concepts and Definitions • Part 2 Testing Process • Part 3 Test Documentation • Part 4 Test Techniques

  3. Tietotekniikan standardoinnista • Kattavin standardointitaho tietotekniikan alueella on ISO eli International Organisation for Standardisation. Elektroniikan alueella standardoinnista huolehtii maailmanlaajuinen järjestö IEC eli International Electrotechnical Committee. • Näillä on yhteinen standardointikomitea JTC1. Ohjelmistotuotannon ja järjestelmäkehityksen standardointi on tämän komitean alainen työryhmä SC7. • Suomessa tietotekniikan standardoinnin yleisvastuu kuuluu ISO-jäsenyyttä hoitavalle Suomen Standardisoimisliitolle (SFS). • Politiikkansa mukaisesti SFS on delegoinut SC7-standardoinnin käytännön työn FiSMA-yhteisölle (Finnish Software Measurement Association ry).

  4. SWG 5 SWG 1 Standards Management Group Business Planning Group WG25 WG21 WG7 IT Service Management Software Asset Management Life Cycle Management WG2 WG10 SWG22 WG28 WG27 Vocabulary Maintenance Systems & Software Documentation Process Assessment IT Enabled Services (BPO) CIF Usability WG4 WG19 WG23 WG42 WG40 Tools and Environment Techniques for Specifying IT Systems Systems Quality Management Architecture IT Governance WG6 WG20 WG24 Software Product Measurement and Evaluation Software Engineering Body of Knowledge SLC Profiles and Guidelines for VSE JTC1 SC7 working groups (WG) LCPHAG Life Cycle Process Harmonization Advisory Group SC7 Secrétariat WG26 Software Testing

  5. Standardin synty • ISO/IEC pitää vuosittain Plenary ja Interim –kokoukset, joissa työryhmien ehdotuksia käsitellään. • Standardit valmistellaan työryhmissä ja hyväksytään äänestysmenettelyillä. • SC7 –alakomitea työryhmä WG26 testaus. • Suomen edustajana WG26 ryhmässä on TT Ossi Taipale LUT

  6. ISO/IEC ITTF Doc JTC 1 Doc Committee Doc Working Group Doc Working Group Doc Stages of Development: Standards Stabilized Standard A stabilized standard is one that has ongoing validity and effectiveness but is mature and insofar as can be determined will not require further maintenance of any sort. While a standard is in stabilized status it will no longer be subject to periodic maintenance but will be retained to provide for the continued viability of existing products or servicing of equipment that is expected to have a long working life. Stabilized Standard ~15-20 years Publication Stage 5 International Standard (ISO/IEC) 36 months Approval Stage 4 Final Draft International Standard (FDIS) Draft International Standard (DIS) 33 months Committee Stage 3 Committee Drafts (CD) Final Committee Draft (FCD) 12 months Typical Timeframes Nominal schedule durations are the typical cumulative number of months following Approval of work item. Duration of stages may be shortened or lengthened depending on time To reach consensus to proceed to the next Stage.. Preparatory Stage 2 Working Draft(s) (WD) Proposal Stage 1 New Work Item Proposal (NP) • New Work Item Proposal (NP) is a proposal for: • a new standard • a new part of an existing ISO standard • revision of an existing ISO standard or part • an amendment to an existing ISO standard or part • a Technical Specification or a Publicly Available Specification Preliminary Stage 0 Preliminary Work Item (PWI)

  7. FiSMA r.y. • FiSMA r.y. Finnish Software Measurement Assoiciation • M = Measurement, Management, Models • 5 työryhmää ja foorumia: Standards and Models, Testing Models, IT Service Management, IT Effectiveness, SPIN • FiSMAn jäsenistössä on nelisenkymmentä suomalaista ohjelmistotyötä tekevää ja palveluja ostavaa yritystä, sekä yliopistoja ja muita julkisen hallinnon organisaatioita. Jäseniä kiinnostaa standardien hyödyntäminen ja keskinäinen kokemusten vaihto jäsenyritysten välillä. • Jäseniä mm. Trafi ja Endero.

  8. Testauksen kaksi standardia • Kaksi standardia, jotka täydentävät toisiaan • ISO 29119 Testausprosessi • ISO 33063 Testausprosessin arviointimalli • ISO 29119 –standardi valmistunee tänä vuonna keskeisiltä osiltaan • ISO 33063 –standardista on rakenne ja sisältö päätetty, työryhmä työstää sitä äänestyskuntoon. • ISO 29119 on jo nyt siinä kunnossa, että sitä voidaan ja kannattaa hyödyntää • Standardit täydentävät toisiaan, 29119 määrittelee prosessin ja 33063 perustuu siihen ja antaa perusteet arvioinnille.

  9. Testauksen prosessit ja niiden kyvykkyyden arviointi

  10. ISO 29119 Software Testing • ISO 29119 Software Testing –standardin tarkoitus on määritellä ohjelmistotestaukselle kansainvälinen standardi, jota voidaan käyttää kaikissa organisaatioissa kaikenlaiseen ohjelmistotestaukseen. • Standardi muodostuu neljästä osasta: • 1. Concepts and Definitions Käsitteet ja määrittelyt • 2. Test Process Testausprosessi • 3. Test Documentation Dokumentointi • 4. Test Techniques Testaustekniikat

  11. Part 1 Concepts and Definitions • Systems and Software Engineering • Software Testing • Part 1: Concepts and Definitions

  12. Part 1 Concepts and Definitions • Part 1 on informatiivinen osa, joka sisältää testauksen määrittelyt, käsitteet ja standardin sisällön. • Part1 on myös johdatus standardin muihin osiin. • Siinä kuvataan testauksen rooli organisaation ja projektin kannalta. • Siinä kuvataan miten testaus sovitetaan erilaisiin ohjelmiston kehitysmallehin (agile, evolutionary, sequential ). • Liitteissä kuvataan testauksen metriikkaa ja muita käytännön esimerkkejä. • Määrittelyesimerkkinä test case, miten se ilmenee standardin eri osissa.

  13. Part 1 Multi-layered text context Diagram

  14. Part 1 Concepts and Definitions Esimerkki • Test case • a set of test inputs (and actions, where applicable), execution conditions, and expected results developed to exercise specific test coverage item(s).

  15. Part 2 Test Process • Systems and Software Engineering • Software Testing • Part 2: Test Process

  16. Part 2 Test Process • Part 2 määrittelee yleisen testausprosessin, joka soveltuu kaikkien organisaatioiden käyttöön kaikessa ohjelmistojen testauksessa. • Se sisältää testausprosessin prosessikaaviot ja -kuvaukset. • Testausprosessi on kolmitasoinen: • Organisaatiotason testausprosessi • Testauksen hallintatason prosessi • Dynaaminen testausprosessi • Koska testauksen tarkoitus on riskien vähentäminen, niin standardin lähestymistapa on riskiperusteinen

  17. Part 2 Test Process

  18. Part 2 Test Process

  19. Part 2 Organizational Test process • Organizational test prosess -toiminto käsittää kaksi osaa: • Testauspolitiikka Test Policy • Testausstrategia Test Strategy • Test Policy on lyhyt, jossa kuvataan testauksen tarkoitus, tavoitteet ja laajuus organisaatiossa. Se määrittää testauksen käytännöt, antaa puitteet testauspolitiikan ja strategian luonnille, katselmoinnille ja jatkuvalle kehitykselle. • Test Strategy on yksityiskohtainen, tekninen dokumentti joka määrittelee miten organisaatio suorittaa testausta. • Dokumentit ovat yleisiä, eivät projektikohtaisia

  20. Part 2 Organizational Test Process implementation

  21. Part 2 Test management process • Muodostuu kolmesta aliprosessista • Test Planning Process • Tet Monitoring and Control Process • Test Completion Process

  22. Part 2 Test Management Process

  23. Part 2 Test management process

  24. Part 2 Dynamic Test Processes • Dynaaminen testausprosessi kuvaa kuinka testaus suoritetaan testaustyypeittäin tai testausvaiheittain. • Neljä prosessia • Test design and implementation • Test environment set-up and maintenance • Test execution • Test incident reporting

  25. Part 2 Dynamic Test Processes

  26. Part 2 Test design and implementation process

  27. Part 2 Test Design & Implementation Process • The Test Design & Implementation Process describes how test cases and test procedures are derived; these are normally documented in a test specification, but may be immediately executed, for instance, if performing exploratory testing, in which case they are unlikely to be documented in advance. In figure 10 the activities are shown in a logical sequence, but in practice iteration will occur between many of the activities, often with activities TD3 to TD5 occurring in parallel for substantial periods. • 7.2.1 Purpose • 7.2.2 Outcomes • 7.2.3 Activities and tasks 7.2.3.1 Identify Feature Sets (TD1) 7.2.3.2 Derive Test Conditions (TD2) 7.2.3.3 Derive Test Coverage Items (TD3) 7.2.3.4 Derive Test Cases (TD4) 7.2.3.5 Assemble Test Sets (TD5) 7.2.3.6 Derive Test Procedures (TD6)

  28. Part 2 Esimerkki • 7.2.3.4 Derive Test Cases (TD4) • This activity consists of the following tasks: • a) One or more test cases shall be derived by determining pre-conditions, selecting input values and, where necessary, actions to exercise the selected test coverage items, and by determining the corresponding expected results. When deriving the test cases be aware that one test case may exercise more than one test coverage item and thus there is the opportunity to optimize derived test cases by combining coverage of multiple test coverage items in a single test case. • b) Where appropriate, the test cases should be recorded in the test case specification. If recorded, the traceability between the test basis, feature sets, test conditions, test coverage items and test cases shall be explicitly described. • d) Where appropriate, the contents of the test case specification should be approved by the stakeholders. This may require repeating tasks a) and b), and in some case first repeating the Derive Test Conditions (TD2) and/or Derive Test Coverage Items (TD3) activities.

  29. Part 3 Test Documentation • Systems and Software Engineering • Software Testing • Part 3: Test Documentation

  30. Part 3 Test Documentation • Standardi kattaa kaikenlaisen ohjelmistotestauksen dokumentoinnin mallit. • Siinä määritellään dokumenttien käyttö ja sisältö. • Dokumentaatio noudattaa Part 2 mukaista tasomallia: • Organisaatiotason dokumentaatio • Hallintatason dokumentaatio • Testaustason dokumentaatio

  31. Part 3 Test Documentation • 7.3.4 Test cases • Defines the test cases derived from the test coverage items. A test case specifies how one or more test coverage item(s) are exercised to determine whether or not that part of the test item has been implemented correctly. • This section in the Test Case Specification could be formatted to list test cases under corresponding feature sets and/or test conditions. The test cases may be described in lists or in tables in a document or using a tool, e.g. A relational database or a dedicated test tool. • The information for a test case includes: • 7.3.4.1 Unique identifier • Describes the unique identifier needed by each test case so that it can be distinguished from all other test cases. An automated tool may control the generation of the identifiers. • 7.3.4.2 Objective • Identifies and briefly describes the special focus or objective for the test case. This is typically in the form of a title. • 7.3.4.3 Priority • Defines the priority for the testing of this particular test case, if needed. • 7.3.4.4 Traceability • Describes traceability to the test coverage item that the test case exercises or lists reference(s) to the associated requirement(s) and/or design description(s) in the test basis. This may be documented in a Test Traceability Matrix.

  32. Part 3 Test Documentation • 7.3.4.5 Preconditions • Describes the required state of the test environment and any special constraints pertaining to the execution of the test case, e.g. the state the test item shall be in before execution may start, including existence of specific test data and the currently active form or screen. This could be described explicitly or it could include references to other test cases, whose execution will set the preconditions. • The environment needed may be described collectively for one or more feature sets, or it may not be described in this specification if the description in the Test Plan is sufficient. • 7.3.4.6 Inputs • Specifies each action required to bring the test item into a state where the expected result can be compared to the actual result. This may require provision of input data and events, e.g. button clicks, to the test item. • Some of the input data may be specified by value, while others, such as constant tables or transaction files, may be specified by name. All appropriate databases, files, terminal messages, memory resident areas, and values passed by the operating system must be considered. • All required relationships between input events, e.g. timing, must be described. • The actions or steps may be numbered within the test case, if needed.

  33. Part 3 Test Documentation • 7.3.4.7 Expected results • Specifies the expected outputs and behaviour (e.g. response time) that is required of the test item in response to the inputs. Provides the expected value(s) (with tolerances where appropriate) for each required output. • The actions required to compare the expected result to the actual result may also be specified. • For instance, examining the output in a field on a form that is not active when the input is provided, waiting for a batch job to run and a report to be printed out and examined, or closing down the test item and restarting it to examine stored data. • 7.3.4.8 Test outcome and test result • The description of a test case may include placeholders to record test outcome and/or test results during execution of the test case. Alternatively, this may be recorded in the Test Procedure Specification (see clause 7.10).

  34. Part 4 Test techniques • Testaustekniikoiden kuvaus • Testauksen mittaamisen tekniikoiden kuvaus

  35. Part 4 Test techniques,luokittelu

  36. Part 4 Test techniques, esimerkki • 5.3 Structure-Based Testing Techniques • 5.3.1 Statement Testing • 5.3.1.1 Derive Test Conditions (TD2) • A model of the source code of the test item (program) which identifies statements as either executable or nonexecutable • shall be derived. Each executable statement shall be a test condition. • 5.3.1.2 Derive Test Coverage Items (TD3) • The executable statements shall be identified as test coverage items (i.e. they are the same as the test • conditions).

  37. Part 4 Test techniques, esimerkki • 5.3.1.3 Derive Test Cases (TD4) • The following steps shall be followed: • 1. Identify control flow subpaths that reach one or more test coverage items (executable statements) that • have not yet been executed during testing. • 2. Determine the test inputs that will cause the control flow subpath(s) to be exercised. • 3. Determine the expected outcome from exercising the control flow subpath(s) by applying the • corresponding test inputs to the test basis. • 4. Repeat steps 1 to 3 until the required level of test coverage of the executable statements is achieved.

  38. Part 4 Test techniques, Testing of Quality Characteristics • A.1 Quality Characteristics • A.1.1 Overview • Software testing can be carried out to collect evidence that required quality criteria have been satisfied by a • test item. Required quality characteristics should be specified in the test basis. Definitions of quality • characteristics could be derived from ISO/IEC 25010 System and Software Product Quality Requirements and • Evaluation (SQuaRE) – System and Software Quality Models.

  39. ISO/IEC 25010 Product Quality Model

  40. Part 4 Test Techniques • A.2 Mapping Quality Characteristics to Test Design Techniques • The test design techniques described in this standard can be used to test a variety of the qualitycharacteristics listed above in Figure 3. The following table provides a mapping between them. • Table 1 – Mapping of ISO/IEC 25010 product quality characteristics to test design techniques • Statement Testing • Functional Suitability • Functional completeness • Functional correctness • Functional appropriateness

  41. Further information about SC7 standards • www.sfs.fi, IT standardisation at general level • www.fisma.fi, Software, Systems, Services standards • FiSMA: risto.nevalainen ( at ) fisma.fi

More Related