Software Testing

1. Department ofComputer Science & Engineering SOFTWARE TESTING

2. Software testing can be stated as the process of verifying and validating that a software or application is bug free, meets the technical requirements as guided by it’s design and development and meets the user requirements effectively and efficiently with handling all the exceptional and boundary cases. • The process of software testing aims not only at finding faults in the existing software but also at finding measures to improve the software in terms of efficiency, accuracy and usability. It mainly aims at measuring specification, functionality and performance of a software program or application. • Testing is the process of evaluating a system or its component(s) with the intent to find whether it satisfies the specified requirements or not. In simple words, testing is executing a system in order to identify any gaps, errors, or missing requirements in contrary to the actual requirements. • According to ANSI/IEEE 1059 standard, Testing can be defined as - A process of analyzing a software item to detect the differences between existing and required conditions (that is defects/errors/bugs) and to evaluate the features of the software item. DEFINITION OF TESTING • Software Testing is a process of evaluating the functionality of a software application to find any software bugs. It checks whether the developed software met the specified requirements  and identifies any defect in the software in order to produce a quality product. It is basically executing a system in order to identify any gaps, errors, or missing requirements in contrary to the actual requirements.

3. Who does Testing? • It depends on the process and the associated stakeholders of the project(s). In the IT industry, large companies have a team with responsibilities to evaluate the developed software in context of the given requirements. Moreover, developers also conduct testing which is called Unit Testing. In most cases, the following professionals are involved in testing a system within their respective capacities − • Software Tester • Software Developer • Project Lead/Manager • End User • Different companies have different designations for people who test the software on the basis of their experience and knowledge such as Software Tester, Software Quality Assurance Engineer, QA Analyst, etc. • It is not possible to test the software at any time during its cycle. The next two sections state when testing should be started and when to end it during the SDLC. • When to Start Testing? • An early start to testing reduces the cost and time to rework and produce error-free software that is delivered to the client. However in Software Development Life Cycle (SDLC), testing can be started from the Requirements Gathering phase and continued till the deployment of the software. • It also depends on the development model that is being used. For example, in the Waterfall model, formal testing is conducted in the testing phase; but in the incremental model, testing is performed at the end of every increment/iteration and the whole application is tested at the end. • Testing is done in different forms at every phase of SDLC − • During the requirement gathering phase, the analysis and verification of requirements are also considered as testing. • Reviewing the design in the design phase with the intent to improve the design is also considered as testing. • Testing performed by a developer on completion of the code is also categorized as testing. • When to Stop Testing? • It is difficult to determine when to stop testing, as testing is a never-ending process and no one can claim that a software is 100% tested. The following aspects are to be considered for stopping the testing process − • Testing Deadlines • Completion of test case execution • Completion of functional and code coverage to a certain point • Bug rate falls below a certain level and no high-priority bugs are identified • Management decision

4. Let’s break down the basic definition of Software testing into the following parts: • 1)  Process:  Testing is a process rather than a single activity. • 2)  All Life Cycle Activities: Testing is a process that’s take place throughout the Software Development Life Cycle (SDLC). • The process of designing tests early in the life cycle can help to prevent defects from being introduced in the code. Sometimes it’s referred as “verifying the test basis via the test design”. • The test basis includes documents such as the requirements and design specifications. • 3)  Static Testing:  It can test and find defects without executing code. Static Testing is done during verification process. This testing includes reviewing of the documents (including source code) and static analysis. This is useful and cost effective way of testing.  For example: reviewing, walkthrough, inspection, etc. • 4)  Dynamic Testing:  In dynamic testing the software code is executed to demonstrate the result of running tests. It’s done during validation process. For example: unit testing, integration testing, system testing, etc. •  5)  Planning:  We need to plan as what we want to do. We control the test activities, we report on testing progress and the status of the software under test. • 6)  Preparation:  We need to choose what testing we will do, by selecting test conditions and designing test cases. • 7)  Evaluation:  During evaluation we must check the results and evaluate the software under test and the completion criteria, which helps us to decide whether we have finished testing and whether the software product has passed the tests. • 8)  Software products and related work products:  Along with the testing of code the testing of requirement and design specifications and also the related documents like operation, user and training material is equally important.

5. Why do we need Software Testing? The software application’s success rate controls the growth of our business. It plays an important role for the development of software applications and products. We need software testing for the following reasons: Cost Effective – Testing has many benefits and one of the most important ones is cost-effectiveness. Testing our project on time can save money in the long run. Software development consists of many stages and if bugs are caught in the earlier stages it costs much less to fix them. 2. Security – This is the most sensitive and vulnerable part of software testing. Users are always looking for trusted products that they can rely on. It helps in removing problems and risks beforehand. 3. Product Quality – In order to make your product vision come to life, it has to work as planned. It is important to follow the product requirements because it helps you get the required end results. 4. Customer Satisfaction – The ultimate goal for a product owner is to give the best customer satisfaction.  Software should be tested in order to bring the best user experience possible. Being the best product in this saturated market will help you gain trustworthy clients which will have great long-term effects.

6. Difference between Testing and Debugging What is Testing? Testing: Fundamentally, testing is a process to check if the system is working same as it was supposed to do, and not working as it was not supposed to do. It is done by the tester to identify the defects in the system (actual result of test case execution is not matching with expected result). It can be done using Manual and automated way. The issues are logged against the all failed cases and communicate to the dev team for debugging and fixing. After the bug fixes then tester retest the bug and checks if the issue is no more exists. You may interest in – Defect Life Cycle. Testinginvolves identifying software bug/error/defect without its correction. Usually professionals with a background of quality assurance are involved in the process of bugs identification. This process is performed in the testing phase.

7. What is Debugging? Debugging is the activity performed by developers to fix the bug found in the system. This is manual step by step unstructured and unreliable process to find and removes a specific bug from the system. It is done by the developers to fix the defect in the system. Based on the defects reported, the developer starts analyzing the root cause of the defect to make system defect free. The developers need to fix the code to make actual result same as expected result. With the help of details added in the defect developer follow steps mentioned and check the loopholes in the particular code area. They may use a different tool for debugging. On fixing the defect developer send back the defect to a tester for retesting. Debugging involves identifying, isolating, and fixing the bugs/problems. Developers who code the software carry out debugging in order to encounter an error in the code. This process is a part of Unit Testing or White Box Testing. Debugging can be performed either while conducting Unit Testing, i.e. development phase or while fixing the reported bugs. Debugging is deeper process than the  Testing. Debugging involves identifying, isolating and fixing the bugs.

9. Following are the important differences between Testing and Debugging.

10. What are Software Testing Objectives?

11. To evaluate the work products such as requirements, design, user stories, and code: • The work products such as Requirement document, Design, and User Stories should be verified before the developer picks it up for development. Identifying any ambiguity or contradicting requirements at this stage saves considerable development and test time. The static analysis of the code (reviews, walk-thru, inspection, etc.) happens before the code integrates/is ready for testing.  This idea of testing is known as Verification. It is the process of evaluating the product during the development phase of each work product. • To verify the fulfillment of all specified requirements: • This objective reveals the fact that the essential elements of testing should be to fulfill the customer’s needs. Testers test the product and ensure the implementation of all the specified requirements have. Developing all the test cases, regardless of the testing technique ensures verification of the functionality for every executed test case. The Tester should also create a requirement traceability matrix (RTM), which will ensure the mapping of all the test cases to requirements. RTM is an effective way to ensure that test cases have got the right requirement coverage. • To validate if the test object is complete and works as per the expectation of the users and the stakeholders: • Testing ensures the implementation of requirements along with the assurance that they work as per the expectation of users. This idea of testing is called Validation. It is the process of checking the product after development. Validation can be a manual or automation. It usually employs various types of testing techniques, i.e., Black Box, White Box, etc. Generally, testers perform validation, whereas customers can also validate the product as part of User acceptance testing. Every business considers the customer as the king. Thus the customer’s satisfaction is a predominant need for any business.  • For example, customer satisfaction and loyalty in online shopping and e-commerce environments is a useful indicator for long-term business success.

12. To build confidence in the quality level of the test object: • One of the critical objectives of software testing is to improve software quality. High-Quality software means a lesser number of defects. In other words, the more efficient the testing process is, the fewer errors you will get in the end product. Which, in turn, will increase the overall quality of the test object. Excellent quality contributes to a significant increase in customer satisfaction as well as lower maintenance costs.  • To prevent defectsin the software product: • One of the objectives of software testing is to avoid the mistakes in the early stage of the development. Early detection of errors significantly reduces the cost and effort. The prevention of defects involves doing a root cause analysis of the defects found previously and after that, taking specific measures to prevent the occurrence of those types of errors in the future. Efficient testing helps in providing an error-free application. If you prevent defects, it will result in reducing the overall defect count in the product, which further ensures a high-quality product to the customer. • To find defects in the software product: • Another essential objective of software testing is to identify all defects in a product. The main motto of testing is to find maximum defects in a software product while validating whether the program is working as per the user requirements or not. Defects should be identified as early in the test cycle as possible. • E.g., a defect found in the UAT phase will be much costlier to fix than the same defect found in the Sprint testing phase. 

13. To provide sufficient information to stakeholders to allow them to make informed decisions, especially regarding the level of quality of the test object: • The purpose of testing is to provide complete information to the stakeholders about technical or other restrictions, risk factors, ambiguous requirements, etc. It can be in the form of test coverage, testing reports covering details like what is missing, what went wrong. The aim is to be transparent and make stakeholders fully understand the issues affecting quality. • To reduce the level of risk of insufficient software quality: • The possibility of loss is also known as risk. The objective of software testing is to reduce the occurrence of the risk. Each software project is unique and contains a significant number of uncertainties from different perspectives, such as market launch time, budget, the technology chosen, implementation, or product maintenance. If we do not control these uncertainties, it will impose potential risks not only during the development phases but also during the whole life cycle of the product. So, the primary objective of software testing is to integrate the Risk management process to identify any risk as soon as possible in the development process. • To comply with contractual, legal, or regulatory requirements or standards, and to verify the test object’s compliance with such requirements or standards: • This objective ensures that software developed for a specific region must follow the legal rules and regulations of that region. Moreover, the software product must be compatible with the national and international standards of testing. We have ISO/IEC/IEEE 29119standards that deal with the software testing concept. • E.g., each country has laws specific to accessibility requirements which must be fulfilled to avoid legal implications. The European Union has strict rules on how the Personal Identifiable Information (PII) like Social security number etc. should be handled. Failure to adhere to such requirements will lead to failure of the product, no matter how defect-free it has been working!

14. Software Testing - Types of Testing 1) Manual Testing Manual testing includes testing a software manually, i.e., without using any automated tool or any script. In this type, the tester takes over the role of an end-user and tests the software to identify any unexpected behavior or bug. There are different stages for manual testing such as unit testing, integration testing, system testing, and user acceptance testing. Testers use test plans, test cases, or test scenarios to test a software to ensure the completeness of testing. Manual testing also includes exploratory testing, as testers explore the software to identify errors in it. 2) Automation Testing Automation testing, which is also known as Test Automation, is when the tester writes scripts and uses another software to test the product. This process involves automation of a manual process. Automation Testing is used to re-run the test scenarios that were performed manually, quickly, and repeatedly. • What to Automate? • It is not possible to automate everything in a software. The areas at which a user can make transactions such as the login form or registration forms, any area where large number of users can access the software simultaneously should be automated. • Furthermore, all GUI items, connections with databases, field validations, etc. can be efficiently tested by automating the manual process. • When to Automate? • Test Automation should be used by considering the following aspects of a software − • Large and critical projects • Projects that require testing the same areas frequently • Requirements not changing frequently • Accessing the application for load and performance with many virtual users • Stable software with respect to manual testing • Availability of time

15. How to Automate? • Automation is done by using a supportive computer language like VB scripting and an automated software application. There are many tools available that can be used to write automation scripts. Before mentioning the tools, let us identify the process that can be used to automate the testing process − • Identifying areas within a software for automation • Selection of appropriate tool for test automation • Writing test scripts • Development of test suits • Execution of scripts • Create result reports • Identify any potential bug or performance issues • Software Testing Tools • The following tools can be used for automation testing − • HP Quick Test Professional • Selenium • IBM Rational Functional Tester • SilkTest • TestComplete • Testing Anywhere • WinRunner • LoadRunner • Visual Studio Test Professional • WATIR

16. Software Testing - Myths Myth 1: Testing is Too Expensive Reality − There is a saying, pay less for testing during software development or pay more for maintenance or correction later. Early testing saves both time and cost in many aspects, however reducing the cost without testing may result in improper design of a software application rendering the product useless. Myth 2: Testing is Time-Consuming Reality − During the SDLC phases, testing is never a time-consuming process. However diagnosing and fixing the errors identified during proper testing is a time-consuming but productive activity. Myth 3: Only Fully Developed Products are Tested Reality − No doubt, testing depends on the source code but reviewing requirements and developing test cases is independent from the developed code. However iterative or incremental approach as a development life cycle model may reduce the dependency of testing on the fully developed software. Myth 4: Complete Testing is Possible Reality − It becomes an issue when a client or tester thinks that complete testing is possible. It is possible that all paths have been tested by the team but occurrence of complete testing is never possible. There might be some scenarios that are never executed by the test team or the client during the software development life cycle and may be executed once the project has been deployed. Myth 5: A Tested Software is Bug-Free Reality − This is a very common myth that the clients, project managers, and the management team believes in. No one can claim with absolute certainty that a software application is 100% bug-free even if a tester with superb testing skills has tested the application.

17. Myth 6: Missed Defects are due to Testers Reality − It is not a correct approach to blame testers for bugs that remain in the application even after testing has been performed. This myth relates to Time, Cost, and Requirements changing Constraints. However the test strategy may also result in bugs being missed by the testing team. Myth 7: Testers are Responsible for Quality of Product Reality − It is a very common misinterpretation that only testers or the testing team should be responsible for product quality. Testers’ responsibilities include the identification of bugs to the stakeholders and then it is their decision whether they will fix the bug or release the software. Releasing the software at the time puts more pressure on the testers, as they will be blamed for any error. Myth 8: Test Automation should be used wherever possible to Reduce Time Reality − Yes, it is true that Test Automation reduces the testing time, but it is not possible to start test automation at any time during software development. Test automaton should be started when the software has been manually tested and is stable to some extent. Moreover, test automation can never be used if requirements keep changing. Myth 9: Anyone can Test a Software Application Reality − People outside the IT industry think and even believe that anyone can test a software and testing is not a creative job. However testers know very well that this is a myth. Thinking alternative scenarios, try to crash a software with the intent to explore potential bugs is not possible for the person who developed it. Myth 10: A Tester's only Task is to Find Bugs Reality − Finding bugs in a software is the task of the testers, but at the same time, they are domain experts of the particular software. Developers are only responsible for the specific component or area that is assigned to them but testers understand the overall workings of the software, what the dependencies are, and the impacts of one module on another module.

18. Estimation Techniques in Software Testing

19. One of the most important factors while estimating testing efforts is the hands-on experience on varied projects for the software test life cycle. No longer can one just take a guessing approach about the number of days for any task or working on the old-time formula of one-third of the development effort. • Although this method is not based on any scientific principle or technique, it is one of the most widely used estimation technique by companies offering software testing services. Unfortunately, the development versus testing effort method has given many failures in software projects in the past, thereby compromising the software or mobile apps on quality. • In recent years there have been many techniques that have been developed for estimating the software testing timeframe. These techniques are: 3-Point Software Testing Estimation Technique, Use-Case Point Method and Wide Band Delphi Method. • 3-Point Software Testing Estimation Technique • 3-Point Software Testing Estimation Technique is based on statistical methods in which each testing task is broken down into sub-tasks and then three types of estimation are done on each tasks. • The formula used by this technique is:Test Estimate = P + (4*N) + E / 6Where, • P = Positive Scenarios • N = Negative Scenarios • E = Exceptional Scenarios • Standard deviation for the technique is calculated as, • Standard Deviation (SD) = (N – E)/6

20. Use – Case Point Method • Use-Case Point Method is based on the use cases, where we calculate the unadjusted actor weights and unadjusted use case weights to determine the software testing estimation. • The formula used for this technique is: • Unadjusted actor weights = Total no. of actors (positive, negative and exceptional) • Unadjusted use case weight = Total no. of use cases • Unadjusted use case point = Unadjusted actor weights + Unadjusted use case weight • Adjusted use case point = Unadjusted use case point * [0.65+ (0.01 * 50] • Total Effort = Adjusted use case point * 2 • Wideband Delphi • In Wideband Delphi Method, work structure is broken down for each task and is distributed to a team comprising of 3-7 members for re-estimating the task. The final estimate is the result of the summarized estimates based on the team consensus. This method speaks more on experience rather than any statistical formula. The Wideband Delphi testing estimation technique logically estimates the group iteration efforts required in a visual manner for the testing team. This test was coined by Barry Boehm and is widely accepted software testing estimation technique to solve complex problems. • How to evaluate a software testing estimation technique? • For any software testing estimation technique, it is highly recommended that the following factors should be taken into account: • Domain knowledge and core requirements • Risks and complexity of the application • Team knowledge on the subject/skills • Historical data for the previous estimation for improvement and accuracy • Estimation should include buffer time • Bug cycles for the project • Resources availability

21. Aravali College of Engineering And Management Jasana, Tigoan Road, Neharpar, Faridabad, Delhi NCR Toll Free Number : 91- 8527538785 Website : www.acem.edu.in