Software Testing Handbook: An In-Depth Look at QAT Types

Software testing is an essential part of the software development process. It helps to identify bugs, defects, and errors in the software code and ensures that the final product is of high quality.

There are different types of software testing methods that are used to test software products, such as functional testing, performance testing, security testing, and more. The choice of testing method depends on the type of software being developed and the requirements of the project.

A report by the National Institute of Standards and Technology (NIST) states that the most common type of software testing method used is functional testing, which accounts for 80% of all testing activities.

Effective software testing requires a skilled workforce that understands the different testing methods and techniques. However, there is a shortage of skilled software testers globally.

According to a report by Indeed, the demand for software testing professionals has increased by 48% over the last five years, but the supply of qualified candidates has not kept up with this demand. As a result, organizations are investing heavily in training and upskilling their workforce to fill the gap.

In this article, we will take an in-depth look at the art of software testing, exploring the different types of software testing, their importance, and the trends and data that are shaping the software testing industry.

Whether you’re an experienced software tester or new to the field, this article will provide you with valuable insights to enhance your software testing skills and knowledge.

What is exploratory testing?

Exploratory testing is a testing approach where the tester actively explores the software in an unscripted and unplanned manner, looking for any unexpected behavior. It involves creating and executing test cases on the fly, based on the tester’s intuition and experience. The goal of exploratory testing is to find defects that may have been missed by scripted testing.

For example, if testing a web application, an exploratory testing session might involve navigating the website without a set plan, trying different inputs and interactions to see if any unexpected behavior occurs. The tester might also take notes on any defects found during the session, which can be used to create more formal test cases later on.

How do you perform ad-hoc testing?

Ad-hoc testing is a testing approach where the tester tests the software without a specific plan or script. It involves using the software in an unstructured and unplanned manner, with the goal of finding defects that might have been missed by formal testing methods.

To perform ad-hoc testing, the tester simply starts using the software and tries different inputs and interactions to see if any unexpected behavior occurs. The tester may also try to reproduce any defects found during previous testing sessions.

For example, if testing a mobile application, ad-hoc testing might involve using the app in different environments, trying different inputs and interactions, and seeing how the app responds. The tester might also try to break the app by performing actions that are not part of the normal workflow, such as entering invalid data or navigating to different screens in an unexpected order.

What is a test suite?

A test suite is a collection of related test cases that are executed together as part of a larger testing effort. It is typically used to test a specific feature or functionality of the software. A test suite can be executed manually or using automated testing tools.

For example, if testing an e-commerce website, a test suite might include test cases for creating an account, adding items to a shopping cart, and checking out. These test cases would be executed together as part of the larger testing effort to ensure that the entire e-commerce workflow is working as expected.

What is boundary testing?

Boundary testing is a testing technique where the tester tests the software at the limits of its input parameters. It involves testing the minimum and maximum values for each input parameter to ensure that the software behaves as expected.

For example, if testing a calculator application, boundary testing might involve testing the minimum and maximum values for each mathematical operation, such as dividing by zero, multiplying by very large numbers, or subtracting from very small numbers.

The goal of boundary testing is to ensure that the software can handle all possible inputs within the specified range.

What is negative testing?

Negative testing is a technique used in software testing to verify how the software behaves when it is presented with invalid, incorrect or unexpected input. It is a type of testing that focuses on the system’s ability to handle invalid data or unexpected events. The goal of negative testing is to identify defects and errors in the software that could potentially cause harm to the system or the end-users.

For example, in an e-commerce website, negative testing can involve entering invalid data in the payment page such as incorrect credit card number, wrong expiry date or entering a wrong address in the shipping section. The system should respond to these invalid inputs by providing appropriate error messages or by rejecting the input.

Negative testing can be performed manually or through automated tests using tools like Selenium, JUnit, and TestNG.

Some of the advantages of negative testing are:

• Helps in uncovering critical defects that could potentialy harm the system or the users
• Improves the overall quality of the software
• Ensures that the software can handle unexpected inputs and events

However, some of the challenges of negative testing are:

• It can be time-consuming to create and execute negative test cases
• Requires a good understanding of the system and the potential invalid inputs
• It is not possible to test all the possible invalid inputs and scenarios
• Overall, negative testing is an important technique in software testing to ensure the software’s quality and reliability.

What is equivalence partitioning?

Equivalence partitioning is a technique used in software testing to reduce the number of test cases required while ensuring adequate test coverage. The goal of equivalence partitioning is to divide the input domain of a software system into a set of equivalent classes that have similar behavior.

For example, suppose we have a system that accepts a numerical input in the range of 1 to 100. We can divide the input domain into three equivalent classes – inputs less than 1, inputs between 1 and 100, and inputs greater than 100. We can then test one value from each class to ensure that the software behaves similarly for all values in the class.

Equivalence partitioning can be applied to both input and output data. This technique helps in reducing the number of test cases required to achieve adequate test coverage, thus saving time and effort. However, it requires a good understanding of the system’s behavior and the input/output domains.

What is a test report?

A test report is a document that summarizes the results of a software testing effort. It provides an overview of the testing activities performed, the issues identified, and the status of the software under test. A test report is typically created at the end of the testing cycle and is used to communicate the testing results to the stakeholders.

A typical test report includes information such as:

• Test objectives and scope
• Test environment and setup
• Test execution summary
• Test case results and status
• Defects identified and their severity
• Test coverage metrics
• Recommendations for further testing or improvements
• Test reports can be customized to meet the specific needs of the stakeholders. They can be in various formats such as excel sheets, word documents, or PDFs.

The importance of a test report cannot be overstated, as it provides valuable information to stakeholders about the quality of the software being tested. A well-written test report helps in making informed decisions about the readiness of the software for release and helps in identifying areas that need further improvement.

What is the difference between black-box testing and white-box testing?

Black-box testing and white-box testing are two different approaches to testing software applications.

Black-box testing is a method of testing where the tester does not have access to the internal workings of the software being tested. The tester focuses on the inputs and outputs of the system, without knowledge of how the software processes the inputs or generates the outputs. This type of testing is focused on validating the functionality of the software and ensuring that it meets the specified requirements. Examples of black-box testing techniques include functional testing, system testing, and acceptance testing.

On the other hand, white-box testing is a method of testing where the tester has access to the internal workings of the software being tested. The tester focuses on testing the code and the logic of the system. This type of testing is focused on validating the design and architecture of the software, as well as ensuring that the code is optimized and efficient. Examples of white-box testing techniques include unit testing, integration testing, and performance testing.

What is acceptance testing?

Acceptance testing is a type of testing performed to ensure that a software application meets the requirements and specifications of the customer or end-user. It is typically the final stage of testing before the application is released to production. The goal of acceptance testing is to ensure that the software is usable and meets the needs of the customer.

There are two types of acceptance testing: user acceptance testing (UAT) and business acceptance testing (BAT). User acceptance testing is performed by the end-users of the software, while business acceptance testing is performed by the business stakeholders who are responsible for approving the software for release.

What is usability testing?

Usability testing is a type of testing performed to evaluate how easy it is to use a software application. The focus of usability testing is on the user interface and the user experience. The goal of usability testing is to identify any usability issues and to ensure that the software is user-friendly.

Usability testing can be performed in a variety of ways, including user surveys, focus groups, and user testing sessions. During a usability testing session, users are asked to perform tasks using the software while being observed by a tester. The tester records any issues the user encounters and uses that feedback to improve the usability of the software.

What is compatibility testing?

Compatibility testing is a type of non-functional testing that checks whether a software application can function correctly and efficiently in different environments, configurations, and systems. The goal of compatibility testing is to ensure that the software works as intended across a range of platforms, devices, operating systems, web browsers, databases, and other related components. The purpose of this testing is to identify compatibility issues and ensure that the application is fully functional in different environments.

For example, if a website is designed to work in Google Chrome, compatibility testing will ensure that the website also works well in other browsers such as Firefox, Safari, and Edge. Similarly, if an application is designed for Windows 10, compatibility testing will ensure that it also works on other operating systems like Linux and macOS.

Compatibility testing is important to ensure a good user experience for all users and to ensure the software is widely accessible. By testing compatibility, we can ensure that the software runs smoothly and without errors on all possible platforms and configurations.

How do you measure the effectiveness of your testing?

Measuring the effectiveness of testing can be done by various metrics such as code coverage, defect density, and test execution progress.

Code coverage: Code coverage measures the percentage of the source code that has been executed during testing. The higher the code coverage, the more thoroughly the software has been tested.
Defect density: Defect density measures the number of defects found per unit of code or per test case. A low defect density indicates that the software is of good quality and has fewer defects.
Test execution progress: Test execution progress measures the percentage of test cases executed versus the total number of test cases. This metric provides insight into the progress of testing.

What is the difference between a test case and test scenario?

A test case is a detailed set of instructions or steps that a tester follows to execute a test. It includes preconditions, inputs, expected outcomes, and post-conditions. A test case is designed to test a specific functionality or feature of the software.

On the other hand, a test scenario is a broader and more high-level description of a test. It is a collection of related test cases that are grouped together based on a common objective or goal. A test scenario is designed to test a particular aspect of the software and can consist of multiple test cases.

What is the role of a defect triage meeting in testing?

Defect triage meeting is a process of analyzing and prioritizing defects found during testing. It is a meeting where the project team comes together to discuss and categorize the defects based on their severity, impact, and priority.

The purpose of a defect triage meeting is to:

• Determine the root cause of the defect
• Prioritize the defects based on their severity and impact
• Decide on the corrective actions to be taken to resolve the defects
• Ensure that the defects are resolved in a timely manner
• Identify any patterns or trends in the defects and take corrective actions to prevent similar defects from occurring in the future.

Defect triage meeting helps to ensure that the project team is aligned on the defects and their priority, and helps to make decisions on how to address them.  

What is the difference between a bug and a defect?

The terms “bug” and “defect” are often used interchangeably in the software testing industry, but there is a subtle difference between the two. A bug is a general term used to describe any unexpected behavior in the software. It can refer to any kind of issue, whether it is a coding error, a design flaw, or a functional problem.

A defect, on the other hand, is a specific type of bug that occurs when the software fails to meet its intended requirements or specifications.

For example, if a software application crashes unexpectedly, it would be considered a bug. However, if the application crashes only when a specific input is entered, this would be considered a defect because it is a specific failure to meet a requirement.