What is Software Testing?

To err is human. No matter how much of a perfectionist we are, we are all bound to make mistakes. Software testing can help prevent and discover mistakes in software development.

If you’d had to answer “What is software testing?” what would you say? For me it is something pretty difficult to compress into a couple of words. It gets even more challenging considering the many misconceptions about what software testing is, what testers do and the different perceptions among the testers themselves. Testing as a skill and as industry is constantly evolving. Similarly our answer to what software testing is might be different today compared to what it was in the past, and might still be different to what it will be in the future.

Many may answer that software testing is the process of verifying a system with the purpose of identifying any errors, gaps or missing conditions versus the actual requirement. This is not wrong per se but for me testing is much more than that.

Valuable

Can you describe the value of software testing in software development? It is often misunderstood due to its unpredictable and creative nature. This comes with the difficulty to measure the tester’s output. At least in comparison it’s relatively easy for developers whose output is the code or for designers whose output are new features and documentation. On top of this the very nature of a tester’s work can make it challenging for one to communicate their plans, progress and outcomes. In turn this makes it difficult for others to understand what has been done, how it has been done, and why. Hence the struggle to understand the value of testers and testing.

The lack of countable things created by testers lets many people fall back to measurable things like test cases as they are tangible and countable output or other metrics. However, often testers find the more interesting bugs by not following a scripted path. So these measurements often focus on the wrong things. They can be misleading and motivate the wrong types of behavior for testers.

We do test at all stages of the software development lifecycle, not just when code has been written, which includes many more items which can be tested:

• Ideas
• Designs
• Assumptions
• Documentation
• Requirements
• Processes

It is our job to ask questions, to explore and think critically about all these things, as it could result in something that could have turned into a bug later in the process.

Communication

A massive part of our job is communication. We provide information about the quality of the product, so it’s important we can communicate it clearly and accurately to enable the right decisions.

As testers we need to use the right words and phrasing to avoid ambiguity and to reduce the risk of misunderstanding. Sometimes what you mean to say is not what you end up saying, leading to assumptions. This can be quite bad if incorrect actions are taken as a result of poor or insufficient communication.

We need to communicate regularly with people in different roles, at different levels of seniority and with different levels of knowledge. This ranges from developers over product owners, to other testers, to users and managers. Everyone has different needs and requirements on information which has to be communicated in different ways.

Written communication is equally as important as the spoken word. It’s easy to produce brilliantly written, extensive documentation which turns out to be unnecessary and which no one reads. We need to ensure we choose the right way to communicate which is most valuable to the recipient, the process, and the project.

Exploration

You have a list of requirements? That’s great! But is this list actually complete? Regardless of how comprehensive your list already is, there might always be implicit requirements, requirements which are not stated, which are assumed or have been omitted. As we don’t know the full final details of our product upfront, exploratory testing can help us.

Exploratory testing is described as simultaneous learning, test design and test execution. In other words it means performing tests while learning things that may influence the testing. It is about exploring the product, gaining new information and new things to learn which define what to test further. This can be done alone, in pairs or even in groups where also other stakeholders can take part as they bring their own perspective into the testing.

In that sense testing should not be perceived as going through prepared tests or test cases. It is of course important to have these in mind when looking at the requirements of a user story but checking and validation should always be combined with exploration and investigation. Ask questions about the product like “What happens if..?”, questions where you don’t know the answer before you start.

Investigation

To investigate is defined as to examine something carefully, especially to discover the truth about it. When we test we may not know what the outcome will be. Discovering new information is part of testing. Information that will help the stakeholders to make informed decisions. This is much more than checking whether the actual results match the expected results. It is about thinking critically, engaging with the product, questioning premises, managing assumptions, noticing things which seem trivial at first but are deemed much more important after a closer look and which need further investigation.

This investigation can come in different forms. It can come as something which has no impact on the user or the product itself and could be simply ignored, but when starting to question this behavior we may discover some underlying structural issues which in return can put the product at risk. It is similar with bugs, some may look trivial at first with no big impact but when actually digging into them, it can open Pandora’s box. No matter which information is discovered by the investigation, it will help the stakeholders to make informed decisions on whether it needs to be fixed right now, in the next version or even never.

Mitigation

Why do you test? One reason we test is to discover issues, risks and other unknown information we didn’t know about the product. This enables us to take action so that our users are not adversely impacted by them. These actions might be:

• Fixing defects
• Communicating known issues
• Creating documentation
• Providing user support
• Re-evaluating requirements

Independent of the complexity of the product, it is impossible to resolve every single issue a user may face. However, by testing we can reduce the risk or the severity of issues they experience.

Infinite

Last but not least, testing is never finished. All testing is sampling, as for any non trivial software there is an unimaginable amount of parameters. We can’t test all possible combinations of these parameters. We can’t test everything.

Hence we use different strategies to select the tests which are feasible for the available time and resources. It is our responsibility to make the decision about what to test, how to test, understanding the consequences of only testing those things and being able to explain our decisions.

These are just some of the things that software testing is and by far this is not an exhaustive list. If we would include the things which software testing is not, the list would be just as long. This also shows that testing is everything else than simple. It isn’t scripted, ad hoc or just playing around with the software to see what happens. It is a structured, technical activity. To explore an application at a deeper level could require knowing something about the architecture, the technologies used, as well as the psychological aspect of thinking like different types of user.

Testing in this definition cannot be automated. Automation can significantly help the project with checks and validation, to ensure the functionality is still working and thus is extremely valuable. But testing as an exploratory and investigative activity, cannot be replaced by automated checks. A person will pick up on other things while they are carrying out the testing, will take notice of any feeling telling them something doesn’t seem right, and provide feedback beyond a pass or fail result.

You can see that here is no “one” definition of software testing and it depends on the context, the industry, the team, the project and the people on what software testing is. You may have gotten an understanding that testing is much more than executing software with the aim of finding bugs.

InnoGames is hiring! Check out open positions and join our awesome international team in Hamburg at the certified Great Place to Work®.