Many types of software testing are conducted along a software development’s life cycle to ensure that the software you release to your customers is stable and behaves as expected.
Read more: smoke testing and user acceptance testing
One such type of testing is performance testing which seeks to validate the performance of the system and that your application provides an optimal user experience. It seeks to determine how the components of a system perform under certain conditions and various workloads.
Performance testing will primarily highlight any improvements you need to make to your software to ensure its stability, responsiveness and speed before it’s released into production.
In this article, we will focus on a subcategory of performance testing known as soak testing.
What is soak testing?
Soak testing, which is sometimes referred to as endurance testing, is a type of performance and load test that evaluates how a software application handles a growing number of users for an extended period of time.
Therefore, the goal of this type of testing is to ensure that the software application can handle a huge volume of load and determine whether it will exhibit abnormal behavior, or even crash, under such circumstances.
In other words, as its name suggests and from the word ‘soak’, the software is ‘soaked’ in or subjected to high load for a specified period of time.
Soak testing will be able to reveal any issues that can only come up after hours, or even days, of usage and to observe what would happen outside its design expectations.
Soak testing is usually run before the build is deployed. The application will then undergo a series of load tests after which soak testing is performed. Soak testing will then be the last step to ensure the reliability of your software.
Since this testing is done for a long period of time, it should usually be done on the weekends or overnight to ensure minimal disruption to the development team.
Due to the long duration of this test, it will need to be conducted with the help of an appropriate automation tool and so your team will need to possess the technical expertise to use such tools to run a successful soak test.
Why is soak testing necessary?
The reason this type of testing is performed is to check how the system behaves when it’s used continuously and under sustained load. As a result, the primary purpose of this test is to validate system behavior in production use for extended periods of time.
The main issues that soak testing seeks to uncover include:
- Memory leaks that may occur after extended usage- developers can find any potential leaks which might negatively impact performance and fix them
- Response time degradation– to ensure that response time after continuous use remains the same and as good as it was at the beginning of the test
- System resources degradation that would usually surface when a test is run for a long period of time
- Database connection issues– failure to close database cursors under certain conditions may result in the application crashing
- Layer connections failure– soak testing will reveal whether there is a failure of close connections between the layers of a system which could interrupt the modules of the system
It may behave normally for a couple of hours or so during other stages of software testing but soak testing will reveal whether the system remains stable if it’s used for a longer period of time or whether it would crash or behave abnormally.
As we’ve mentioned, the main problems this testing seeks to uncover are ones such as memory leaks or degradation in performance that could occur over time due to continuous usage.
Consequently, for this test, teams will seek to mimic real-world usage.
This means that soak testing gives developers the peace of mind that the system would run smoothly for longer periods of time without any maintenance required, thereby reducing higher maintenance costs if not identified during the testing stages.
As with other types of testing, there are some disadvantages associated with soak testing. For one, as this test usually runs for a long period of time, it can then be a time-consuming (and costly) process which could delay the release of the software.
It is also sometimes hard to determine how long to run the test exactly. You may run it for a good period of time and it passes the test but perhaps if you had just waited an hour longer, the application might end up crashing. In short, soak testing requires very careful planning.
Additionally, there’s the issue of having to run this test in a separate environment rather than the live environment to avoid major issues such as data loss and this will require extra resources and additional costs.
Soak vs load vs stress testing
There are many different types of performance tests and in this section we will discuss the differences between soak testing and other types of testing such as load and stress testing because while they may seem similar on the surface, they’re not exactly the same.
Load testing tests how a system behaves with a large number of users. The system is tested by slowly adding more load to it until it reaches the threshold limit. Thus, load testing is used to identify how many users a system can handle without any degradation to its performance.
Meanwhile, the purpose of stress testing is to investigate how a system behaves in extreme conditions, under high levels of stress. Stress testing may go beyond the limits of what the system can withstand, beyond the peak levels covered in load testing, and go so far as to purposely break it down.
Therefore, it looks to test the stability of the system beyond its bandwidth capacity.
This will help teams investigate how the system recovers under such heavy stress, especially if you’re expecting unusually heavy traffic at a certain period and you want to test whether your system can withstand this sudden influx of very high traffic. If the stress test is successful then that means the system was able to return to normal after a breakdown.
Soak testing, for its part, is concerned with system reliability over a longer period of time. It’s a means to monitor and verify that the system can sustain continuous high levels of load.
To sum up:
- A soak test focuses on determining the reliability of a system over an extended period of time
- A load test seeks to monitor performance of the system under increasing load up to a certain threshold
- Meanwhile, a stress test measures the system’s stability beyond its maximum level, under extreme load
The following graphs illustrate the aforementioned points:
Load testing (load is gradually increased until peak load which is maintained for a short period)
Soak testing (gradual load over an extended period)
Stress testing (increasing load until the breaking point)
Conclusion
As we’ve seen, soak testing is a powerful form of load and performance testing that helps to ensure the reliability of your software.
Though it can be a time-consuming process and requires careful planning, it is useful in identifying performance issues that may arise during continuous use of an application and to determine whether an application can handle continuous load.
How imperative soak testing is will depend on your own circumstances. If, for example, you’re an e-commerce website that is running an online sale for a few days then it would be important to run a soak test in this scenario to ensure you’re prepared beforehand by making sure that your website can handle such an unexpected high load without crashing.
As with any type of software testing, it’s always best to uncover any underlying issues and bugs before it’s launched into production, after which it becomes more costly to fix these issues.
