A JMeter Soak Test is a long running load test using JMeter. We’ll talk more below. But if you’re going to run a long test, there are costs that need to be understood. We’ll talk about that with an example from RedLine13 customer University of California.
What is Soak Testing
There is plenty of information out there. Software Testing Help says: “It is a type of performance testing to check if an Application Under Test (AUT) can withstand continuous load for a pre-determined time frame.” They go on to talk about the need, when to start, strategies and constraints.
What it Costs to Soak Test
Since load testing is almost entirely done in the cloud where there is a per hour or per minute cost, any team doing soak testing is going to have to answer the question of what this will cost. We decided it would be most meaningful to get information from a RedLine13 customer.
University of California and Soak Testing
The University of California is a RedLine13 customer. We asked Anukrati Manerikar, from the University of California Office of the President, about the JMeter soak testing that they do. They were running soak tests on the HCM PeopleSoft online web portal application. She provided information on an actual test with a duration of 13 hours for 3,500 concurrent users. The Load Agent ran on a AWS m5.12Xlarge server.
We need to next explain the RedLine13 pricing model which is unlike others in the load testing industry. RedLine13 charges a subscription price like others but it’s much cheaper and has almost no limitations. The only other cost for our customers is the AWS cost to run the load agents. The load agents are running on the customer AWS servers. That means AWS charges the customer directly without any markup by RedLine13. And RedLine13 doesn’t get back any of the cost charged by AWS.
Back to our example, the University of California ran their test on an m5.12Xlarge server with 48 vCPUs for 13 hours. And the cost was a little less than $36 as you can see in the screenshot below.
Yes, that’s their total cost.
We can look at this summary screenshot and see some other interesting information. During the soak test, over 2 million requests were sent. You can also see response time information, data usage, etc. RedLine13 also provides much more detailed information.
Anukrati spoke at the RedLine13 Customer Advisory Board meeting. She spoke about moving from LoadRunner, the quality of the RedLine13 documentation, and more. Read about her talk here with a link to a video of her talk.
JMeter Soak Testing on 30 AWS Servers for 27 Hours – A RedLine13 Customer Shares Their Story
Read about another customer running an even bigger soak test and what their costs were.
Try a JMeter Soak Test on RedLine13 for Yourself
You can try your own soak test on RedLine13 for free and see what it will cost you.