Scalability is one of the greatest strengths of using a cloud-based architecture. However, there are occasionally limitations to how quickly we can scale. This is especially true for high volume load testing. In these scenarios, we need to instantaneously scale a practically nonexistent cloud architecture to our maximum desired size. It is not always guaranteed that cloud providers like AWS will have these resources idle and waiting in the necessary quantities. In this post we will discuss using AWS capacity reservations for your large load tests to ensure you have the load generators you need.
Even though it may seem like cloud compute instances are abundantly available, in reality the number of instances that you request cannot always be guaranteed. If you have a JMeter load test requiring dozens ‒ or even hundreds ‒ of load generators, during peak demand you may not be able to spin up these instances in the desired quantities.
Capacity reservations in AWS allow you to “reserve” a certain number of EC2 instances for a particular duration. For the duration of your reservation, you can be assured that you can provision an arbitrary number of instances up to your reserved quantity.
In order for your RedLine13 load generators to launch within capacity reservations, you will need to ensure the following conditions are met:
- Instances launched into the capacity reservation must match exactly in terms of instance type (e.g., m3.medium).
- The platform specified in your capacity reservation must be “Linux/UNIX“.
- The availability zone of your capacity reservation must also include the specific subnet that you will be launching instances into (e.g., US-East-1a vs. US-East-1b subnets, even though in the same availability zone, would correspond to different capacity reservations).
- Instances must be launched as “On Demand” – i.e., spot instances will not launch into the capacity reservation.
In addition, you will want to make sure that you have selected the “Any instance with matching details” option. This will ensure that any matching on-demand instance will launched into the capacity reservation:
These conditions are set by AWS as outlined in their documentation. As long as all of the above conditions are met, your on-demand instances should launch into your capacity reservation.
Here is one customer’s experience using AWS Capacity Reservations in conjunction with RedLine13 :
Our professional services had to run a performance test on Video Streaming servers at very high scale (150,000 Players simulation), so we had to run hundreds of EC2 servers. To ensure AWS could provision enough servers during our test, we used the AWS capacity reservation feature which works very smoothly in Redline13. We thank their great support who provided all required information.
– Philippe M., team leader of UbikLoadPack solution
The primary disadvantage to capacity reservations is that they are billed at the on-demand rate for the entire reserved duration. If the exact time frame is not known upfront, this can be problematic. One effective solution to mitigate costs associated with capacity reservations is to combine them with reserved instances.
Reserved instances allow you to specify a known compute configuration for a specified duration of time. Unlike a capacity reservation, reserved instances only guarantee a certain discount in price over on-demand rates ‒ not necessarily that you can provision those instances at any time. If, however, you hold a capacity reservation that matches a reserved instance provisioning, then your capacity reservation can benefit from reserved instance pricing discounts.
Did you know that RedLine13 offers a fully-featured free trial? Sign up now and begin testing today!