As the combination of cloud and CI technology enables the “shift left” of DevOps, load testing is joining the crowd.
Traditionally, load testing was used as a check before large releases of software or changes in infrastructure or an expected spike in usage to assure applications were performant. Solutions like Load Runner, Blazemeter and Apica were perfect for this use case.
With the focus on automating the delivery of software in small, incremental steps with continuous integration tools like Jenkins, there is the ability to do load testing incrementally as well. As with the shift left concept in general, the idea is to catch problems early before they compound. This means putting at least some load testing into the CI pipeline. This badly breaks the legacy tools from a scaling and pricing model basis.
RedLine13 is becoming increasingly popular for the early adopters of shifting load testing into the CI process. It has the following characteristics that appeal to DevOps shops:
- Auto deploys and shuts down load agent EC2 instances and spot instances. This saves money since instances are just running for a few minutes at a time, and there is no manual intervention required. And all instances run in the customer’s chosen account so they are secure.
- Any language or load testing framework – JMeter, Gatling, WebDriver, Selenium, Node, Python, PHP, Executable subprocesses, etc.
- Web Application and API Endpoint Testing.
- Jenkins Integration
- API’s for integration into any CI process
- Unlimited tests included in base price of $500 or $2,500 per year. Your only cost is your own EC2 and bandwidth usage.
- Unlimited scale. RedLine13 has customers that are testing 1.5 Million simulated users.
RedLine13 customer Renaissance is fully bought into shift left. They have 400 developers and one performance engineer with the developers doing the load testing. They integrated RedLine13 into the Jenkins pipeline, and use S3 to store their test plans. This provides a combination of security and flexibility since all the load tests run in their own environment.
As DevOps matures, more and more companies will leverage automation and bring load testing into the CI pipelines.