From the Open Source Load Testing presentation http://www.slideshare.net/richardfriedman/open-source-load-testing
Perhaps in a perfect world, every time code is submitted and reviewed, a Jenkins job kicks off a load test against your specific change. However, in reality, this has quite a few issues:
- Too early. Load testing too early in your process is noise
- Time. Realistically, most projects are not scoped with time for this
- Scope. Changing a readme file should not generate a load test
For me, I have found a set of reasons different than supporting a continuous software development lifecycle to integrate load testing with my continuous delivery tools.
- Faster iteration. Remember, kicking off a load test takes time, having a location that gives you single click to kick off a load tests saves time
- Master Location of Data. You can configure how many previous runs of your job for Jenkins to hold on to, now you have one simple place to see your results.
- Trend Lines. Tools could aggregate results from job to job and show you a trend line of your results.
- Recorded Tests is work done. Not sure the bullet represents the intention, but it is a quicker way for your team members to find what tests have been built already.
- Agile Development Process. And if you do want to integrate with your SDLC, it will be there for you. Alexander Podelko covers Agile Performance Testing in detail and also opens up the idea of Unit Performance Testing. He has covered load testing from so many aspects, his perspective on the topic is always valuable.
- Bonus: Multiple tools. With Jenkins you could run a Selenium functional and JMeter load test within the same build, keeping results and workflow together.
I am suggesting you integrate with CD tools, but not always with the purpose of a more iterative process. Instead, the minimum, I believe it will save you time and headaches by being integrated early.
Good read on this over sys-con as well – Automation in Performance Testing
