{"id":8266,"date":"2021-08-31T14:26:03","date_gmt":"2021-08-31T18:26:03","guid":{"rendered":"https:\/\/www.redline13.com\/blog\/?p=8266"},"modified":"2021-08-31T14:26:03","modified_gmt":"2021-08-31T18:26:03","slug":"peak-performance-testing","status":"publish","type":"post","link":"https:\/\/www.redline13.com\/blog\/2021\/08\/peak-performance-testing\/","title":{"rendered":"Measure True Peak Performance Testing by Removing Outliers"},"content":{"rendered":"

Sometimes we test a system with a specific goal of measuring sustained performance capability. For that we want to measure true peak performance testing by removing outliers at the beginning or end of our tests that may otherwise influence performance characteristics. In this post we will describe how to accomplish this using a basic JMeter<\/a> post-processor script. \"peak<\/p>\n

Suppose for a moment I were a cyclist, and I was attempting to calculate my average biking speed. For very long rides, this might be as straightforward as dividing distance by time. However, if I were to calculate it the same way over a series of short rides my average speed would be slower. This is because a greater portion is spent accelerating at the beginning, and decelerating at the end.<\/p>\n

With a cloud-based load test (especially those of shorter duration) the amount of time spent ramping up a test can be significant. Depending on the statistics we are collecting, we may want to exclude results collected during these times as they will be influenced by extraneous factors (internal server processes, setup, etc<\/em>.) and not be indicators of true performance. We will show you a simple way to measure the true peak performance of your test by removing these outliers.<\/p>\n

Removing Outliers<\/h3>\n

We will use JMeter for our example because it has become by far the most popular<\/a> load testing framework on RedLine13. Using JMeter, we can add a basic post-processor<\/a> script (BeanShell<\/a> or JSR223<\/a> + Groovy<\/a>) that will automatically exclude all results outside a predetermined time frame. We will illustrate this here by adding a JSR223 post-processor to our Thread Group<\/em>:<\/p>\n

\"Adding
Adding a post-processor: JSR223 is preferred over BeanShell for performance reasons.<\/figcaption><\/figure>\n

We can set the \u201cLanguage<\/em>\u201d property of this post-processor to either \u201cgroovy<\/strong><\/span>\u201d (recommended) or \u201cbeanshell<\/strong><\/span>\u201d, and to execute the following code:<\/p>\n\n\n\n
\n
if(args[0].toBoolean()) {<\/strong><\/span>\n    int before_ms = (args[1].toInteger() * 60 * 1000);<\/strong><\/span>\n    int after_ms = (args[2].toInteger() * 60 * 1000);<\/strong><\/span>\n\n    long start_time_ms = vars.get(\"TESTSTART.MS\").toLong();<\/strong><\/span>\n    long current_duration_ms = (new Date().getTime() - start_time_ms);<\/strong><\/span>\n\n    if ((current_duration_ms < before_ms) ||<\/strong><\/span>\n        (current_duration_ms > after_ms)) {<\/strong><\/span>\n        prev.setIgnore();<\/strong><\/span>\n    }<\/strong><\/span>\n}<\/strong><\/span><\/pre>\n<\/td>\n<\/tr>\n<\/thead>\n<\/table>\n
Finally, under the \u201cParameters<\/em>\u201d section of the JSR223 post-processor, we will want to specify our \u201cbefore\u201d and \u201cafter\u201d durations which will constrain our results:<\/p>\n
\"Specifying<\/a>
Multiple parameters for the post-processor script are separated with a space.<\/figcaption><\/figure>\n
Here we have set the test to exclude results for the first 5 minutes, and then after the 30 minute mark. (These values are picked up as args[…]<\/strong><\/span> and converted to milliseconds in the first few lines of our above script.)\u00a0 The first argument is simply a flag which we have configured to conveniently enable or disable the conditions within our script.<\/p>\n
Alternatively we could also pass these parameters into JMeter as properties rather than hard-coding them into our test.\u00a0 Our script would remain the same, but we might specify our arguments using the __P() property function<\/a>.\u00a0 This offers the advantage of being able to configure test timing by setting these properties in RedLine13 as described in this post<\/a>.\u00a0 From the perspective of our above example, this may be set as follows:<\/p>\n
\"Specifying<\/a><\/p>\n
Running this test with the sampler enabled will show us a reduction in captured results:<\/p>\n
\"Comparing<\/p>\n
This is both expected and desired. If we compare the output of the original test plan, a subsequent run with the post-processor enabled will only consider the area shaded below:<\/p>\n
\"peak
Ramp up and ramp down periods can be excluded from true peak performance testing metrics.<\/figcaption><\/figure>\n
This effectively considers only the region of the test unaffected by confounding ramp up and ramp down factors. The effect on the average throughput of the sample test used here can be seen as follows:<\/p>\n
\"Comparing<\/p>\n
Free Trial<\/h3>\n
Did you know that RedLine13 offers a full-featured free trial? Sign up today<\/a> and try out this example for yourself!<\/p>\n","protected":false},"excerpt":{"rendered":"
Sometimes we test a system with a specific goal of measuring sustained performance capability. For that we want to measure true peak performance testing by removing outliers at the beginning or end of our tests that may otherwise influence performance characteristics. In this post we will describe how to accomplish this using a basic JMeter post-processor script. Suppose for a moment I were a cyclist, and I was attempting to calculate my average biking speed. ForRead More →<\/a><\/p>\n","protected":false},"author":1,"featured_media":8267,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2],"tags":[283,384,394],"class_list":{"0":"entry","1":"post","2":"publish","3":"author-user","4":"post-8266","6":"format-standard","7":"has-post-thumbnail","8":"category-blog","9":"post_tag-jmeter","10":"post_tag-peak-performance-testing","11":"post_tag-performance-testing"},"yoast_head":"\nMeasure True Peak Performance Testing by Removing Outliers<\/title>\n<meta name=\"description\" content=\"Measure true peak performance testing by removing outliers at the beginning or end of our tests that may otherwise influence performance characteristics\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.redline13.com\/blog\/2021\/08\/peak-performance-testing\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Measure True Peak Performance Testing by Removing Outliers\" \/>\n<meta property=\"og:description\" content=\"Measure true peak performance testing by removing outliers at the beginning or end of our tests that may otherwise influence performance characteristics\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.redline13.com\/blog\/2021\/08\/peak-performance-testing\/\" \/>\n<meta property=\"og:site_name\" content=\"RedLine13\" \/>\n<meta property=\"article:published_time\" content=\"2021-08-31T18:26:03+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.redline13.com\/blog\/wp-content\/uploads\/2021\/08\/word-image-1.png\" \/>\n\t<meta property=\"og:image:width\" content=\"400\" \/>\n\t<meta property=\"og:image:height\" content=\"300\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"RedLine13\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"RedLine13\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.redline13.com\/blog\/2021\/08\/peak-performance-testing\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.redline13.com\/blog\/2021\/08\/peak-performance-testing\/\"},\"author\":{\"name\":\"RedLine13\",\"@id\":\"https:\/\/www.redline13.com\/blog\/#\/schema\/person\/4acbcdcb8a9c72ec5a274e69c0ebea28\"},\"headline\":\"Measure True Peak Performance Testing by Removing Outliers\",\"datePublished\":\"2021-08-31T18:26:03+00:00\",\"dateModified\":\"2021-08-31T18:26:03+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.redline13.com\/blog\/2021\/08\/peak-performance-testing\/\"},\"wordCount\":611,\"publisher\":{\"@id\":\"https:\/\/www.redline13.com\/blog\/#organization\"},\"keywords\":[\"JMeter\",\"peak performance testing\",\"performance testing\"],\"articleSection\":[\"Blog\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.redline13.com\/blog\/2021\/08\/peak-performance-testing\/\",\"url\":\"https:\/\/www.redline13.com\/blog\/2021\/08\/peak-performance-testing\/\",\"name\":\"Measure True Peak Performance Testing by Removing Outliers\",\"isPartOf\":{\"@id\":\"https:\/\/www.redline13.com\/blog\/#website\"},\"datePublished\":\"2021-08-31T18:26:03+00:00\",\"dateModified\":\"2021-08-31T18:26:03+00:00\",\"description\":\"Measure true peak performance testing by removing outliers at the beginning or end of our tests that may otherwise influence performance characteristics\",\"breadcrumb\":{\"@id\":\"https:\/\/www.redline13.com\/blog\/2021\/08\/peak-performance-testing\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.redline13.com\/blog\/2021\/08\/peak-performance-testing\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.redline13.com\/blog\/2021\/08\/peak-performance-testing\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.redline13.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Measure True Peak Performance Testing by Removing Outliers\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.redline13.com\/blog\/#website\",\"url\":\"https:\/\/www.redline13.com\/blog\/\",\"name\":\"RedLine13\",\"description\":\"(Almost) Free Load Testing in the Cloud\",\"publisher\":{\"@id\":\"https:\/\/www.redline13.com\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.redline13.com\/blog\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.redline13.com\/blog\/#organization\",\"name\":\"RedLine13\",\"url\":\"https:\/\/www.redline13.com\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.redline13.com\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.redline13.com\/blog\/wp-content\/uploads\/2013\/06\/cropped-rl13-header-logo.jpg\",\"contentUrl\":\"https:\/\/www.redline13.com\/blog\/wp-content\/uploads\/2013\/06\/cropped-rl13-header-logo.jpg\",\"width\":300,\"height\":68,\"caption\":\"RedLine13\"},\"image\":{\"@id\":\"https:\/\/www.redline13.com\/blog\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.redline13.com\/blog\/#\/schema\/person\/4acbcdcb8a9c72ec5a274e69c0ebea28\",\"name\":\"RedLine13\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.redline13.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/b4c9a289323b21a01c3e940f150eb9b8c542587f1abfd8f0e1cc1ffc5e475514?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/b4c9a289323b21a01c3e940f150eb9b8c542587f1abfd8f0e1cc1ffc5e475514?s=96&d=mm&r=g\",\"caption\":\"RedLine13\"},\"sameAs\":[\"http:\/\/127.0.0.1\"],\"url\":\"https:\/\/www.redline13.com\/blog\/author\/user\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Measure True Peak Performance Testing by Removing Outliers","description":"Measure true peak performance testing by removing outliers at the beginning or end of our tests that may otherwise influence performance characteristics","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.redline13.com\/blog\/2021\/08\/peak-performance-testing\/","og_locale":"en_US","og_type":"article","og_title":"Measure True Peak Performance Testing by Removing Outliers","og_description":"Measure true peak performance testing by removing outliers at the beginning or end of our tests that may otherwise influence performance characteristics","og_url":"https:\/\/www.redline13.com\/blog\/2021\/08\/peak-performance-testing\/","og_site_name":"RedLine13","article_published_time":"2021-08-31T18:26:03+00:00","og_image":[{"width":400,"height":300,"url":"https:\/\/www.redline13.com\/blog\/wp-content\/uploads\/2021\/08\/word-image-1.png","type":"image\/png"}],"author":"RedLine13","twitter_card":"summary_large_image","twitter_misc":{"Written by":"RedLine13","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.redline13.com\/blog\/2021\/08\/peak-performance-testing\/#article","isPartOf":{"@id":"https:\/\/www.redline13.com\/blog\/2021\/08\/peak-performance-testing\/"},"author":{"name":"RedLine13","@id":"https:\/\/www.redline13.com\/blog\/#\/schema\/person\/4acbcdcb8a9c72ec5a274e69c0ebea28"},"headline":"Measure True Peak Performance Testing by Removing Outliers","datePublished":"2021-08-31T18:26:03+00:00","dateModified":"2021-08-31T18:26:03+00:00","mainEntityOfPage":{"@id":"https:\/\/www.redline13.com\/blog\/2021\/08\/peak-performance-testing\/"},"wordCount":611,"publisher":{"@id":"https:\/\/www.redline13.com\/blog\/#organization"},"keywords":["JMeter","peak performance testing","performance testing"],"articleSection":["Blog"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.redline13.com\/blog\/2021\/08\/peak-performance-testing\/","url":"https:\/\/www.redline13.com\/blog\/2021\/08\/peak-performance-testing\/","name":"Measure True Peak Performance Testing by Removing Outliers","isPartOf":{"@id":"https:\/\/www.redline13.com\/blog\/#website"},"datePublished":"2021-08-31T18:26:03+00:00","dateModified":"2021-08-31T18:26:03+00:00","description":"Measure true peak performance testing by removing outliers at the beginning or end of our tests that may otherwise influence performance characteristics","breadcrumb":{"@id":"https:\/\/www.redline13.com\/blog\/2021\/08\/peak-performance-testing\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.redline13.com\/blog\/2021\/08\/peak-performance-testing\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.redline13.com\/blog\/2021\/08\/peak-performance-testing\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.redline13.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Measure True Peak Performance Testing by Removing Outliers"}]},{"@type":"WebSite","@id":"https:\/\/www.redline13.com\/blog\/#website","url":"https:\/\/www.redline13.com\/blog\/","name":"RedLine13","description":"(Almost) Free Load Testing in the Cloud","publisher":{"@id":"https:\/\/www.redline13.com\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.redline13.com\/blog\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.redline13.com\/blog\/#organization","name":"RedLine13","url":"https:\/\/www.redline13.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.redline13.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/www.redline13.com\/blog\/wp-content\/uploads\/2013\/06\/cropped-rl13-header-logo.jpg","contentUrl":"https:\/\/www.redline13.com\/blog\/wp-content\/uploads\/2013\/06\/cropped-rl13-header-logo.jpg","width":300,"height":68,"caption":"RedLine13"},"image":{"@id":"https:\/\/www.redline13.com\/blog\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/www.redline13.com\/blog\/#\/schema\/person\/4acbcdcb8a9c72ec5a274e69c0ebea28","name":"RedLine13","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.redline13.com\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/b4c9a289323b21a01c3e940f150eb9b8c542587f1abfd8f0e1cc1ffc5e475514?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/b4c9a289323b21a01c3e940f150eb9b8c542587f1abfd8f0e1cc1ffc5e475514?s=96&d=mm&r=g","caption":"RedLine13"},"sameAs":["http:\/\/127.0.0.1"],"url":"https:\/\/www.redline13.com\/blog\/author\/user\/"}]}},"_links":{"self":[{"href":"https:\/\/www.redline13.com\/blog\/wp-json\/wp\/v2\/posts\/8266","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.redline13.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.redline13.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.redline13.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.redline13.com\/blog\/wp-json\/wp\/v2\/comments?post=8266"}],"version-history":[{"count":0,"href":"https:\/\/www.redline13.com\/blog\/wp-json\/wp\/v2\/posts\/8266\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.redline13.com\/blog\/wp-json\/wp\/v2\/media\/8267"}],"wp:attachment":[{"href":"https:\/\/www.redline13.com\/blog\/wp-json\/wp\/v2\/media?parent=8266"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.redline13.com\/blog\/wp-json\/wp\/v2\/categories?post=8266"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.redline13.com\/blog\/wp-json\/wp\/v2\/tags?post=8266"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}