{"id":3527,"date":"2016-10-17T12:45:41","date_gmt":"2016-10-17T16:45:41","guid":{"rendered":"https:\/\/www.redline13.com\/blog\/?p=3527"},"modified":"2016-10-17T12:45:41","modified_gmt":"2016-10-17T16:45:41","slug":"using-amazon-efs-make-load-testing-faster-cheaper","status":"publish","type":"post","link":"https:\/\/www.redline13.com\/blog\/2016\/10\/using-amazon-efs-make-load-testing-faster-cheaper\/","title":{"rendered":"Using Amazon EFS to Make Load Testing Faster & Cheaper"},"content":{"rendered":"

We are always looking for ways to make load testing even cheaper. Many of our frequent users are running larger and longer load tests as they progress through integrating performance testing into the software development lifecycle. \u00a0With that growth, each load agent could potentially generate logs or .jtl files which are multiple gigabytes per load agent.<\/p>\n

As a post-test process, we merge\u00a0JMeter JTL files with the generated Apache JMeter Dashboard Report (similar is done for the Gatling Generated Report). This makes for extremely large size reports and merged files. While the CPU needed for these large tests was not an issue, larger tests were taking up too much storage on the EC2 instances, thus fighting for disk resources and negatively impacting performance.<\/p>\n

Our initial reaction was to dramatically increase the provisioned storage on the server which does the number crunching. However, this data comes and goes, and the costs\u00a0would increase linearly per month.<\/p>\n

Solution: Amazon Elastic File System<\/span><\/h3>\n

Amazon Elastic File System<\/a> is a distributed file system that changes based on consumption and not provisioned storage.\u00a0Amazon EFS can be mounted to an EC2 instance, offering a file system interface and file system access semantics. \u00a0Multiple EC2 instances can access the same Amazon EFS file system at the same time, so data can easily be aggregated and CPU for job processing can be scaled up easily.<\/p>\n

As we complete the analysis and report generation, we can clean up and store results back in S3. This makes EFS a great place to do the work and then remove artifacts. Instead of attaching larger volumes to EC2 and see a linear cost increase, EFS gave us a petabyte scale and a cost structure that will match customer workload.<\/p>\n

So what does this mean for RedLine13? You can keep enjoying cheap load testing at even larger scales.\u00a0Instead of investing more into EC2 hardware or adding fixed storage that would make our costs go up, we were able to implement a scalable solution.<\/p>\n","protected":false},"excerpt":{"rendered":"

We are always looking for ways to make load testing even cheaper. Many of our frequent users are running larger and longer load tests as they progress through integrating performance testing into the software development lifecycle. \u00a0With that growth, each load agent could potentially generate logs or .jtl files which are multiple gigabytes per load agent. As a post-test process, we merge\u00a0JMeter JTL files with the generated Apache JMeter Dashboard Report (similar is done for theRead More →<\/a><\/p>\n","protected":false},"author":1,"featured_media":3511,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[6,2,50],"tags":[72,88,120,121,188,189,209,217,318],"class_list":{"0":"entry","1":"post","2":"publish","3":"author-user","4":"post-3527","6":"format-standard","7":"has-post-thumbnail","8":"category-aws-faq","9":"category-blog","10":"category-updates","11":"post_tag-amazon-web-services","12":"post_tag-aws","13":"post_tag-cheap","14":"post_tag-cheaper","15":"post_tag-efs","16":"post_tag-elastic-file-system","17":"post_tag-faster","18":"post_tag-free","19":"post_tag-load-testing"},"yoast_head":"\nUsing Amazon EFS to Make Load Testing Faster & Cheaper - RedLine13<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.redline13.com\/blog\/2016\/10\/using-amazon-efs-make-load-testing-faster-cheaper\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Using Amazon EFS to Make Load Testing Faster & Cheaper - RedLine13\" \/>\n<meta property=\"og:description\" content=\"We are always looking for ways to make load testing even cheaper. Many of our frequent users are running larger and longer load tests as they progress through integrating performance testing into the software development lifecycle. \u00a0With that growth, each load agent could potentially generate logs or .jtl files which are multiple gigabytes per load agent. As a post-test process, we merge\u00a0JMeter JTL files with the generated Apache JMeter Dashboard Report (similar is done for theRead More →\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.redline13.com\/blog\/2016\/10\/using-amazon-efs-make-load-testing-faster-cheaper\/\" \/>\n<meta property=\"og:site_name\" content=\"RedLine13\" \/>\n<meta property=\"article:published_time\" content=\"2016-10-17T16:45:41+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.redline13.com\/blog\/wp-content\/uploads\/2016\/10\/AWS-Update.png\" \/>\n\t<meta property=\"og:image:width\" content=\"901\" \/>\n\t<meta property=\"og:image:height\" content=\"469\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"RedLine13\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"RedLine13\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"2 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.redline13.com\/blog\/2016\/10\/using-amazon-efs-make-load-testing-faster-cheaper\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.redline13.com\/blog\/2016\/10\/using-amazon-efs-make-load-testing-faster-cheaper\/\"},\"author\":{\"name\":\"RedLine13\",\"@id\":\"https:\/\/www.redline13.com\/blog\/#\/schema\/person\/4acbcdcb8a9c72ec5a274e69c0ebea28\"},\"headline\":\"Using Amazon EFS to Make Load Testing Faster & Cheaper\",\"datePublished\":\"2016-10-17T16:45:41+00:00\",\"dateModified\":\"2016-10-17T16:45:41+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.redline13.com\/blog\/2016\/10\/using-amazon-efs-make-load-testing-faster-cheaper\/\"},\"wordCount\":341,\"publisher\":{\"@id\":\"https:\/\/www.redline13.com\/blog\/#organization\"},\"keywords\":[\"amazon web services\",\"AWS\",\"Cheap\",\"Cheaper\",\"EFS\",\"Elastic File System\",\"Faster\",\"Free\",\"Load Testing\"],\"articleSection\":[\"AWS FAQ\",\"Blog\",\"Updates\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.redline13.com\/blog\/2016\/10\/using-amazon-efs-make-load-testing-faster-cheaper\/\",\"url\":\"https:\/\/www.redline13.com\/blog\/2016\/10\/using-amazon-efs-make-load-testing-faster-cheaper\/\",\"name\":\"Using Amazon EFS to Make Load Testing Faster & Cheaper - RedLine13\",\"isPartOf\":{\"@id\":\"https:\/\/www.redline13.com\/blog\/#website\"},\"datePublished\":\"2016-10-17T16:45:41+00:00\",\"dateModified\":\"2016-10-17T16:45:41+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/www.redline13.com\/blog\/2016\/10\/using-amazon-efs-make-load-testing-faster-cheaper\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.redline13.com\/blog\/2016\/10\/using-amazon-efs-make-load-testing-faster-cheaper\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/www.redline13.com\/blog\/2016\/10\/using-amazon-efs-make-load-testing-faster-cheaper\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/www.redline13.com\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Using Amazon EFS to Make Load Testing Faster & Cheaper\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.redline13.com\/blog\/#website\",\"url\":\"https:\/\/www.redline13.com\/blog\/\",\"name\":\"RedLine13\",\"description\":\"(Almost) Free Load Testing in the Cloud\",\"publisher\":{\"@id\":\"https:\/\/www.redline13.com\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.redline13.com\/blog\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.redline13.com\/blog\/#organization\",\"name\":\"RedLine13\",\"url\":\"https:\/\/www.redline13.com\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.redline13.com\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.redline13.com\/blog\/wp-content\/uploads\/2013\/06\/cropped-rl13-header-logo.jpg\",\"contentUrl\":\"https:\/\/www.redline13.com\/blog\/wp-content\/uploads\/2013\/06\/cropped-rl13-header-logo.jpg\",\"width\":300,\"height\":68,\"caption\":\"RedLine13\"},\"image\":{\"@id\":\"https:\/\/www.redline13.com\/blog\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.redline13.com\/blog\/#\/schema\/person\/4acbcdcb8a9c72ec5a274e69c0ebea28\",\"name\":\"RedLine13\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.redline13.com\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/b4c9a289323b21a01c3e940f150eb9b8c542587f1abfd8f0e1cc1ffc5e475514?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/b4c9a289323b21a01c3e940f150eb9b8c542587f1abfd8f0e1cc1ffc5e475514?s=96&d=mm&r=g\",\"caption\":\"RedLine13\"},\"sameAs\":[\"http:\/\/127.0.0.1\"],\"url\":\"https:\/\/www.redline13.com\/blog\/author\/user\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Using Amazon EFS to Make Load Testing Faster & Cheaper - RedLine13","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.redline13.com\/blog\/2016\/10\/using-amazon-efs-make-load-testing-faster-cheaper\/","og_locale":"en_US","og_type":"article","og_title":"Using Amazon EFS to Make Load Testing Faster & Cheaper - RedLine13","og_description":"We are always looking for ways to make load testing even cheaper. Many of our frequent users are running larger and longer load tests as they progress through integrating performance testing into the software development lifecycle. \u00a0With that growth, each load agent could potentially generate logs or .jtl files which are multiple gigabytes per load agent. As a post-test process, we merge\u00a0JMeter JTL files with the generated Apache JMeter Dashboard Report (similar is done for theRead More →","og_url":"https:\/\/www.redline13.com\/blog\/2016\/10\/using-amazon-efs-make-load-testing-faster-cheaper\/","og_site_name":"RedLine13","article_published_time":"2016-10-17T16:45:41+00:00","og_image":[{"width":901,"height":469,"url":"https:\/\/www.redline13.com\/blog\/wp-content\/uploads\/2016\/10\/AWS-Update.png","type":"image\/png"}],"author":"RedLine13","twitter_card":"summary_large_image","twitter_misc":{"Written by":"RedLine13","Est. reading time":"2 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.redline13.com\/blog\/2016\/10\/using-amazon-efs-make-load-testing-faster-cheaper\/#article","isPartOf":{"@id":"https:\/\/www.redline13.com\/blog\/2016\/10\/using-amazon-efs-make-load-testing-faster-cheaper\/"},"author":{"name":"RedLine13","@id":"https:\/\/www.redline13.com\/blog\/#\/schema\/person\/4acbcdcb8a9c72ec5a274e69c0ebea28"},"headline":"Using Amazon EFS to Make Load Testing Faster & Cheaper","datePublished":"2016-10-17T16:45:41+00:00","dateModified":"2016-10-17T16:45:41+00:00","mainEntityOfPage":{"@id":"https:\/\/www.redline13.com\/blog\/2016\/10\/using-amazon-efs-make-load-testing-faster-cheaper\/"},"wordCount":341,"publisher":{"@id":"https:\/\/www.redline13.com\/blog\/#organization"},"keywords":["amazon web services","AWS","Cheap","Cheaper","EFS","Elastic File System","Faster","Free","Load Testing"],"articleSection":["AWS FAQ","Blog","Updates"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.redline13.com\/blog\/2016\/10\/using-amazon-efs-make-load-testing-faster-cheaper\/","url":"https:\/\/www.redline13.com\/blog\/2016\/10\/using-amazon-efs-make-load-testing-faster-cheaper\/","name":"Using Amazon EFS to Make Load Testing Faster & Cheaper - RedLine13","isPartOf":{"@id":"https:\/\/www.redline13.com\/blog\/#website"},"datePublished":"2016-10-17T16:45:41+00:00","dateModified":"2016-10-17T16:45:41+00:00","breadcrumb":{"@id":"https:\/\/www.redline13.com\/blog\/2016\/10\/using-amazon-efs-make-load-testing-faster-cheaper\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.redline13.com\/blog\/2016\/10\/using-amazon-efs-make-load-testing-faster-cheaper\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/www.redline13.com\/blog\/2016\/10\/using-amazon-efs-make-load-testing-faster-cheaper\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.redline13.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Using Amazon EFS to Make Load Testing Faster & Cheaper"}]},{"@type":"WebSite","@id":"https:\/\/www.redline13.com\/blog\/#website","url":"https:\/\/www.redline13.com\/blog\/","name":"RedLine13","description":"(Almost) Free Load Testing in the Cloud","publisher":{"@id":"https:\/\/www.redline13.com\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.redline13.com\/blog\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.redline13.com\/blog\/#organization","name":"RedLine13","url":"https:\/\/www.redline13.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.redline13.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/www.redline13.com\/blog\/wp-content\/uploads\/2013\/06\/cropped-rl13-header-logo.jpg","contentUrl":"https:\/\/www.redline13.com\/blog\/wp-content\/uploads\/2013\/06\/cropped-rl13-header-logo.jpg","width":300,"height":68,"caption":"RedLine13"},"image":{"@id":"https:\/\/www.redline13.com\/blog\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/www.redline13.com\/blog\/#\/schema\/person\/4acbcdcb8a9c72ec5a274e69c0ebea28","name":"RedLine13","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.redline13.com\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/b4c9a289323b21a01c3e940f150eb9b8c542587f1abfd8f0e1cc1ffc5e475514?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/b4c9a289323b21a01c3e940f150eb9b8c542587f1abfd8f0e1cc1ffc5e475514?s=96&d=mm&r=g","caption":"RedLine13"},"sameAs":["http:\/\/127.0.0.1"],"url":"https:\/\/www.redline13.com\/blog\/author\/user\/"}]}},"_links":{"self":[{"href":"https:\/\/www.redline13.com\/blog\/wp-json\/wp\/v2\/posts\/3527","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.redline13.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.redline13.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.redline13.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.redline13.com\/blog\/wp-json\/wp\/v2\/comments?post=3527"}],"version-history":[{"count":0,"href":"https:\/\/www.redline13.com\/blog\/wp-json\/wp\/v2\/posts\/3527\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.redline13.com\/blog\/wp-json\/wp\/v2\/media\/3511"}],"wp:attachment":[{"href":"https:\/\/www.redline13.com\/blog\/wp-json\/wp\/v2\/media?parent=3527"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.redline13.com\/blog\/wp-json\/wp\/v2\/categories?post=3527"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.redline13.com\/blog\/wp-json\/wp\/v2\/tags?post=3527"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}