{"id":8388,"date":"2021-10-22T05:01:54","date_gmt":"2021-10-22T09:01:54","guid":{"rendered":"https:\/\/www.redline13.com\/blog\/?p=8388"},"modified":"2022-05-23T13:07:01","modified_gmt":"2022-05-23T17:07:01","slug":"merging-very-large-jtl-files","status":"publish","type":"post","link":"https:\/\/www.redline13.com\/blog\/2021\/10\/merging-very-large-jtl-files\/","title":{"rendered":"Merging Very Large JTL Files"},"content":{"rendered":"

\"Merging<\/strong><\/p>\n

RedLine13 has made changes to handle the cases where it is necessary for merging very large JTL files<\/a>, which are output by JMeter<\/a>.\u00a0 RedLine13 has for years been able to handle large JTL files needed to support high volume load tests with no problem.\u00a0 Now RedLine13 can handle JTL files of 70GB and greater.\u00a0 We\u2019ve included details on why we made the change for those that want to understand more.<\/p>\n

A customer was running tests which generated JTL files in excess of 70GB across multiple servers.\u00a0 To give you some idea of the size of their test, it ran for over 2 hours, generated over 4.8 million requests at an average rate of more than 55,000 requests per second.\u00a0 The JTL files that were generated were as large as 70GB.\u00a0 While our system handled the process of collecting and merging the JTL files, there were some external factors which caused issues.\u00a0 Here we will explain what the nature of the problem was, how we solved it, and ultimately what we learned to make RedLine13 even better.<\/p>\n

Issues that we overcame:<\/h4>\n

The primary issues were as follows:<\/p>\n

    \n
  1. We were unable to write the merged JTL file back to S3<\/a> because of the massive file size.\u00a0 This is due to the limitations of the S3 \u201cputObject<\/span>\u201d method which only permits files up to about 5GB to be uploaded.\n