info@pragmatictesters.com +94 11 253 8512

Pragmatic Test Labs

Removing outliers from test results at runtime. A missing plugin ?

You may have noticed that the test results generated during the start and end of the JMeter test have a negative impact on the overall test metrics, especially when the test duration is shorter. Usually the response times are higher during the ramp ups and get errors at the end when the tests are stopped.

Hence there is a need for removing the test results during the ramp-up and ramp-down period.

We have few plugins available for filtering the test results.

There is no way to exclude test results from all listeners out of the box at the moment.

Hence following custom implementation could be useful for removing the test results during ramp-up and ramp-down phases.

In this post we will discuss a solution with JSR223 Post Processor

Approach

Post processors are executed after the samplers. You can use a JSR223 Post Processor to run the scripts for removing the outliers. The JSR223 Post Processor can be placed either at the Test Plan level or at the Thread Group level to ensure it is applied to all samplers.

We can get the test start time in milliseconds from the predefined JMeter variable/property TESTSTART.MS

long startTimeInMillis=vars.get("TESTSTART.MS").toLong()

Following statements could be used for getting the current test duration

long currentTimeInMillis = new Date().getTime()

long currentTestDurationInMillis=currentTimeInMillis-startTimeInMillis

Ignoring the test results at runtime is easy with the prev.setIgnore() after checking the current test duration with the offset values.

Test Script

Following is the complete script for ignoring the test results during the ramping times.

if(args[0].toBoolean()){
	int testResultToIgnoreBeforeInMin=args[1].toInteger()
	int testResultToIgnoreAfterInMin=args[2].toInteger()
	
	long testResultToIgnoreBeforeInMillis=testResultToIgnoreBeforeInMin*60*1000
	long testResultToIgnoreAfterInMillis=testResultToIgnoreAfterInMin*60*1000
	
	long startTimeInMillis=vars.get("TESTSTART.MS").toLong()
	long currentTimeInMillis = new Date().getTime()
	long currentTestDurationInMillis=currentTimeInMillis-startTimeInMillis
	
	log.info("currentTestDurationInMillis ${currentTestDurationInMillis}")
	
	
	if(currentTestDurationInMillis< testResultToIgnoreBeforeInMillis || currentTestDurationInMillis >testResultToIgnoreAfterInMillis){
		prev.setIgnore()
		log.info("Test result is ignored")
	}
}

Configuration

Start and end offset times can be defined in seconds as parameters. Optionally the parameters can be defined in user.properties

#Set true if you want to enable the script

ignore_ramping_time_results=true

#Set the ramping up time in seconds

start_offset=10

#Set the ramp down start time in seconds

end_offset=60

How to use in Test Plans (JMX)

Option 1

You can save the JSR223 post processor with the script and parameters as a Test Fragment and merge with new JMeter test plans. Drag and drop the JSR223 Post Processor to the Test Plan level or Thread Group level.

Option 2

You can include the test fragment into the JMeter template and merge with JMeter test plans.

Select the JSR223 Post processor with the script
Save the selection as a JMX (file into the JMETER_HOME/bin/templates folder

JSR223PostProcessorIgnoreTestResults.jmx

Open JMETER_HOME/bin/templates/templates.xml file in a text editor
Include the template and save the file

<template isTestPlan="false">
   <name>JSR223 Sampler - Ignore Test Result</name>
   <fileName>JSR223PostProcessorIgnoreTestResults.jmx</fileName>
   <description><![CDATA[
   <h1>Ignore Test Results</h1>
   <h2>Overview</h2>
   This is a sample JSR223 sampler removes the test results during ramping-up and ramping down times
   <br/>
   ]]></description>
</template>

Open JMeter and click the template icon. Template popup will be launched
Select the template name from the dropdown and click merge to add the template

Usage

This component can be used with

JMeter GUI mode
JMeter CLI mode
JMeter on cloud such as RedLine13

About Pragmatic Test Labs

Pragmatic Test Labs is a software testing company based in Sri Lanka. We provide software testing services including performance testing and JMeter training.
Please contact janesh@pragmatictesters.com for more information