I have been using JMeter lately for some functional testing of a REST interface. Why JMeter? Well simply because I know how to use it.

However I wanted to run the tests in the JMeter commandline mode and use the exit code as an indication the functional tests failed.

So the first attempt was like this:

1
2
3
4
5
6
7
#!/usr/bin/bash
jmeter -n  -t script.jmx
if [ $? -ne 0 ]; then
    echo '[TEST FAILED]'
else
    echo '[TEST SUCCESSFUL]'
fi

What I found out is that when using JMeter in the command line mode the exit code is always 0 (zero). So after looking for different solutions I decided to parse the output with a python script to extract the number of errors

The output of a JMeter run in command line looks like this

Creating summariser <summary>
Created the tree successfully using script.jmx
Starting the test @ Sat Jun 20 11:58:26 CEST 2015 (1434794306135)
Waiting for possible shutdown message on port 4445
summary =    689 in     2s =  350.3/s Avg:     4 Min:     0 Max:    84 Err:    24 (3.48%)
Tidying up ...    @ Sat Jun 20 11:58:28 CEST 2015 (1434794308267)
... end of run

So the script looks like this now:

1
2
3
4
5
6
7
#!/usr/bin/bash
jmeter -n  -t script.jmx | process_results.py
if [ $? -ne 0 ]; then
    echo '[TEST FAILED]'
else
    echo '[TEST SUCCESSFUL]'
fi

And the source code of the process_results.py:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
#!/usr/bin/python
# script for parsing jmeter result and return 
# exit code 1 if a test has failed
import re
import sys

for line in sys.stdin:
    match = re.search('summary =[\s].*Err:[ ]{0,10}(\d{0,10})[ ].*',line)
    if match :
        sys.exit(1)
    else :
        sys.exit(0)

As you can see there is a fantastic regular expression that I wanted to save for myself in case I might need it in the future.

References