Saturday, May 4, 2013

A command-line testing harness for multiple RESTful HTTP requests


Tasked with the creation of a quick-and-dirty REST service, I wanted an easy way to throw in a bunch of test data and verify it with the correct response data and HTTP response codes.  Most testing tools require some infrastructure and editing through their UI; I wanted to throw data (both input and output) into an easy-to-edit text file.  Here's the result.

First, here's the input test data:
POST /entry/ffff/3456
POST /entry/
POST /entry/12
POST /entry/12/qqqq
GET /entry/ffff
GET /entry/fffe
GET /entry/
POST /entry/zz/zyx
POST /entry/az/zyx
POST /entry/34/zyx
POST /entry/AF/zyx
POST /entry/zez/zyx
GET /list/0
GET /list/1
(I put this in the file named "test-input")

Sidebar 1: eyeballing the results

This was actually given to me as the full telnet command, so every entry had "HTTP/1.1" on the end, for example:
POST /entry/ffff/3456 HTTP/1.1
So first iteration was to start my server separately, then run a script that would echo that line and pipe it through "telnet 127.0.0.1 8080".  For example, here are the first 2 tests, located inside my "entries.in" file:


sleep 2
echo POST /entry/ffff/3456 HTTP/1.1      # should return a 200 OK response
echo
sleep 2
echo POST /entry/ HTTP/1.1       # should return 400 BAD REQUEST
echo 
(Interestingly, those comments at the end of each line are acceptable.)


Then I ran the following command:

.  ./entries.in | telnet 127.0.0.1 8080

(Note the "." at the first of the line, meaning to source that file to run the commands inside.)

That works great for me to run all the tests quickly, but I have to visually check the output.

End Sidebar 1

Here is the output expected; of course, only the GET commands result in output.


3456               # GET /entry/ffff
ffff -> 3456    # GET /entry/
ffff -> 3456    # following 4 are generated by GET /entry/
12 -> qqqq
zz -> zyx
az -> zyx
34 -> zyx       # following 3 are generated by GET /list/1
AF -> zyx
zez -> zyx
(That is in the file I named "test-output.body.good", without the # commentary.)


And here are the HTTP response codes expected:

HTTP/1.1 200 OK
HTTP/1.1 400 BAD REQUEST
HTTP/1.1 400 BAD REQUEST
HTTP/1.1 200 OK
HTTP/1.1 200 OK
HTTP/1.1 404 NOT FOUND
HTTP/1.1 200 OK
HTTP/1.1 200 OK
HTTP/1.1 200 OK
HTTP/1.1 200 OK
HTTP/1.1 200 OK
HTTP/1.1 200 OK
HTTP/1.1 200 OK
HTTP/1.1 200 OK
(That is in the file I named "test-output.HTTP-response.good")


Now for the scripting.  Note that this requires Resty, a very convenient wrapper to access RESTful services via curl.

Here are the commands to run those HTTP inputs into my server and check the body of the output.
# run the HTTP server in the background, throwing away any output
python entries.py > /dev/null 2>&1 &
# save the process ID for that server
PID=$!
# wait for the server to fully start
sleep 1
# prepare resty, without verbose output from curl
resty http://127.0.0.1:8080 -s
# use the contents of the test-input file as one command per line
# (via Resty, which has made GET and POST into valid commands)
. ./test-input > test-output.body.out
# check that output against the expected output
diff test-output.body.good test-output.body.out
# stop the HTTP server
kill $PID
# wait for the server to fully stop
sleep 1


And here is how to do the same to check the HTTP response codes of the output.  Note that only the 3 commands after "sleep" are different.

# run the HTTP server in the background, throwing away any output
python entries.py > /dev/null 2>&1 &
# save the process ID for that server
PID=$!
# wait for the server to fully start
sleep 1
# prepare resty, without verbose output and with only the HTTP headers from curl
resty http://127.0.0.1:8080 -s -I
# use the contents of the test-input file as one command per line
# (via Resty, which has made GET and POST into valid commands)
# (Why does it end up in DOS format? Is that curl?!  ... and printing "not found" is dumb.)
. ./test2-input 2>&1 | grep HTTP > test2-output.HTTP-response.out
# check that output against the expected output
# (See the previous command for the reason behind --strip-trailing-cr)
diff --strip-trailing-cr test2-output.HTTP-response.good test2-output.HTTP-response.out 
# stop the HTTP server
kill $PID
# wait for the server to fully stop
sleep 1



That's it!  Now I've got those 3 files (input HTTP commands, output body content, and output HTTP codes) and a script that will run all those inputs and verify the outputs exactly.

Note: as this grows, I've found that I'd prefer to have the input, the output HTTP code, and the output body all next to each other, or maybe on subsequent lines of the same file.  But this fit today's use cases!





No comments: