You can build this test suite image in OpenShift with Red Hat's Java S2I image or with this fabric8/s2i-java image. I'm going to use the latter, and incorporate it into the BuildConfig for my test suite under "strategy."
When the image is built, you'll get both a packaged JAR in /deployments and your entire repository stored in /tmp/src. Rather than running the JAR file like you normally would with a deployed Java service, you can run mvn test (or mvn clean install --projects etc. if you have a multi-module test suite) inside the container and have the Cucumber results files output there. Jenkins can then grab these results files to be parsed with the Cucumber Reports Plugin and shown all beautifully like this:
Creating the OpenShift Objects
Just like in my JMeter setup, we'll need a BuildConfig, an ImageStream, and a Job. These are saved as YAML files in the test suite repo.
All of these files can also be found here.
It's also super important that, within your tests, you assign Cucumber an output with @CucumberOptions(plugin = {"json:target/cucumber.json" } etc.); (You can make it whatever name you want, as long as ends with .json and you search for the right results file name in your runjob.sh script a few steps down.)
BuildConfig
ImageStream
Job Template
Running the Tests
When we kick off the job pod in OpenShift, it's going to run the script listed under "command." This runjob.sh script runs the tests, organizes the resulting Cucumber JSON files, and then calls back to Jenkins that it's finished running. Jenkins can then copy the folder of Cucumber results files into the agent and run the Cucumber Reports Plugin on that directory.
In our job template, we pass in the URL for the Jenkins input that will be pinged when the tests are complete, your Jenkins username, and your Jenkins API token (get this by clicking on your username, clicking "Configure" then "Show API Token").
The test suite I've been working with is multi-modular, so I'm running mvn clean install --projects Hello --projects World to go through everything, but go with whatever Maven command runs your tests. After this line, all the cucumber.json results files will exist in target directories throughout the repo. The next steps pull every JSON file into a central reports/ directory that Jenkins can grab after you ping back to the pipeline.
Running this from a Jenkins Pipeline
Using the Kubernetes Plugin, these Jenkins agents are spun up inside OpenShift and have the oc client and git installed. Jenkins' role in this is to orchestrate the building of the test suite image and startup of the job inside OpenShift, and then to read the Cucumber results. Here are the steps we take in the pipeline code:
- spin up a Jenkins agent to work from -- node('jenkins-agent')
- login to the OpenShift cluster
- git clone the test suite repo to your Jenkins agent's workspace
- create the ImageStream and BuildConfig in OpenShift and build the test suite image
- kick off the test suite job in OpenShift using the image you just built and get the name of the pod
- in parallel:
- use an input step to wait for tests to finish, then copy the Cucumber reports from the pod into the Jenkins agent workspace and parse them
- follow the logs of the test suite pod
Instead of sending the Cucumber results to Jenkins with a file input, we're grabbing the reports directory right out of the test suite pod with oc rsync ${jobPod}:/tmp/src/reports . -n ${project}. This copies the folder right into the working directory of the Jenkins agent.
I'm running the last two steps in parallel so I can watch the test suite logs as they run, but you can also have the input step run first and wait for the job to finish, then run oc logs -f ${jobPod} -n ${project} at the end to print the logs all at once.
Output
Your pipeline logs will look something like this:
And a link to your Cucumber reports will show up in the sidebar of your build!