Level 7: Contract Testing and Spring Batch mumblemumble

CONTRACT TESTING!

The preamble

Contract testing is one of the neater things that I’ve come across while working within a microservices architecture. According to the age-old adage, we should try to have as bullet-proof as possible code and the structure in place to support it.

I’m sure if you’ve been in the tech industry anytime in the past decade, you’ve most likely seen this propaganda pyramid. It seems like that’s all we ever talk about when it comes to the agile manifesto and good testing practices. Not that we’re wrong, but it takes a lot of work and foresight to get to a good place in terms of testing. What does any of this have to do with contract testing? A lot, apparently.

General breakdown of tests, if you don’t know by now

Unit tests are all about breaking down the individual logic and testing them in isolation at the lowest level of your code. Does your sum(param1, param2) method actually return param1 + param2? How does it handle edge cases, like either parameter being negative, infinity, or undefined?

Component tests group several of these units of code together and test to see if they all play nicely. No mocking of the internal code, but some mocking may be required if there are interactions with third-party code, databases, http endpoints, etc.

Integration tests will then start to add in the dbs, endpoints, and any other yucky stuff we didn’t want to deal with before.

API tests verify that the API service agreement is followed by both the producer and the various consumers of the service endpoint. wink wink contract testing sure comes in handy here.

GUI means time to use your BDD tools or get your hands dirty and do some manual exploratory testing.

Quick comparison of available libraries

Contract testing, according to the agile testing pyramid, is considered an integration/API test when using a microservices-oriented architecture. Writing this test involves interacting with third-party code (another microservice) and also verifies both the producer and consumer sides of the REST API.

There are two libraries/frameworks that I looked into that achieve my goal of contract testing: Spring Cloud Contract and Pact.

There are a few differences between the two approaches, but the gist is that Pact is polyglot and has a much bigger open-source following. The heart of the contract testing mechanism is very similar but they require an extra hosted server to maintain the various contracts for all the tested microservices in your desired system. However, Spring offers their own Cloud Contract product. Also free, for Kotlin, Java and intended for spring applications and instead makes use of a nexus repository to host the pacts/stubs. This difference marginally lowers the overhead. I’ve had the pleasure to communicate and get troubleshooting advice from the current maintainer of the Cloud Contract product, Marcin Grzejszczak. He’s very on top of the documentation and provides great community support.

By all means, learn more about contract testing and the internals through the source code and documentation.

Anyway, on to the actual testing portion.

The dirty

To be perfectly honest, it took me about two or so days of reading the documentation and trying out the tutorials to get a below average understanding of the whole mechanism. For me, it felt like a chicken-and-egg scenario with regards to who first creates/fulfills the contract. The biggest hurdle, in my opinion, is the confusing/inconsistent use of terminology. In the docs, often they would use the producer/consumer dichotomy and then switch to server/client, source/target. I ended up having to draw a simple mapping chart just to make sure I understood who was who and what was what, especially when applying it to my own business case. Here’s what I found.

Step #1

We start with the consumer. The consumer is the application that, you guessed it, consumes the data retrieved from the REST endpoint. What are they consuming, exactly? What is their input/output? Well, the consumer should know almost everything by the time they have to begin coding the story. They know what kind of input they can provide and in what format (generally their own POJO definition). They also know what kind of output they’re expecting to process whatever they need to process. They may not know the exact form, but they know what data. Armed with this knowledge, the consumer can begin writing, TDD-style, a contract test with the expectation that the I/O is as expected.

My specific example is used in conjunction with Spring Batch, which limited my ability to run the suite with the @SpringBootTest annotation. Without it, all the autowiriing is manual and actually ended up causing quite a few headaches. Marcin was unable to help past a certain point, as I guess they didn’t test much with having a joint Spring Batch + Microservices architecture.

@TestExecutionListeners({DependencyInjectionTestExecutionListener.class,
        StepScopeTestExecutionListener.class})
@RunWith(SpringRunner.class)
@Import(NoOpContractVerifierAutoConfiguration.class) // Required to disable some error messages
@AutoConfigureStubRunner(workOffline = true, ids = {"<group-id>:<artifact-id>:+:stubs:<desired-port-number>"})
public class FunServiceContractTest {

    @Autowired
    private FunProcessor funProcessor;

    @Test
    public void regular_fun_update() {
        HistoricalFun historicalFun = new HistoricalFun();
        List<FunStuff> expectedFunStuff = funProcessor.process(Arrays.asList(historicalFun));
        // The reader is initialized and bound to the input data
        assertNotNull(expectedFunStuff);
        // other assertions
    }

There’s a lot of automatic happening at this point, but suffice it to say that the @TestExecutionListeners sections is to disable certain Spring Batch functionality to allow for isolated testing. @Import(NoOpContractVerifierAutoConfiguration.class) stops the following error from being thrown.

Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'contractVerifierMessageExchange' defined in class path resource [org/springframework/cloud/contract/verifier/messaging/stream/ContractVerifierStreamAutoConfiguration.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.cloud.contract.verifier.messaging.MessageVerifier]: Factory method 'contractVerifierMessageExchange' threw exception; nested exception is org.springframework.beans.factory.NoSuchBeanDefinitionException: No qualifying bean of type 'org.springframework.cloud.stream.test.binder.MessageCollector' available

More Spring Cloud Contract magic, no doubt. The @AutoConfigureStubRunner is much more obvious in that it simply asks for where your stubs are and what they’re called, and they’ll run when required for the tests. The initial code will have the workOffline flag set to true, and that’s because we’re still in the initial phase of setting up our contract test. No need to deploy code to the nexus repository until we’re more certain about the whole structure of our test. When we run the tests offline, the consumer will expect the stubs jar to be available in the local ~/.m2/repository.

If you try to run the workOffline flag set to false but have the stubs in your local repository, you’ll have to delete it as an error will be thrown.

Alright, step 1 complete.

Step #2

Let’s move on to the producer. I believe the consumer would create a groovy DSL representation of the request/response expected and then pass this along. The producer has to create a base test class as well as include some dependencies specified by the Spring Cloud Contract introduction. Here is an example of an abstract BaseContractTest class which only has a setup method.

public abstract class BaseContractTest {
    @Before
    public void setup() {
        RestAssuredMockMvc.standaloneSetup(new FunProvidingController());
    }
}

The following abstract class is really important and is what is used to run against the contracts.

@RunWith(SpringRunner.class)
@SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.MOCK)
public abstract class FunBase {

    @MockBean
    FunMapper mapper;
    @MockBean
    OtherFunService otherFunService;
    @Autowired
    private FunProvidingService funProvidingService;
    @Autowired
    private FunProvidingController funProvidingController;

    @Before
    public void setup() {
        // any required setup or mocks
        RestAssuredMockMvc.standaloneSetup(funProvidingController);
    }

If you so choose, it’s possible to have a fully-fledged test with database and feign calls. In order to keep the complexity (and headache) to a minimum, I chose to mock out any other microservices I require in my process OtherFunService and any database mapping, in this case, the MyBatis FunMapper. The goal of contract testing is to prove that you can actually deliver on the promise defined in the contract. As long as you have tested the database access layer thoroughly and you have good retry scenarios for your services through Feign/RestTemplate and other Contract tests, I believe this setup will suffice for most cases.

Following the setup, the producer then loads in the file under the required folder src/test/resources/contract/ and then they can run a mvn clean install and generate the tests from the contract. If you go under target/generated-test-sources/contracts/<group-id-here>/*test.java, you’ll find a generated test. Run this, and it should indicate whether or not you meet the required contract. The test will be run automatically when you install or package or test, but if you ever need more help debugging, the generated test source file can come in handy.

Step #3

Once everything is honky dorey on the producer side, we’ll go ahead and update the workOffline = true flag to instead be the repositoryRoot = <nexus_URL_here>. I added snapshots because that’s where my stubs were being deployed to. An important note is that, if your nexus requires credentials, you’ll have to add some metadata to your src/test/resources/application.* file with those in order for this next section to work.

@AutoConfigureStubRunner(repositoryRoot = "<nexus_URL_here>/snapshots", ids = {"<group-id>:<artifact-id>:+:stubs:<desired-port-number>"})

My src/test/resources/application.properties file:

stubrunner.username=dabes
stubrunner.password=dabes456

And voila, you have yourself a contract test.

Didn’t think you’d get here, did you? I sure didn’t

This stuff is by no means simple. It takes some mind-bending and patience to really ensure that you have a good understanding of the topic. Especially with so much configuration and back-and-forth between the producer and the consumer projects, it is easy to get confused. I was putting maven dependencies all over the place because the documentation was poor and I was feeling dumb that day. Take your time and read the docs and maybe even read this article again. It may help. But then again, it may not.

Good luck with contract testing and with Spring Batch!