We recently discovered some major issues with the TMForum Conformance Test Kit (CTK) v5.0.0 and will demonstrate how using Specmatic can address these problems.
Contract Driven Development
Specmatic’s API coverage report helps identify any mismatches between an OpenAPI specification and an application’s endpoints early in the development lifecycle. The report lists all the endpoints, their coverage, and indicates if they are missing in the spec or not implemented in the application. You can define success criteria for the API coverage report and fix issues.
With Specmatic JDBC stub, you can easily test APIs without the need for a complex database setup. By switching out the real database with a Specmatic JDBC stub, we can set expectations for the stub to respond to specific queries. These expectations can be recorded based on actual interactions. Once the expectations are set, we no longer need the database for our tests!
With Spring Cloud Contract we can start by authoring the API contract in one of the supported DSLs, based on which the provider / producer is verified and in the process stubs are generated for consumers. This makes it possible to leverage it as both a “consumer driven” or “provider driven” contract testing tool. There are also some interesting points about how we share contracts and stubs (through artifact managers, source control, etc.) with Spring Cloud Contract. Let us look at the detailed comparison between Specmatic and Spring Cloud Contract. Please do leave a comment if there are other areas of comparison on which you would like to see additional details.
API mocking is only effective if the mocks are truly representative of the provider / services they are emulating. Deviations between mocks and providers can lead to integration issues much later in the development cycle. In order to avoid such issues it is important to keep in mind that selecting an API mocking tool is not an isolated decision. It has to be in line with your overall microservices development, testing and deployment goals.
Specmatic and Contract Driven Development differ significantly from Pact. We built Specmatic out of our own necessity where Pact did not fit the bill for us. We have tried to cover as many areas of comparison as possible, however we would love to hear if you have any other criteria on which you would like to see additional details.