How Teal Keeps Their API, Tests and Documentation in Sync

There's a common problem in web development of creeping divergence between API documentation and API functionality. Its so easy to update functionality and forget to update the documentation! We can also just accidentally not align the two right from the start. This is solved by using a tool that tests the API and generates documentation at the same time. In the Rails ecosystem, we have the RSWAG gem which almost prevents divergence:

Rswag extends rspec-rails "request specs" with a Swagger-based DSL for describing and testing API operations. You describe your API operations with a succinct, intuitive syntax, and it automatically runs the tests. Once you have green tests, run a rake task to auto-generate corresponding Swagger files and expose them as YAML or JSON endpoints

I said Rswag almost prevents divergence because it does not enforce a few key items that ensure divergence is eradicated. This blog post shows what is really necessary to ensure the problem is solved, and also offers a few patterns to help with keeping the code DRY and clean.

Part 1: What RSwag gives you out of the box, and the problems that still exist

Part 2: A simple solve using Ruby JSON Schema Validator and the problems that still exist

Part 3: A robust solve that uses Json Schema Builder along with some patterns
‍
‍

Part 1: What RSwag gives you out of the box, and the problems that still exist

‍

Let's assume RSWAG installed and you have a basic Article model, associated route and controller as follows:

‍

Let's also assume you have a basic RSwag test set up for the above endpoint:

‍

When we run the test, it passes, and per the RSwag promise we can generate some Swagger documentation with rails rswag, start up our server and navigate to localhost:3000/api-docs to see the docs.
‍

The Problem

‍

But what is actually getting tested here? The response status code, given the params. That's all.

Given the goal of preventing creeping divergence, this is a massive issue: There is no coupling that ensures that the request schema is what is actually being used in the test! (and similarly, there is nothing testing that the response object matches the response schema)

If you look at the code above, you can see that the request object defined by let(:params) could be different than the schema defined in the request specification. The tests will still pass. This completely fails the promise of coupling between tests and documentation.

We want some sort of validation that the test's object passed to the controller and the request schema match.

We want something like:

‍
‍

Part 2: A simple solve using Ruby JSON Schema Validator and the problems that still exist

‍

We can achieve this with the Ruby JSON Schema Validator gem, which offers a simple way to check if a JSON blob adheres to a JSON schema.

With the gem installed we can re-write the actual tests:

‍

Now if we decided to add a parameter to the test's request object, but forget to change the request schema, we get a failed test! This means there can never be a divergence between the API functionality, the test suite and the docs! For example if we change the params being sent to the controller to include the param extra, then the test will throw:
‍

Are we done? No! There are still a few problems remaining which really reveal themselves as you try to apply the above to an entire application. The first problem is that with the method above, we will potentially start to have duplicates of schemas. This might happen as we build out various update or index endpoints. Furthermore, if we create an Author class, and an author has multiple articles, does that mean when we test our author GET show endpoint (which contains an array of articles), we need to copy/paste the article schema into the author schema?? Yikes. That doesn't feel very DRY. We are getting into the world of building and composing schemas. That means it's time for some more help.
‍
‍

Part 3: A robust solve that uses Json Schema Builder along with some patterns

‍

Enter the Json Schema Builder. With this gem, we are going to do a few things. The first is that we are going to consolidate our schemas into a specs/schemas directory. While we lose the visual convenience of the schemas and tests being in the same file, placing schemas into a common directory enables easier schema composition. The second change is that since this gem comes with its own validation feature, we can stop using (and remove) the aforementioned Ruby JSON Schema Validator gem.

We are going to add some lines to the top of the swagger_helper.rb file so we know where to find our awesome schema definitions:

With this in place we can create our schemas:

A shoutout here to the schema builder gem. It's DSL is fantastic and allows for very readable schemas.

We can now update our integration test to use these brand new schemas:

This is starting to look better but we still have a bit more to do. Right now there is a lot of duplication that we will see once we add more integration tests. What we really want to do is wrap everything up in a reusable, shared test that we can add to various endpoints.. The three tests above (checking for request structure, response structure and status code) should be included in every endpoint test, so let's move them to a shared test.

First let's make sure that we required our shared examples. In your swagger_helper.rb file add the following line:

Then you can define a shared example:

And finally in your spec, you can utilize the shared example::

‍

Now when you make another endpoint, your API functionality and documentation will remain in sync with just a few lines of code! What will you do with all of your reclaimed spare time? I hear a lot of people are learning to bake sourdough bread these days...

‍