There's a common problem in web development of creeping divergence between API documentation and API functionality. Its so easy to update functionality and forget to update the documentation! We can also just accidentally not align the two right from the start. This is solved by using a tool that tests the API and generates documentation at the same time. In the Rails ecosystem, we have the RSWAG gem which almost prevents divergence:
Rswag extends rspec-rails "request specs" with a Swagger-based DSL for describing and testing API operations. You describe your API operations with a succinct, intuitive syntax, and it automatically runs the tests. Once you have green tests, run a rake task to auto-generate corresponding Swagger files and expose them as YAML or JSON endpoints
I said Rswag almost prevents divergence because it does not enforce a few key items that ensure divergence is eradicated. This blog post shows what is really necessary to ensure the problem is solved, and also offers a few patterns to help with keeping the code DRY and clean.
Part 1: What RSwag gives you out of the box, and the problems that still exist
Part 2: A simple solve using Ruby JSON Schema Validator and the problems that still exist
Part 3: A robust solve that uses Json Schema Builder along with some patterns
Part 1: What RSwag gives you out of the box, and the problems that still exist
Let's assume RSWAG installed and you have a basic Article model, associated route and controller as follows:
# app/models/article.rb
class Article < ApplicationRecord
end
# config/routes.rb
Rails.application.routes.draw do
mount Rswag::Api::Engine => '/api-docs' # added by Rswag
mount Rswag::Ui::Engine => '/api-docs' # added by Rswag
resources :articles, only: :create
end
# app/controllers/articles_controller.rb
class ArticlesController < ApplicationController
def create
create_params = params.require(:article).permit(:title, :body)
article = Article.create!(create_params)
render json: article, status: :created
end
end
Let's also assume you have a basic RSwag test set up for the above endpoint:
# specs/integration/articles_spec.rb
RSpec.describe "Articles", type: :request, swagger_doc: "v1/swagger.json" do
path "/articles" do
post "creates an article" do
tags "Article"
consumes "application/json"
produces "application/json"
parameter name: :params, in: :body, schema: {
type: :object,
properties: {
article: {
type: :object,
properties: { title: { type: :string }, body: { type: :string } },
required: [ 'title', 'body' ]
}
}
}
response '201', 'article created' do
let(:params) {
{ article: { title: 'foo', content: 'bar' } }
}
schema type: :object,
properties: {
article: {
type: :object,
properties: { id: {type: :integer }, title: { type: :string }, body: { type: :string } },
required: [ 'id', 'title', 'body' ]
}
}
run_test!
end
end
end
end
When we run the test, it passes, and per the RSwag promise we can generate some Swagger documentation with rails rswag, start up our server and navigate to localhost:3000/api-docs to see the docs.
The Problem
But what is actually getting tested here? The response status code, given the params. That's all.
Given the goal of preventing creeping divergence, this is a massive issue: There is no coupling that ensures that the request schema is what is actually being used in the test! (and similarly, there is nothing testing that the response object matches the response schema)
If you look at the code above, you can see that the request object defined by let(:params) could be different than the schema defined in the request specification. The tests will still pass. This completely fails the promise of coupling between tests and documentation.
We want some sort of validation that the test's object passed to the controller and the request schema match.
We want something like:
# pseudo code:
it "matches the request object to the documented request schema" do
expect(request_object).to match_schema(request_schema)
end
Part 2: A simple solve using Ruby JSON Schema Validator and the problems that still exist
We can achieve this with the Ruby JSON Schema Validator gem, which offers a simple way to check if a JSON blob adheres to a JSON schema.
With the gem installed we can re-write the actual tests:
# specs/integration/articles_spec.rb
require "swagger_helper"
RSpec.describe "Articles", type: :request, swagger_doc: "v1/swagger.json" do
path "/articles" do
post "creates an article" do
tags "Article"
consumes "application/json"
produces "application/json"
# notice we pull out the request and response schemas.
expected_request_schema = {
type: :object,
properties: {
article: {
type: :object,
properties: { title: { type: :string }, body: { type: :string } },
required: [ 'title', 'body' ]
}
}
}
expected_response_schema = {
type: :object,
properties: {
id: { type: :integer },
title: { type: :string },
body: { type: :string }
},
required: [ 'id', 'title', 'body' ]
}
parameter name: :params, in: :body, schema: expected_request_schema
response '201', 'article created' do
let(:params) {
{ article: { title: 'foo', body: 'bar' } }
}
before do |example|
submit_request(example.metadata)
end
schema expected_response_schema
it "matches the documented request schema" do |example|
JSON::Validator.validate!(expected_request_schema, params, strict: true)
end
it "matches the documented response schema" do |example|
json_response = JSON.parse(response.body)
JSON::Validator.validate!(expected_response_schema, json_response, strict: true)
end
it 'returns a valid 201 response' do |example|
expect(response.status).to eq(201)
end
end
end
end
end
Now if we decided to add a parameter to the test's request object, but forget to change the request schema, we get a failed test! This means there can never be a divergence between the API functionality, the test suite and the docs! For example if we change the params being sent to the controller to include the param extra, then the test will throw:
JSON::Schema::ValidationError:
The property '#/article' contained undefined properties: 'extra'
Are we done? No! There are still a few problems remaining which really reveal themselves as you try to apply the above to an entire application. The first problem is that with the method above, we will potentially start to have duplicates of schemas. This might happen as we build out various update or index endpoints. Furthermore, if we create an Author class, and an author has multiple articles, does that mean when we test our author GET show endpoint (which contains an array of articles), we need to copy/paste the article schema into the author schema?? Yikes. That doesn't feel very DRY. We are getting into the world of building and composing schemas. That means it's time for some more help.
Part 3: A robust solve that uses Json Schema Builder along with some patterns
Enter the Json Schema Builder. With this gem, we are going to do a few things. The first is that we are going to consolidate our schemas into a specs/schemas directory. While we lose the visual convenience of the schemas and tests being in the same file, placing schemas into a common directory enables easier schema composition. The second change is that since this gem comes with its own validation feature, we can stop using (and remove) the aformentioned Ruby JSON Schema Validator gem.
We are going to add some lines to the top of the swagger_helper.rb file so we know where to find our awesome schema definitions:
# specs/swagger_helper.rb
require 'rails_helper'
require "json/schema_builder"
# this block ensures if any schema element is extra or missing, the test fails.
JSON::SchemaBuilder.configure do |opts|
opts.validate_schema = true
opts.strict = true
end
# let spec know where your shiny new schemas are!
Dir["./spec/schemas/*.rb"].each { |file| require file }
RSpec.configure do |config|
# ...
With this in place we can create our schemas:
# spec/schemas/article_schemas.rb
module SpecSchemas
class ArticleCreateRequest
include JSON::SchemaBuilder
def schema
object do
object :article do
string :title, required: true
string :body, required: true
end
end
end
end
class ArticleResponse
include JSON::SchemaBuilder
def schema
object do
number :id, required: true
string :title, required: true
string :body, required: true
end
end
end
end
A shoutout here to the schema builder gem. It's DSL is fantastic and allows for very readable schemas.
We can now update our integration test to use these brand new schemas:
require "swagger_helper"
RSpec.describe "Articles", type: :request, swagger_doc: "v1/swagger.json" do
path "/articles" do
post "creates an article" do
tags "Article"
consumes "application/json"
produces "application/json"
expected_request_schema = SpecSchemas::ArticleCreateRequest.new
parameter name: :params, in: :body, schema: expected_request_schema.schema.as_json
response '201', 'article created' do
expected_response_schema = SpecSchemas::ArticleResponse.new
let(:params) {
{ article: { title: 'foo', body: 'bar' } }
}
before do |example|
submit_request(example.metadata)
end
schema(expected_response_schema.schema.as_json)
it "matches the documented request schema" do |example|
errors = expected_request_schema.schema.fully_validate(params)
expect(errors).to be_empty
end
it "matches the documented response schema" do |example|
json_response = JSON.parse(response.body)
errors = expected_response_schema.schema.fully_validate(json_response)
expect(errors).to be_empty
end
it 'returns a valid 201 response' do |example|
expect(response.status).to eq(201)
end
end
end
end
end
This is starting to look better but we still have a bit more to do. Right now there is a lot of duplication that we will see once we add more integration tests. What we really want to do is wrap everything up in a reusable, shared test that we can add to various endpoints.. The three tests above (checking for request structure, response structure and status code) should be included in every endpoint test, so let's move them to a shared test.
First let's make sure that we required our shared examples. In your swagger_helper.rb file add the following line:
# /specs/swagger_helper.rb
Dir["./spec/support/**/*.rb"].sort.each { |file| require file }
Then you can define a shared example:
# spec/support/integration/shared_examples.rb
RSpec.shared_examples "a JSON endpoint" do |expected_response_status|
before do |example|
submit_request(example.metadata)
end
describe "response status" do
it "returns expected response status" do
expect(response.status).to eq(expected_response_status)
end
end
describe "request body" do
it "matches the documented request schema" do |example|
errors = expected_request_schema.schema.fully_validate(params)
puts params if errors.count > 0 # for debugging
expect(errors).to be_empty
end
end
describe "response body" do
let(:json_response) { JSON.parse(response.body) }
it "matches the documented response schema" do |example|
errors = expected_response_schema.schema.fully_validate(json_response)
puts json_response if errors.count > 0 # for debugging
expect(errors).to be_empty
end
end
end
And finally in your spec, you can utilize the shared example::
# spec/integration/articles_spec.rb
require "swagger_helper"
RSpec.describe "Articles", type: :request, swagger_doc: "v1/swagger.json" do
path "/articles" do
post "creates an article" do
tags "Article"
consumes "application/json"
produces "application/json"
expected_request_schema = SpecSchemas::ArticleCreateRequest.new
parameter name: :params, in: :body, schema: expected_request_schema.schema.as_json
response '201', 'article created' do
expected_response_schema = SpecSchemas::ArticleResponse.new
schema(expected_response_schema.schema.as_json)
let(:params) {
{ article: { title: 'foo', body: 'bar' } }
}
before do |example|
submit_request(example.metadata)
end
it_behaves_like "a JSON endpoint", 201 do
let(:expected_response_schema) { expected_response_schema }
let(:expected_request_schema) { expected_request_schema }
end
end
end
end
end
Now when you make another endpoint, your API functionality and documentation will remain in sync with just a few lines of code! What will you do with all of your reclaimed spare time? I hear a lot of people are learning to bake sourdough bread these days...