Chapter 10 of 10
What to Look for in an API Governance Tool
Eight criteria that decide whether a governance tool is still running in a year, written as questions to ask rather than as a list of vendors.
Governance tools are easy to compare on the wrong things. This chapter names the criteria that still matter after the trial, written as questions you can put to any tool including the one you already run.
The criteria, and why each one is on the list
1. Where do the checks run
A finding that appears where the decision is being made is a correction. The same finding in a report is a task for somebody in two weeks.
Ask whether violations appear while a description is being edited, whether they can be fixed from where they appear, and whether the same rules run again at the release. Anything that only runs on a schedule will be read for a month and ignored afterwards.
2. Can you express your own conventions
Every organisation has conventions no tool ships with, and this is where tools differ most.
Ask what a rule of your own can target, what it can match, and whether it is declarative or a script. A declarative rule is readable by the next person, while a scripted one is a small program somebody has to maintain. Both are legitimate. The failure case is a tool where the shipped rules are all you get.
3. Is enforcement optional per rule
A tool where everything blocks and a tool where nothing blocks fail in opposite directions and at the same speed.
Ask whether severity is yours to set, whether blocking is a separate switch from the rules themselves, and whether a warning can still be published after a confirmation. If enforcement is all or nothing, your rollout has no middle step. That step is what makes the whole thing survivable.
4. How is an exception recorded
Every standard meets an API it cannot fit, so the handling of that case is a core feature rather than an edge case.
Ask whether an exception can be scoped to one project or one response rather than switching a rule off everywhere. Ask whether the exception shows its origin, so somebody reading the settings can tell an inherited value from a local decision. A tool with no escape hatch produces teams that stop running it.
5. Does the standard cascade
One standard with local departures stays one standard, while two standards drift apart within a year.
Ask whether an organisation-level default exists, whether a project can override it, and whether the override is visible as an override. The same question applies to shared components and to header policies, since a cascade that covers rules but nothing else leaves the copies where they were.
6. Can it remove violations instead of reporting them
This is the criterion most comparisons leave out. It is also the one with the largest effect.
Ask whether the tool can hold a definition once and have several APIs reference it, and whether it can apply a set of response headers rather than checking that somebody added them. Inspection scales with the number of copies, so a tool that can only inspect will generate work forever.
7. What does it measure, and against what
A number is useful for choosing where to look, provided you can see what it is made of.
Ask whether the weighting is visible and adjustable, whether you can see a specification's own history rather than only its current value, and whether thresholds produce an alert rather than a leaderboard. A score you cannot decompose is a grade. A grade invites exactly the behaviour the measurement chapter warns about.
8. What can you take with you
Governance configuration is a real asset once it has been calibrated against your APIs.
Ask how the rules and their severities can be read out, whether by export or through an API, and what happens to your custom rules if you move. The answer will not usually decide between two tools. It does tell you how much of the calibration work you would be repeating.
| Criterion | The question | The answer that should worry you |
|---|---|---|
| Where checks run | Do violations appear while editing | A scheduled report |
| Own conventions | What can a rule target and match | A fixed rule set with no extension |
| Enforcement | Is blocking optional per rule | All or nothing |
| Exceptions | Can one project depart from one rule | Switch it off for everybody |
| Cascade | Is there an org default with visible overrides | One flat configuration |
| Removing violations | Can a definition be shared rather than checked | Inspection only |
| Measurement | Is the weighting visible and adjustable | One opaque number |
| Portability | How do the rules come back out | No way to read the configuration |
How to actually evaluate
Do not evaluate on a tidy API, because the first run over a tidy API tells you nothing you did not know.
Load the least consistent real API you have and read the whole first run. Sort it into what you want fixed, what you want noted and what you would switch off. That is the same calibration exercise the rollout chapter describes, and you have to do it anyway.
Then take one convention your organisation genuinely has and try to express it as a rule. Then take the API that legitimately cannot satisfy one of your rules and record the exception. Those three exercises take an afternoon and they cover the three things that decide whether the tool is still running next year.
In Routebase
Routebase answers these criteria in a specific way, and the previous chapters describe each answer in full.
Rules run continuously against the specification you are editing and again in the publish dialog, with an issues panel that offers a fix for many findings. Severity is set per rule at the organisation and overridable per project, with a source column that shows where the current value came from. Blocking on publish is a separate switch that starts off. Custom rules extend the set with a target, a field and a condition. The shared library and header policies remove classes of violation rather than reporting them, and both cascade from the organisation down. The quality score exposes its weights and requires them to total one hundred, and its thresholds raise alerts rather than rankings.
Two limits are worth knowing before you plan around them. The configuration lives in the organisation rather than as a file in your repository, so rules are read and written through the settings pages or the API rather than reviewed as a pull request. There is also no import or export of rule sets from other linters, so a ruleset you maintain elsewhere is expressed again rather than loaded.
The Style Guide, Shared Library and Header Policies guides cover the three surfaces in the product documentation. The comparison with Stoplight covers where this sits when linting is one stage of the lifecycle rather than the product. The API governance page is the short version of these answers, with the plan each surface sits on.
Frequently asked questions
What should you look for in an API governance tool?
Ask where the checks run, whether you can express your own conventions, whether enforcement is optional per rule, and how an exception is recorded. Those four decide whether a team keeps the tool switched on. Rule counts and dashboards demo well and tell you almost nothing about the year after the trial.
Do you need a dedicated API governance tool?
Not for one team and one API, where a linter in the pipeline covers it. The case starts when several teams ship APIs that consumers see as one product. You then need a standard set in one place, exceptions that are visible, and a view of where the standard does not hold yet. Those are coordination problems rather than linting problems.
How do you evaluate an API governance tool?
Load your least tidy real API rather than a sample, and read the first run. Then take one convention your organisation actually has and try to express it as a rule. Then take an API that legitimately cannot meet one rule and see how the tool records the exception. The three answers together tell you more than any feature comparison.
Last reviewed by The Routebase Team.