Chapter 08 of 10
Measuring API Documentation
The signals that tell you whether documentation is working, why failed searches are the best backlog you have, how to read reader feedback, and which numbers to ignore.
Documentation is unusual among product surfaces because success is almost invisible. A reader who found what they needed leaves without a trace, so most of what you can measure is people struggling.
That is fine, as long as you measure the struggling deliberately.
Failed searches are the best list you will get
Your documentation search records what readers expected to find. The subset that returned nothing is the most actionable list in the whole toolbox.
Each entry is a reader who believed something should exist, written in their vocabulary rather than yours. That makes the list useful twice, once as a backlog of missing pages and once as a glossary. A term that readers type repeatedly and your pages never use is a naming problem you can fix in an afternoon.
Read it weekly rather than quarterly. The terms are seasonal, and the ones that follow a release are telling you something about that release.
Feedback is the only per-page signal
Analytics tell you which pages are visited. They cannot tell you whether the visit worked, and a rating on the page can.
Keep the ask small, because a thumbs up or down with an optional comment is something a reader will actually do. The comments are where the value is, since they name the specific missing thing. The failure mode is collecting them for a year and never opening the list, which turns a measurement into decoration.
One caution about reading the results. Negative feedback clusters on the hardest pages rather than the worst ones, so an authentication page with a poor rating may be doing a good job on a genuinely difficult subject. Compare a page against its own history rather than against the pages next to it.
Support tickets are documentation bugs
Every question your team answers more than twice is a page that did not do its job.
The useful habit is cheap. When somebody answers a question, they note which page should have contained the answer. After a month you have a ranked list of documentation defects, ranked by how much time they cost.
This is also the signal that catches the failure the others miss, which is documentation that is present, correct and unfindable.
The numbers that mislead
Three metrics get reported often and decide very little.
Page views measure traffic and interest. A high-traffic page is worth keeping correct and tells you nothing about whether it works.
Time on page is ambiguous in both directions, because a long visit can mean thorough or lost.
Coverage percentages are a floor rather than a target. Knowing that every endpoint carries a description is worth having, since a missing description is a certain defect. It does not tell you whether the descriptions are any good, and a team that optimises the number will get sentences that repeat the endpoint name.
Measure the first call
If you only instrument one thing, instrument the path from landing on the documentation to a successful first request.
That is the moment your API is won or lost, and it is measurable through your own API rather than through the documentation site. It is also the number that moves when the Getting Started page actually improves.
In Routebase
A published portal reports page views, unique visitors and search behaviour, including the search terms that returned no results, which is the content-gap list described above. The feedback dashboard sits next to it and groups pages by how readers rated them.
The version overview adds endpoint coverage for the embedded reference, meaning how many of the endpoints in it carry a description, so a missing one is visible before a reader finds it. Feedback can be switched off per portal if you would rather not collect it. The Publishing guide covers the analytics and feedback panels.
Frequently asked questions
How do you measure documentation quality?
By the questions your documentation failed to answer rather than by how much of it people read. Searches that returned nothing name the page you have not written, negative feedback on a page names the section that is unclear, and repeated support questions name both. Page views tell you what is popular, which is rarely the same as what is working.
What are content gaps in documentation?
They are search terms that readers typed into your documentation and that returned no results. Each one is a reader who expected something to exist and did not find it, expressed in their own vocabulary rather than yours. That also makes the list a good source of the words your pages should use.
Should documentation pages have feedback buttons?
Yes, as long as somebody reads the results. A thumbs up or down with an optional comment costs a reader almost nothing and produces the one signal that is attached to a specific page. The failure mode is collecting it for a year without looking, at which point it is a widget rather than a measurement.
Is time on page a useful documentation metric?
Rarely, because it is ambiguous in both directions. A long visit can mean the page was thorough or that the reader could not find the sentence they needed. Treat it as a prompt to look at a page rather than as a verdict on it, and reach for the signals that carry an explicit question instead.
Last reviewed by The Routebase Team.