API Documentation for AI Agents

An agent reading your documentation is doing what a new developer does, with two differences. It cannot ask a colleague, and it will not scroll past a wall of navigation to find the sentence that matters.

That makes this a formatting problem before it is anything else.

What an agent actually receives

When an assistant fetches a documentation page, it gets HTML. Inside that HTML the useful content is wrapped in navigation, a sidebar, a cookie banner and whatever the framework emitted.

Converting that back to clean text is possible and lossy. Tables come out mangled, code blocks lose their language, and the parts that carried meaning visually lose it entirely. Every wasted token also costs context that the agent needed for your actual content.

So the first thing an agent-friendly documentation site does is offer the content without the page around it.

The llms.txt conventions

The llms.txt proposal covers both halves, and it is a proposal rather than a web standard, retrieved 2026-09-13.

The first half is an index. A file at /llms.txt carries a short description of the site and a curated list of links, so an agent that lands on your root can see the shape of the documentation without crawling it. The file stays small on purpose, because it is meant to fit in a context window while the detail waits behind the links.

The proposal fixes the order of the parts. An H1 is the only required section, a blockquote with a short summary follows it, and sections under H2 headings then list the links with optional notes. An index for a fictional orders API follows that order.

# Orders API
 
> REST API for placing, tracking and cancelling orders. All amounts are
> in minor units of the currency named on the order.
 
## Guides
 
- [Getting started](https://docs.example.com/getting-started.md): First authenticated call in five steps
- [Authentication](https://docs.example.com/authentication.md): API keys and OAuth 2.0 client credentials
 
## API reference
 
- [Create an order](https://docs.example.com/reference/create-order.md)
- [Cancel an order](https://docs.example.com/reference/cancel-order.md): Only before shipping
 
## Optional
 
- [Changelog](https://docs.example.com/changelog.md)

The proposal treats the section named Optional as the part an agent may skip when its context is short.

The second half is a Markdown version of each page, served next to the HTML. The proposal suggests appending .md to the URL, or index.md where the URL has no filename. An agent that follows a link from the index then gets the content rather than the chrome.

A single-file variant is common as well, where the whole site is concatenated into one Markdown document. It suits an agent that would otherwise fetch twenty pages one at a time.

The same URL, in Markdown

A .md link only helps an agent that knows the convention and finds the link. Most agents start from a URL they were given, and that URL points at the HTML page.

HTTP already has a mechanism for this, called proactive content negotiation and defined in RFC 9110, section 12. The client names the formats it accepts in the Accept header, and the server picks the representation that fits. A documentation site that honours Accept: text/markdown therefore serves the Markdown version on the normal page URL, so an agent never has to know where the twin lives. The llms.txt proposal doesn't cover this, while the Agent-Friendly Docs Spec checks it as part of Markdown availability.

The difference is visible from a terminal, since the same address answers with a different content type depending on the header.

curl -sI https://docs.example.com/authentication/ | grep -i content-type
# content-type: text/html; charset=utf-8
 
curl -sI -H "Accept: text/markdown" https://docs.example.com/authentication/ | grep -i content-type
# content-type: text/markdown; charset=utf-8

Once one URL has two representations, every cache in between has to keep them apart. Both responses should therefore carry Vary: Accept, which tells a cache that the answer depends on that request header, as section 12.5.5 of the same RFC describes. If either response leaves it out, a shared cache can hand the HTML to an agent or the Markdown to a browser.

Lead every page back to the index

An agent that arrives on a single page has no reason to look for /llms.txt unless the page tells it to. A short sentence at the very top does that job, saying that the site can be read as Markdown and where the index lives. In the HTML it is hidden visually and from screen readers, and in the Markdown version it stands right under the title.

Links inside the Markdown need the same care. A root-relative link such as /authentication/ works in a browser, because the browser knows the host. Once an agent has copied the Markdown into a prompt or a file, the host is gone, so the Markdown version should carry absolute links.

Position matters as well. An agent reads from the top, so a page whose content begins only after a long navigation or a sidebar spends part of the agent's context before it says anything useful.

Generated reference pages have to be text

This is where documentation platforms usually fall down, and it is worth checking on your own site.

Written pages are easy, because they were Markdown before they were HTML. Generated endpoint pages are not, since they are often assembled from a schema at render time into a component tree that has no text representation at all. The result is a site where the guides are readable and the reference, which is the part an agent needs most, is a blank.

A generated endpoint page that serves properly needs its path, its method, its parameters, its bodies and its errors written out as text. Schema references have to be resolved rather than left as pointers an agent would follow and reassemble.

Robots and discovery

An agent that has not been told where your index lives has to guess, and robots.txt is the file it already reads.

Point it at your sitemap and at your llms.txt, which costs one line each. The llms.txt proposal also describes link relations that let a page advertise its own Markdown twin, using rel="alternate" with the Markdown media type.

Check what an agent receives

The Agent-Friendly Docs Spec collects these points into checks you can run against your own site. It is published under CC BY 4.0, and its companion tool afdocs runs 28 checks in seven categories, from the index and the Markdown versions to page size, redirects and bot protection.

npx afdocs check https://docs.example.com --format scorecard

The tool samples a set of pages on every run, so two runs can land on different pages and report different findings. Run it more than once before you read a single result as the state of the whole site, and look at the individual checks rather than only at the score.

The writing still decides the outcome

Formatting gets the content to the agent. It does not make the content good.

Everything in how to write API reference documentation applies twice over here, because an agent has no intuition to fall back on. A summary that restates the endpoint name leaves it choosing between two operations that look identical, and an enum without meanings leaves it guessing what a status value implies.

Reading is only half of it

Everything above assumes the agent is a consumer of your documentation. The other case is an agent that maintains it.

An assistant that can read the content tree, write a page, and publish a version is doing the same job a technical writer does, under the same permissions and with the same review step. That is worth setting up carefully rather than quickly, since documentation is a publishing surface and an agent with write access to it is publishing on your behalf.

In Routebase

Every portal build in Routebase emits the machine-readable outputs alongside the HTML, and they are not a switch you turn on. A build that fails to produce them fails. The product page for agent-ready APIs puts these files next to the descriptions and errors an agent reads first.

There is an index at /llms.txt, a single-document version of the whole portal at /llms-full.txt, and a Markdown twin at {slug}/index.md for every page. Reference pages emit generated Markdown covering the path, the method, the parameters and both bodies with schema references resolved, so an endpoint page is readable as text rather than only as a rendered widget. The robots.txt of the portal points at the sitemap and at the index.

Every portal also answers Accept: text/markdown on the URLs of its content pages, so an agent gets the Markdown version without knowing that the twin exists. Each page opens with a sentence that points agents at the index, and the Markdown versions use absolute links throughout. Portals last built before October 5, 2026 pick up the sentence and the absolute links with their next build. Our own documentation scored 100 out of 100 on the Agent-Friendly Docs Scorecard in a run with afdocs on October 6, 2026. You can run the same scorecard on any of your portals from the Agent Readiness card in the portal settings. It shows the score and every check that did not pass, together with what to change, and an agent can start it over the Routebase MCP server.

Our own product documentation runs on this, so the files are open to check rather than to take on trust. The index, the full document and the Markdown twin of any page all answer right now.

Readers get the same thing through the page actions menu, which offers the page as Markdown, the OpenAPI document as a download, and a hand-off into an assistant. Beyond that, the documentation module is reachable over the Routebase MCP server, so an agent can read the content tree, write pages into a draft version and publish it under the permissions its key carries. The Publishing guide documents the emitted files, and the Documentation Overview covers the agent side.