# How We Got Our Docs to 100 on the Agent-Friendly Docs Scorecard — Routebase > AI agents can read every page of this site as Markdown, and the full index is at https://routebase.dev/llms.txt. > We took routebase.dev from 87 to 100 on the open Agent-Friendly Docs Scorecard in one day. What failed, the four fixes, and the CDN header detail that nearly broke it. Canonical page: https://routebase.dev/blog/agent-friendly-docs-scorecard/ Published: 2026-10-06 · The Routebase Team · AI Agents, Documentation, llms.txt When an agent fetches a page of your documentation, it gets the same HTML your browser gets. In a sample of 20 pages from routebase.dev, the median page weighs 148 KB as HTML and only 8 KB as Markdown. More than nine tenths of what an agent downloads is therefore markup, scripts and navigation. Some agents cut the page off and others summarise it first, and in both cases the answer your user gets rests on less than you published. The [Agent-Friendly Docs Spec](https://agentdocsspec.com/spec/web/) turns that problem into something you can measure. It is an open specification under CC BY 4.0, and its companion tool [afdocs](https://afdocs.dev) runs 28 checks in seven categories against any documentation site. We ran it against routebase.dev on October 5 and scored 87 out of 100. By the end of the same day the score was 100, and this post walks through what we changed. ## What the scorecard measures The checks follow the path an agent takes through a site. They ask whether the agent can find an index of your pages, whether each page exists as Markdown, and whether that Markdown fits into a context window and matches what a human sees. Further checks cover redirects, error codes and caching, and whether bot protection blocks the agent before it reads anything. You can run the same check yourself with one command, and it needs nothing but a URL. ```bash npx afdocs check https://docs.example.com --format scorecard ``` ## Where we started Our `llms.txt` was already in good shape, so the index checks passed. The 87 came from five findings, and each of them pointed at the way the site was built rather than at what the pages said: - A request with `Accept: text/markdown` still received HTML. - Eight of 49 sampled pages differed between their Markdown and HTML versions, mostly because product mockups and calls to action existed only in the HTML. - On guide chapters the content began after the navigation and a chapter list, so across the sample it started at a median of 30 percent of the page. - No page told an agent that an `llms.txt` existed. - The Markdown versions carried 467 links like `/pricing/` that only resolve when you already know the host. ## Fix 1: a Markdown version of every page Most pages of the site already had a Markdown twin at `/.md`. We added a build step that places a copy at `//index.md` next to every `index.html`, and it converts the few pages without a twin from their main element. As a result, every page now exists in both forms under the same path. Parity was the harder half. Live product mockups, pricing widgets and call-to-action blocks mean something to a person and nothing to an agent, so we marked them with `data-markdown-ignore`, which afdocs leaves out of its comparison. We also added the page headline to every Markdown version and rewrote all root-relative links as absolute ones. A link then keeps working after the Markdown has been copied into a prompt or a file somewhere else. ## Fix 2: content negotiation at the edge A `.md` URL helps only an agent that knows to ask for it. Content negotiation lets every agent use the URL it already has, because the request states which format it wants and the server answers in that format. Our sites sit behind a CDN, so the decision happens there in a single edge rule. When the path ends in a slash and the `Accept` header contains `text/markdown`, the rule rewrites the request to the `index.md` in the same folder. _Figure: One URL, two answers. A browser asks for HTML and receives the full page. An agent that sends Accept: text/markdown to the same URL receives the Markdown version at about a twentieth of the size, and both responses carry Vary: Accept so that no cache mixes them up._ The effect is easy to check from a terminal, since the same URL answers with a different content type depending on the header. ```bash curl -sI https://routebase.dev/pricing/ | grep -i content-type # content-type: text/html curl -sI -H "Accept: text/markdown" https://routebase.dev/pricing/ | grep -i content-type # content-type: text/markdown; charset=utf-8 ``` Two details decide whether this stays correct once caches are involved. Every page response has to carry `Vary: Accept`, so that no cache along the way hands the HTML to an agent or the Markdown to a browser. In addition, our negotiated responses bypass the edge cache entirely, which keeps the cached HTML and the Markdown from ever sharing a cache entry. The header itself held the one surprise of the day. Our CDN can modify a response header with either Append or Overwrite, and Append sounds like the safe choice when the origin already sends a `Vary`. Its documentation says, however, that Append joins the two values by plain string concatenation and adds no delimiter, and that is easy to read past. On our docs host the origin already sent `Vary: Origin`, so appending `Accept` produced `Vary: OriginAccept`, which no cache can interpret. We caught it in the probe after the rollout, and the fix was Overwrite with the complete value `Origin, Accept`. ## Fix 3: tell the agent where to look An agent that lands on a single page has no reason to look for `llms.txt` unless something tells it to. Every HTML page now starts with one short sentence for agents, which is hidden visually and from screen readers. Every Markdown version opens with the same sentence right under its headline, and it says that each page can be read as Markdown and where the full index lives. ## Fix 4: content first in the document Agents and most HTML-to-Markdown converters read a page from the top, so anything that comes before the content costs context. On our guide chapters the chapter list stood in the markup before the article. We moved it behind the article and kept its position on screen with CSS grid order. The teasers inside the navigation menus now render only once the page runs in a browser, so they never appear in the server-rendered HTML that an agent fetches. In our own measurement the content on guide chapters now begins at 14 percent of the page instead of 35. Across its sample, the scorecard now reports a median start of one percent. ## The result Two runs on October 5 and another on October 6 each returned 100 out of 100, which is grade A+. The scorecard draws a sample of 50 pages per run, so a single run can land on pages the others missed, and that is why we don't rely on one run alone. Our documentation portal at docs.routebase.dev went through the same kind of changes and scored 100 on its latest run as well. None of the fixes rewrote what our pages say. Every change sat in the build, the markup or the edge configuration, which also means that every page we add from now on inherits it without extra work. ## In Routebase Routebase publishes your [API documentation](https://routebase.dev/api-documentation/) as a portal that is generated from your spec, and every portal comes with this behaviour by default. Each content page exists as Markdown and answers `Accept: text/markdown` on its normal URL. Each one opens with the `llms.txt` directive and uses absolute links in its Markdown, so an agent can follow them wherever the text ends up. You can also measure any of your portals without leaving Routebase. The **Agent Readiness** card in the portal settings runs the same scorecard against the published portal and shows the score, the grade and every check that didn't pass. Checks that Routebase already handles are marked as such, and the others come with a hint on what to change. When a portal has no meta description, for example, the card takes you straight to that field. An agent working over the Routebase MCP server can start the check and read the result under the same permissions as your team. Existing portals pick the changes up with their next build. The ideas behind them, from `llms.txt` to Markdown versions of generated reference pages, are explained in the guide chapter [API documentation for AI agents](https://routebase.dev/guides/api-documentation/api-documentation-for-ai-agents/). --- [Routebase](https://routebase.dev/) — [Sign up](https://app.routebase.dev/): Every account starts with a 14-day Pro trial — no credit card required.