Agent Readiness Score vs. Task Completion: How to Test AX
An agent readiness score checks whether files and headers exist. It can't tell you if an AI agent can sign up, pay, or make a first API call. Here's how to test that.

An agent readiness score tells you whether the right files and headers exist on your site. It doesn’t tell you whether an AI agent can finish anything there. A site can publish a sitemap, serve markdown, list an MCP server and still lose every agent at the signup form. The score checks whether the doors exist. Task completion is whether an agent walks through them, gets an API key, and makes a first call. Those are different measurements, and buyers are starting to depend on the second one.
That matters because agents now do the first round of work. Vercel says more than 50% of its deployments were started by coding agents as of September 2026, up from over 30% in April. On Mintlify-hosted docs, agents made 257 million requests against 131 million human page loads in August 2026, or 66% of traffic. Hi Violet tests whether those agents succeed by running them through the tasks and recording what happens.
What is an agent readiness score?
An agent readiness score is an automated grade, usually 0 to 100, that checks a website for the files, headers and endpoints AI agents use: robots.txt, sitemaps, markdown versions of pages, bot-access rules, OAuth discovery, and MCP or API catalogs. It measures whether those pieces are present. It does not run an agent through a task.
At a glance
- Readiness scores check presence. Cloudflare’s isitagentready.com checks discovery, content, bot-access and capability files. Its commerce checks (x402, UCP, ACP) are listed but not scored.
- Most sites still miss the basics. Across the top 200,000 domains, markdown negotiation passes on 3.9% of sites, and MCP Server Cards and API catalogs appear on fewer than 15.
- Docs format changes what coding agents do. In Mintlify’s 2,400-run benchmark, 404s per task fell from 2.23 on HTML docs to 0.11 on markdown with an llms.txt link.
- Netlify’s AXIS is the closest public method to a real test. It scores agents on scenarios you write. It’s built for developer tasks, not buyer journeys.
- Readability changes recommendations. A September 2026 study of 37,927 agent journeys found agent-ready businesses got 1.9x the recommendation rate.
- Scanners reward files, not answers. In the Hi Violet AX Panel 2026, 25 of 30 developer products publish an MCP server and 29 an llms.txt file, yet their own pages leave 22% of buyer questions unanswered.
- A real agent path clears every run. Six measured Claude Code and Codex runs through Stripe’s signup cleared all five steps, from finding signup to a first API call, through
stripe sandbox create. - Only a run shows where an agent stops. Hi Violet marks every step of every run Cleared, Asked or Stopped, fixes the stops, and re-runs the same tasks.
What does isitagentready.com actually check?
Cloudflare’s scanner, launched April 17, 2026, grades a domain in four scored categories and one unscored one. It’s the most widely cited agent readiness score, and it’s useful: it gives you a punch list of missing files in under a minute.
| Category | What it checks |
|---|---|
| Discoverability | robots.txt, sitemap.xml, Link headers (RFC 8288) |
| Content | Markdown for Agents (serving markdown when an agent asks for it) |
| Bot access control | Content Signals, AI bot rules in robots.txt, Web Bot Auth |
| Capabilities | Agent Skills, API Catalog (RFC 9727), OAuth server discovery (RFC 8414 and RFC 9728), MCP Server Card, WebMCP |
| Commerce (not scored) | x402, Universal Commerce Protocol, Agentic Commerce Protocol |
Cloudflare’s scan of the top 200,000 domains shows how few sites have done this work. 78% have a robots.txt. 4% declare AI usage preferences through Content Signals. Markdown negotiation passes on 3.9%. MCP Server Cards and API catalogs together turn up on fewer than 15 sites in the whole dataset. Cloudflare also reports that on its own docs an agent used 31% fewer tokens and answered 66% faster than on sites without these optimizations.
Notice what’s missing. Nothing checks signup, plan choice or pricing answers, and commerce doesn’t count toward the grade. Those are exactly the gaps that show up when you ask agents the questions buyers ask. The scanner reads your site like a crawler, one request at a time. It never plays an agent trying to get something done.
What do docs scores like Fern’s Agent Score measure?
Fern’s Agent Score grades documentation 0 to 100 against the open Agent-Friendly Docs Spec (AFDocs): 22 checks across seven categories, covering llms.txt, markdown, page size, content structure, URL stability, discoverability and observability. It asks whether agents can find the docs index, get clean markdown instead of heavy HTML, fit a page in their context window, and follow URLs without hitting traps.
Those checks track something real. Mintlify ran 2,400 Claude Code and Codex runs against 20 docs sites, published July 17, 2026. Agents averaged 2.23 dead-URL hits per task on HTML docs, 1.42 on plain markdown, and 0.11 on markdown with an llms.txt link. Accuracy stayed in the mid-to-high 90s across every format. Format changed how many dead ends agents walked into, not whether they eventually got the answer.
llms.txt is widely misread. Ahrefs looked at 137,210 domains in May 2026: 28% published an llms.txt, and 97% of those files got zero requests that month. AI bots never asked for an llms.txt that didn’t exist. That fits Mintlify’s result. llms.txt is a map that coding agents follow when something points them to it. It is not a way to get cited by answer engines. Our AI search work treats it the same way.
A docs score covers one stretch of the journey: reading. It doesn’t touch the signup form, the pricing page or the payment step.
Why can a site pass the scan and still lose every agent?
Because the scan and the agent ask different questions. The scan asks whether a resource exists and is well formed. The agent asks whether it can get from “I need an API for this” to a working first call without a human stepping in. A sitemap doesn’t help when the signup form needs a CAPTCHA. Clean markdown docs don’t help when the price for the plan the agent needs says “contact sales”.
Here is how five agent types fare on a product that passes a static readiness check, all running the same signup-to-first-call task.
A passing scan, five runsHow the runs play out: a product that passes a static readiness check, one signup-to-first-call task, five agent types
- Cleared
- Asked
- Stopped
- Not reached
| Agent | 01Find the docs | 02Choose a plan | 03Create an account | 04Get an API key | 05Make a first call |
|---|---|---|---|---|---|
| Coding agent (CLI) | Cleared | Cleared | Stopped | Not reached | Not reached |
| Coding agent (IDE) | Cleared | Asked | Stopped | Not reached | Not reached |
| Browser agent | Cleared | Cleared | Cleared | Stopped | Not reached |
| Chat assistant | Cleared | Asked | Stopped | Not reached | Not reached |
| Research assistant | Cleared | Stopped | Not reached | Not reached | Not reached |
Figure 1. How the runs play out. Every agent finds the docs, because the discovery files are in place. None reaches a first call. Three stop at account creation, two of them after asking a plan question the docs never answer. The browser agent gets an account but can’t get an API key, and the research assistant stops at a plan page that asks it to contact sales.
Now the opposite case, measured rather than typical. In the AX Panel we sent Claude Code and OpenAI Codex through Stripe’s signup three times each, with one task: get working test keys and reach a first API call without a human. Every run cleared every step. Stripe ships a door built for agents: stripe sandbox create provisions a sandbox with working test keys and no account, after a proof-of-work check instead of a CAPTCHA. The first human step is the claim, after the work is done. Our study of what 30 products ask before the first API call covers the full design.
Stripe: six measured agent runsClaude Code and OpenAI Codex, three runs each, November 16, 2026. Task: get working test keys and reach a first API call without a human
- Cleared
- Asked
- Stopped
- Not reached
| Agent | 01Find signup | 02Read the requirements | 03Sign up with no human | 04Get credentials | 05Make a first call |
|---|---|---|---|---|---|
| Claude Code, run 1 | Cleared | Cleared | Cleared | Cleared | Cleared |
| Claude Code, run 2 | Cleared | Cleared | Cleared | Cleared | Cleared |
| Claude Code, run 3 | Cleared | Cleared | Cleared | Cleared | Cleared |
| Codex, run 1 | Cleared | Cleared | Cleared | Cleared | Cleared |
| Codex, run 2 | Cleared | Cleared | Cleared | Cleared | Cleared |
| Codex, run 3 | Cleared | Cleared | Cleared | Cleared | Cleared |
Figure 2. Stripe, measured. Six runs from the Hi Violet AX Panel 2026, November 16, 2026: Claude Code and OpenAI Codex, three runs each, every step cleared through stripe sandbox create. The agents read the CLI’s help and Stripe’s docs and stopped short of creating anything; the first step that needs a person is claiming the sandbox.
Every failure on the first board sits in a place a scanner never looks. Here’s the difference side by side.
| What a scan checks | What a run observes |
|---|---|
| A sitemap and robots.txt exist | Whether the agent finds the right page for its task, and how many dead ends it hits first |
| Docs are served as markdown | Whether the docs answer the question the agent actually has, or whether it has to guess |
| OAuth discovery metadata is published | Whether the agent can create an account at all, or hits a CAPTCHA, an email code or a card form |
| An MCP Server Card is listed | Whether the agent picks your MCP tools, and calls them correctly on the first try |
| x402 is detected (unscored) | Whether a paid request completes and the agent gets the resource it paid for |
| A pricing page exists | Whether the agent can match its user’s needs to a plan without being told to contact sales |
| One request at a time | A full task, step by step, repeated across agents and runs |
The AX Panel shows the same gap at scale. Across its 30 AI, Web3 and payments products, 25 publish an MCP server, 29 publish an llms.txt file and 20 ship an MCP server, a CLI and an llms.txt together: the files a scanner rewards. Yet the companies’ own pages leave 79 of 360 buyer questions (22%) unanswered. Thirteen of 30 never say whether a card is needed before the first API call, 14 never say whether SAML SSO is available, and 19 never say whether data can stay in the EU. A scanner checks files. An agent needs answers, and when the page is silent, it guesses.
A score measures what’s easy to collect. A run measures the decision you care about. That’s the argument of our growth signal stack: pick the observable that matches the decision, not the one that happens to be available.
Is Netlify’s AXIS the same as an agent run?
It’s the closest public thing to one, and it deserves credit. Netlify coined “agent experience”, which its CEO Mathias Biilmann defined in January 2025 as “the holistic experience AI agents will have as the user of a product or platform”. AXIS, the Agent Experience Index Score, is Netlify’s open-source framework for measuring it, built with Auth0 as a founding contributor.
AXIS runs real agents. A scenario is a JSON file with a prompt, a rubric and optional setup steps. AXIS ships with 23 built-in agents, including Claude Code, Codex, Gemini, Cursor and Copilot, and scores four dimensions:
- goal achievement (40%), judged by an LLM against your rubric
- environment (20%): shell, filesystem, build tools
- service (20%): APIs, MCP tools, third-party services
- agent behavior (20%): planning, tool choice and decisions
It plugs into CI so you can catch regressions the way unit tests do.
That’s a test, not a scan, and it’s aimed at developers: the weights assume a coding agent in a repository with a shell. Out of the box it doesn’t cover the buyer’s side, such as an assistant comparing your pricing to a competitor’s, a browser agent filling in your signup form, or an agent paying for a request. And you still write the scenarios, interpret the scores and build the fixes yourself.
An agent experience audit picks up where AXIS stops.
Does agent readiness change what agents recommend?
Yes, and there’s now public data on it. In “AX is the New AEO”, submitted September 28, 2026, Ido Finder, Assaf Elovic and Gad Shalev ran 37,927 agent journeys across 1,056 real businesses. Each journey was a buyer question about one business, run through four independent harnesses. Their findings:
- Only 7-10% of a finished answer came from the model’s training knowledge. The rest came from what the agent read during the journey.
- Agent-ready businesses had answers built from their own pages 78% of the time, against 56% for the rest.
- Agent-ready businesses were recommended 1.9 times as often.
- Answers built from the business’s own site were 41% more accurate.
Microsoft frames the same idea as two measures. Waldek Mastykarz, a Principal Developer Advocate at Microsoft, wrote on October 6, 2026 that agent experience comes down to propensity (“does the agent find your technology and choose it?”) and efficacy (“once the agent uses your technology, does it use it correctly?”). His team’s example is worth copying. While testing SharePoint Framework upgrades, they watched the agent’s step-by-step trail and saw it read the release notes mid-task. So instead of shipping a new agent skill, they fixed the release notes. After that, every run used the right CLI without being told.
A run trace found that fix. No score would have.
How do you test your website with AI agents?
You run them through real tasks, repeatedly, and write down exactly where each one gets stuck. Here’s the protocol Hi Violet uses. It’s the same method described on how Hi Violet works, applied to agent experience testing.
- Fix the task set. We write the tasks your buyers’ agents actually attempt, in plain language and with a clear finish line. For example: “find a plan that allows 50,000 API calls a month and tell me the price”, “create a free account and get an API key”, “make a first authenticated call”, “buy credits for this project”, “check whether this tool supports EU data residency”. The tasks are frozen before the first run so nobody can tune them to make results look better.
- Fix the agent roster. Coding agents in the terminal and the IDE, browser agents that click through pages, and chat and research assistants that read and answer. Different agents fail in different places. A CLI agent stuck at an email code tells you nothing about a browser agent stuck at a card form.
- Run in a sandboxed copy of the product. Agents work in an isolated environment that mirrors your docs, signup and API (the RL environments described on our method page). That lets them create accounts, hit rate limits and attempt payments without touching production data or real customers.
- Repeat every agent on every task. One run is an anecdote. Agents take different paths through the same task, so we run each agent through each task several times. The run count is set before the first run and kept for the re-test.
- Mark every step. Each step of each run gets one of three marks. Cleared: the agent got through. Asked: it got through only after asking a question your docs or pages never answer. Stopped: it couldn’t go on. Steps after a stop are recorded as not reached. The board in Figure 1 uses exactly these marks.
- Keep the Asked log. Every question an agent had to ask goes into a list, word for word. That list tells you what to write next: the price for overage, whether the free plan allows commercial use, which SDK version the quickstart assumes.
- Fix what stopped them. Each stop gets a cause and a fix. Some fixes are copy. Others are infrastructure.
- Re-run the same tasks. Same tasks, same agents, same run count. The result is a before-and-after board you can compare cell by cell, not a new score on a different test.
It’s the discipline behind all our measurement work: keep what was reported apart from what was verified, as in our ChatGPT Ads measurement study. “Stopped at email verification” is a verified observation. A score of 80 summarizes presence checks.
What fixes usually follow a run?
The fixes follow the stops. Four come up again and again.
Signup walls. CAPTCHA, email codes and card forms stop most agents cold. The fix is a gated free signup: a signed agent can create a capped sandbox account and hand a claim link to a human, who takes ownership later. We cover the pattern in how to let AI agents create accounts.
Payment steps. When an agent needs one paid request, an account plus a subscription is too much. x402 payments let an API answer with a price and accept payment in the same HTTP exchange, so the agent never needs an account. The PIVX Agent Kit gives agents a private, shielded wallet for cases where the transaction itself shouldn’t be public (Hi Violet is affiliated with PIVX Labs). They’re separate rails. A product can support both, for different or overlapping use cases. We compare them with Stripe’s, Google’s and the card networks’ options in agent payment protocols compared.
Evaluation access. An agent evaluating your product for a buyer often needs to see the real thing, not the marketing page. Handing it an API key is the wrong tool for a one-off look. A qURL gives one agent expiring access to one private resource. See temporary access for AI agents.
Unanswered questions. The Asked log becomes docs and pricing copy, often the cheapest fix of all.
How Hi Violet runs agent experience testing
Hi Violet runs agents through your product before your buyers’ agents do. We build the task set with you, run coding agents, browser agents and assistants through sandboxed copies of your docs, signup, pricing and API, and deliver the board: every run, every step, every Cleared, Asked and Stopped. Then we ship the fixes, from docs rewrites to gated agent signups, x402 or PIVX Agent Kit payments and qURL evaluation access, and re-run the same tasks to show what changed.
Start with your score, since it’s free and it’s a fine checklist. Then find out whether agents can finish the job. See how an agent experience (AX) audit works, or request access to scope one for your product.
Frequently asked questions
What is a good agent readiness score?
There’s no shared scale. Cloudflare, Fern and the smaller checkers each grade different checks, so their numbers aren’t comparable. Use your score as a list of missing files, then test whether agents can finish the tasks you care about.
Is my site agent ready if it passes isitagentready.com?
Passing means your discovery, markdown, bot-access and capability files are in place. It doesn’t mean an agent can create an account, choose a plan, pay or make a first API call, and Cloudflare doesn’t score commerce at all. Only a run answers that.
How do you test a website with AI agents?
Fix real tasks with clear finish lines, pick coding agents, browser agents and assistants, and run each through each task several times in a sandboxed copy of the product. Mark every step Cleared, Asked or Stopped, fix the stops, and re-run the same tasks.
Does llms.txt improve agent readiness?
For coding agents reading docs, yes: in Mintlify’s benchmark, an llms.txt link cut dead-URL hits per task from 1.42 to 0.11. But 97% of llms.txt files got no requests in Ahrefs’ May 2026 data. It’s a map for agents pointed to it, not a visibility tactic.
What is the difference between AXIS and an agent experience audit?
AXIS is Netlify’s open-source framework for scoring coding agents on scenarios you write. An agent experience audit applies the same run-and-judge idea to buyer journeys (pricing, signup, payment, evaluation access) and includes the fixes and a re-test.
Can an AI agent sign up for a product without a human?
Only where the product builds a path for it. In the Hi Violet AX Panel 2026, Stripe’s stripe sandbox create gave agents working test keys with no account, and six of six Claude Code and Codex runs cleared every step from finding signup to a first API call. A human claims the sandbox afterwards, which keeps the account and everything the agent built.
How many agent runs does a test need?
Enough to see variance, because agents take different paths through the same task. Set the count before the first run, keep it identical for the re-test, and report every run, not the best one.


