Can AI Agents Answer Questions About Your Product?
Can AI agents answer questions about your product? We checked 30 AI, Web3 and payments products: their own pages leave 22% of buyer questions unanswered.

Can AI agents answer questions about your product? We tested that on 30 developer products across AI, Web3 and payments in the Hi Violet AX Panel 2026. The answer depends less on the agent than on your own website. We asked each company’s official pages 12 buyer questions. On 79 of the 360 product-question pairs, the official pages had no answer. That’s 22%. EU data residency went unanswered on 19 of 30 sites. SAML single sign-on on 14. Whether a card is needed before the first API call on 13.
Agents don’t stop at that gap. Claude Code and OpenAI Codex, the two coding agents in this study, read privacy policies, legal terms, community forums and third-party directories, then answer anyway. Your buyer gets a confident sentence the company never wrote. The fix is boring and it works: publish the answer on a page an agent can fetch.
What does it mean for an AI agent to answer a product question?
An AI agent answers a product question when it researches a buyer’s question on the web and returns a direct answer with sources, for example “Is a card required before the first API call?” The answer is reliable when the company’s own page states the fact. When the page is silent, the agent infers.
That definition matters because the agent is now the first reader of your pricing page. Agents made 66% of the traffic on Mintlify-hosted docs in August 2026. A developer who asks Claude Code “should we use this?” reads your docs through the agent’s summary, not through your layout.
At a glance
- The official pages left 22% of buyer questions unanswered. 79 of 360 product-question pairs across 30 products had no answer on the company’s own site.
- Payments sites leave the most gaps. 38 of 120 payments answers were missing, against 25 of 120 for Web3 and 16 of 120 for AI.
- Three questions account for most of the silence. EU data residency (19 of 30), SAML SSO (14) and card policy (13).
- Signup itself is well documented. All 30 products state how a developer gets API keys, and 29 of 30 publish whether they offer an MCP server, a CLI or an llms.txt file.
- Two products answered everything. Modal’s and QuickNode’s own pages answered all 12 questions.
- Agents fill gaps with inference. On Stripe, both agents answered every question Stripe’s pages leave open, in every run, and both judges flagged those answers as unsupported by Stripe’s own pages.
How did we test whether AI agents can answer product questions?
We built a fixed panel and a fixed question bank, then checked every agent answer against the company’s own words. The method is the same one Hi Violet uses in a client agent experience audit, described step by step in how to test AX with real runs.
The panel is 30 self-serve developer products, 10 per segment:
| Segment | Products |
|---|---|
| AI | Mistral, Groq, Together AI, Fireworks AI, Replicate, Hugging Face, ElevenLabs, Pinecone, Cohere, Modal |
| Web3 | Alchemy, Infura, QuickNode, Helius, Moralis, thirdweb, Privy, Dynamic, The Graph, Tenderly |
| Payments | Stripe, PayPal, Square, Adyen, Paddle, Lemon Squeezy, Mollie, Checkout.com, Coinbase Developer Platform, Circle |
We left out OpenAI and Anthropic so neither agent answers questions about its own maker.
The 12 questions are the ones a developer asks before committing: free tier, entry price, overage, rate limits, official SDKs, self-serve keys, card policy, SAML SSO, EU data residency, sandbox or test mode, status page URL, and whether the product publishes an MCP server, CLI or llms.txt.
Each product went through four stages:
- Ground truth. A separate agent session could fetch only the company’s official domains. Every answer needed the page URL and a verbatim quote. Anything the official pages did not state was recorded as not published. We captured all 30 on November 16, 2026.
- Agent runs. Claude Code and OpenAI Codex each answered all 12 questions per product, using their own web search, three times each. Every run starts in an empty folder with no memory of the last one.
- Audit. Wherever the ground truth said “not published” but an agent cited a page, a third session rechecked the cited official pages before any grade counted.
- Grading. A Claude judge and a Codex judge graded every answer against the ground truth. A grade counts only when both judges agree.
Which questions do company websites leave unanswered?
The questions buyers ask late in a decision: data location, SSO and card policy. Card policy matters twice, because it is also where agent signups stall. The early questions, how to sign up and what it costs, are almost always covered.
| Question | Official site has no answer (of 30) | AI | Web3 | Payments |
|---|---|---|---|---|
| EU data residency | 19 | 1 | 9 | 9 |
| SAML SSO and plan | 14 | 2 | 6 | 6 |
| Card required before first call | 13 | 1 | 4 | 8 |
| Sandbox or test mode | 9 | 9 | 0 | 0 |
| Overage behavior | 7 | 0 | 0 | 7 |
| Entry price | 4 | 0 | 2 | 2 |
| Default rate limits | 4 | 1 | 2 | 1 |
| Status page URL | 4 | 2 | 1 | 1 |
| Free tier and limits | 3 | 0 | 0 | 3 |
| Official SDK languages | 1 | 0 | 1 | 0 |
| MCP server, CLI, llms.txt | 1 | 0 | 0 | 1 |
| Self-serve API keys | 0 | 0 | 0 | 0 |
Figure 1. Buyer questions each company’s own pages left unanswered, by segment, out of 30 products (10 per segment). Source: Hi Violet ground truth from official domains only, captured November 16, 2026.
Three patterns stand out.
Payments companies document integration and stay quiet on commercial terms. Eight of 10 payments sites don’t say whether a card is needed before the first call, and seven of 10 don’t say what happens past a plan limit. Part of that is structural: processors like Stripe and PayPal charge per transaction, so “overage” isn’t a concept on their pricing page. An agent still gets asked the question, and has to reason its way to an answer.
Data residency is the widest gap. Nine of 10 Web3 sites and nine of 10 payments sites don’t say whether customers can keep data in the EU. AI platforms are the exception: Mistral, Cohere, Pinecone, Hugging Face and others publish EU regions plainly, because enterprise buyers ask every time.
AI platforms skip the sandbox question. All nine gaps on sandbox or test mode are AI APIs. For a token-billed API a separate test environment matters less, but a developer comparing vendors still asks, and the agent has nothing official to quote.
Which products answered every question?
Modal and QuickNode. Their official pages answered all 12 questions, including the commercial ones. QuickNode’s own site states its free plan needs no card and offers an EU Central region for flat-rate endpoints. Modal’s docs cover spend limits, EEA regions and its CLI.
At the other end, PayPal and Checkout.com each left six of 12 unanswered on their own pages, and Dynamic and Paddle five. These are excellent products with thorough integration docs. The gaps sit almost entirely in the commercial and compliance questions that an agent gets asked before a developer writes code.
The full panel, product by product:
What each company's own pages answerHi Violet AX Panel 2026: 30 developer products × 12 buyer questions
- Answered on the company's own pages281
- Not stated79
Swipe the grid sideways to see all 12 questions.
| Product | Free tier | Entry price | Overage | Rate limits | SDKs | Self-serve keys | Card policy | SAML SSO | EU data | Sandbox | Status page | Agent tooling |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Mistral AIAI | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Not stated | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Not stated | Answered on the company's own pages | Answered on the company's own pages |
| GroqAI | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Not stated | Answered on the company's own pages | Answered on the company's own pages | Not stated | Not stated | Answered on the company's own pages |
| Together AIAI | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Not stated | Not stated | Answered on the company's own pages |
| Fireworks AIAI | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Not stated | Answered on the company's own pages | Answered on the company's own pages |
| ReplicateAI | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Not stated | Not stated | Not stated | Answered on the company's own pages | Answered on the company's own pages |
| Hugging FaceAI | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Not stated | Answered on the company's own pages | Answered on the company's own pages |
| ElevenLabsAI | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Not stated | Answered on the company's own pages | Answered on the company's own pages |
| PineconeAI | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Not stated | Answered on the company's own pages | Answered on the company's own pages |
| CohereAI | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Not stated | Answered on the company's own pages | Not stated | Answered on the company's own pages | Answered on the company's own pages |
| ModalAI | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages |
| AlchemyWeb3 | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Not stated | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages |
| InfuraWeb3 | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Not stated | Answered on the company's own pages | Not stated | Not stated | Not stated | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages |
| QuickNodeWeb3 | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages |
| HeliusWeb3 | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Not stated | Not stated | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages |
| MoralisWeb3 | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Not stated | Not stated | Not stated | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages |
| thirdwebWeb3 | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Not stated | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Not stated | Not stated | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages |
| PrivyWeb3 | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Not stated | Answered on the company's own pages | Not stated | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages |
| DynamicWeb3 | Answered on the company's own pages | Not stated | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Not stated | Not stated | Not stated | Answered on the company's own pages | Not stated | Answered on the company's own pages |
| The GraphWeb3 | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Not stated | Not stated | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages |
| TenderlyWeb3 | Answered on the company's own pages | Not stated | Answered on the company's own pages | Not stated | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Not stated | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages |
| StripePayments | Answered on the company's own pages | Answered on the company's own pages | Not stated | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Not stated | Answered on the company's own pages | Not stated | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages |
| PayPalPayments | Not stated | Not stated | Not stated | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Not stated | Not stated | Not stated | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages |
| SquarePayments | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Not stated | Answered on the company's own pages | Answered on the company's own pages | Not stated | Not stated | Not stated | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages |
| AdyenPayments | Answered on the company's own pages | Answered on the company's own pages | Not stated | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Not stated | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages |
| PaddlePayments | Not stated | Answered on the company's own pages | Not stated | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Not stated | Not stated | Not stated | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages |
| Lemon SqueezyPayments | Answered on the company's own pages | Answered on the company's own pages | Not stated | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Not stated | Not stated | Answered on the company's own pages | Answered on the company's own pages | Not stated |
| MolliePayments | Answered on the company's own pages | Answered on the company's own pages | Not stated | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Not stated | Answered on the company's own pages | Not stated | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages |
| Checkout.comPayments | Not stated | Not stated | Not stated | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Not stated | Answered on the company's own pages | Not stated | Answered on the company's own pages | Not stated | Answered on the company's own pages |
| Coinbase Developer PlatformPayments | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Not stated | Not stated | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages |
| CirclePayments | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages | Not stated | Not stated | Not stated | Answered on the company's own pages | Answered on the company's own pages | Answered on the company's own pages |
Figure 2. What each company’s own pages answer, for all 30 products in the Hi Violet AX Panel 2026 and all 12 buyer questions. A filled cell means an official page states the answer, with a verbatim quote on file; an empty cell means the official pages are silent. Captured November 16, 2026.
What do AI agents say when the official site is silent?
They answer. Stripe shows the pattern most clearly. Stripe’s official pages, captured for this study, don’t state an overage policy, a card policy for the first API call, or an EU data residency option. Across six runs, Claude Code and Codex answered all three questions every time, and both judges agreed every one of those answers went beyond what Stripe’s own pages say.
The answers were reasonable inferences. On EU data, every run concluded Stripe offers no EU-only option, citing Stripe’s privacy center and data processing agreement, which describe international transfers. The conclusion follows from those documents. It’s still the agent’s reading of legal text, not a sentence Stripe published.
The same evidence also produces different answers. On Paddle’s EU residency question, Claude Code answered “No,” reading Paddle’s GDPR page. Codex, reading the same page, returned “not found.” Two agents, one source, two outcomes for the buyer.
Agents also reach for third-party sources. Asked whether Lemon Squeezy publishes an official MCP server, Claude Code cited community MCP directories to answer that only unofficial servers exist. Codex reported nothing found. Lemon Squeezy’s own pages don’t address it, so the buyer’s answer depends on which agent they use.
Why do agents answer instead of saying “not found”?
Because they’re built to finish the task. A coding agent asked “can we keep data in the EU?” treats a missing answer as a research problem, not a stopping point. It reads the privacy policy, the DPA, the help center and the forum, then commits to a conclusion.
Agents treat questions differently, and the split is consistent. On SAML SSO, agents mostly reported “not found” when the official site was silent. On card policy, they mostly answered, inferring from pricing structure (“fees come out of payments, so no card is needed”). The more a question looks inferable, the more confidently the agent fills it in.
That’s the risk in plain terms: the less your page says, the more the agent writes for you. Hi Violet tracks this as the “Asked” mark in a run, a step the agent cleared only by guessing or by asking something the public material doesn’t answer. Our growth signal stack makes the same argument about measurement: if a fact matters to the decision, record it where it can be checked.
Does llms.txt help agents answer product questions?
It helps coding agents find the right page. It doesn’t write answers your docs are missing.
Mintlify ran 2,400 Claude Code and Codex runs against 20 docs sites: dead-URL hits per task fell from 2.23 on HTML docs to 0.11 on markdown docs with an llms.txt link. That’s a navigation win. Ahrefs, looking at 137,210 domains, found 97% of llms.txt files got zero requests in May 2026, and AI bots never asked for an llms.txt that didn’t exist. llms.txt is a map that coding agents follow when something points them to it. It is not a citation tactic, which is how our AI search work treats it too.
In this panel, 29 of 30 products publish an answer about their MCP server, CLI or llms.txt. The agent-facing plumbing is mostly in place. The content on the other end of it is where the gaps are: an llms.txt that indexes a pricing page with no card policy still leads the agent to a page with no card policy.
Format still matters at the margin. Cloudflare’s scan of the top 200,000 domains found markdown content negotiation on 3.9% of sites. Serving clean markdown makes an answer cheaper for the agent to read. It has to exist first.
How do you make your docs answer the questions agents get asked?
Publish the answers, one fact per sentence, on official pages. The work is mostly writing, not engineering.
- Write down the 12 questions for your product. Start with the bank in this study. Add the three your sales team hears most.
- Answer each one on an official page, in one plain sentence. “A card is not required before the first API call on the free plan.” “SAML SSO is available on the Enterprise plan.” “Customers can choose the EU region (Frankfurt) at signup.”
- Say “no” when the answer is no. “There is no EU-only data option” is an answer. Silence is an invitation to infer.
- Put commercial facts where agents look. Pricing pages and docs, not PDFs, gated sales decks or images of tables.
- Serve the pages agents can fetch. No JavaScript-only rendering for pricing, markdown versions where you can, and an llms.txt that points at the pricing and limits pages.
- Re-test with real agents. Ask the same questions with Claude Code and Codex after the change. A fact you published should now come back as a quote, not an inference.
For buyer-side access to private material, such as a security questionnaire or an enterprise price sheet, give the evaluating agent temporary access to the real document instead of leaving it to guess. For signup questions, the same principle applies to letting agents create accounts.
How Hi Violet tests product Q&A for agents
Hi Violet runs this study on your product. We build the question bank from your buyers’ real questions, capture what your official pages say with verbatim quotes, and run Claude Code, Codex and the assistants your buyers use against it, several times each, in sandboxed sessions. Two independent judges grade every answer. You get the gap list, ranked by how often agents guessed, the exact sentences to publish, and a re-run after the fix that shows each gap closed.
It’s part of our agent experience service, and the runs use the method on our how it works page. To see your own gap list, request access.
Frequently asked questions
Can AI agents answer questions about my product?
Yes, when your own pages state the answer. Across 30 developer products, the companies’ official sites left 79 of 360 buyer answers unstated. Agents answered many of those questions anyway, from privacy policies, legal terms, forums and inference. Publishing the fact on an official page turns a guess into a quote.
Which product questions do agents get wrong most often?
The ones your site never answers. EU data residency went unanswered on 19 of 30 official sites, SAML SSO on 14, and whether a card is needed before the first API call on 13. Those are the questions where agents fill in the answer themselves.
Does llms.txt help AI agents answer questions?
It helps coding agents find pages. Mintlify measured dead-URL hits per task falling from 2.23 to 0.11 with markdown docs plus an llms.txt link. It doesn’t create an answer your docs never wrote, and Ahrefs found 97% of llms.txt files got no requests at all.
What should a pricing page for AI agents include?
Plain statements of the facts agents get asked: free-tier limits, the first paid price, what happens over the limit, whether a card is needed before the first API call, default rate limits, which plan includes SSO, and which data regions customers can choose. One fact per sentence, in text, on a page that renders without JavaScript.
How do you test whether AI agents can answer questions about a product?
Write a fixed bank of buyer questions. Have several agents answer them with their own web search, across repeated runs that start from scratch. Check each answer against verbatim quotes from the company’s own pages, recheck any gap an agent cites a page for, and grade with two independent judges.
Why do agents answer when the official site says nothing?
They’re built to finish the task. When the official page is silent, an agent reads privacy policies, legal terms, community forums and third-party lists, then infers an answer. Two agents reading the same evidence often land differently, which is why the fix is publishing the fact yourself.


