Academy

AI Accessibility Audit for Agent Readiness

Run an AI accessibility audit covering crawler access, extraction, structured data, llms.txt, speed, WebMCP, and agentic commerce readiness across key pages.

16 min read

AI systems cannot quote, recommend, or act on content they cannot reliably retrieve. This phase tests that foundation before the team measures visibility or commissions content intended for answer engines.

Phase: P4, Stage A — Understand. Timebox: 3–5 working days for a typical marketing site; 5–10 for a large ecommerce, marketplace, or heavily client-rendered application. Owner: the technical SEO lead is accountable, with engineering running fetch and rendering tests, the content lead reviewing extractability, and a business or legal owner deciding crawler policy.

Why this phase comes here

A conventional technical audit asks whether search engines can crawl, render, index, and rank the site. This phase asks whether AI crawlers, retrieval systems, and task-oriented agents can reach and interpret the same useful information. They may use different user agents, fetch paths, rendering capabilities, timeouts, and extraction methods. An indexable page can still return an empty shell, consent wall, bot challenge, or disconnected fragments to an AI client.

An AI accessibility audit therefore belongs beside the technical baseline audit , not at the end of content production. Client-side rendering means that JavaScript constructs important content after the initial HTML arrives. Retrieval systems may not execute that code, may stop before it completes, or may extract the initial response only. If the product name, answer, price, availability, or evidence exists only after JavaScript runs, later content optimization cannot repair the access failure.

This phase consumes the verified accounts, analytics, crawler logs, sitemap sources, critical URL set, and ownership map from the access, tracking, and data-sources phase . Running it earlier leaves the team unable to distinguish a genuine absence from missing access. Running it after baseline measurement contaminates the baseline: zero citations may reflect an inaccessible site, not weak content or demand.

Crawler policy is a business decision
Allowing an AI crawler may improve discovery, retrieval, and the chance of citation. Blocking it may protect licensed content, reduce unapproved reuse, control infrastructure cost, or satisfy customer and regulatory obligations. The audit does not choose for the business. It exposes the trade-off, names the decision owner, and verifies that the live configuration matches the recorded decision.

Skipping the phase wastes work: writers improve passages a crawler never receives, developers add schema behind a challenge, or the team mistakes a valid llms.txt for site-wide readiness. Access, extraction, understanding, and action are separate capabilities.

Inputs and outputs

The inputs make tests reproducible. The outputs form a contract with P5: measurement begins only after known access failures are fixed or explicitly accepted.

DirectionItemAcceptance condition
InputP2 access and ownership packIncludes production access, analytics and log sources, robots and CDN owners, legal/business policy owner, and escalation route.
InputRepresentative URL setIncludes the homepage plus at least one high-value URL for every important template: product, category, service, article, documentation, location, and transactional page where applicable.
InputCrawler policy matrixLists relevant crawler families, current rule, intended rule, decision owner, rationale, and review date; unknown intent is recorded as unknown, not “blocked by policy.”
InputP3 technical findingsSupplies canonical, status, rendering, sitemap, performance, and structured-data evidence so this phase can isolate AI-specific behavior.
InputEntity and offer factsNames the canonical organization, products or services, alternate names, official URLs, and facts an extractor must identify correctly.
OutputTest evidence packStores timestamp, URL, user agent, response status, response headers, initial HTML or tree evidence, and screenshots for each test.
OutputPolicy decision recordShows allow, block, or conditional access for each crawler family, with an accountable approver and implementation check.
OutputAgent-readiness findings registerGives every failed condition severity, affected scope, evidence, recommendation, owner, effort, dependency, and retest date.
OutputP2 priority-list updateMerges agent findings into the existing cross-functional backlog instead of creating a separate “AI SEO” queue.
OutputP5 readiness noteStates which limitations would distort baseline measurement and whether P5 may proceed, proceed with annotations, or wait.
Logo

Ready to Monitor Your AI Visibility?

Track how AI chatbots mention your brand across ChatGPT, Perplexity, and other platforms.

The checklist

Test the representative URL set, not only the homepage, and preserve reproducible evidence.

1. Decide crawler access deliberately

What to do: inventory AI-crawler rules in robots.txt , CDN bot controls, web application firewalls, consent layers, and origin configuration. Why it matters: a robots allow is only a preference; an edge service can still block the request, while an accidental wildcard block is not policy. How to do it: compare live rules with the policy matrix, fetch representative URLs with applicable user agents, and record the trade-off. Tool: AmICited Robots.txt & Sitemaps, an approved request client, and CDN/origin logs. Done when: every crawler family has an approved allow, block, or conditional decision and live behavior matches it.

2. Compare the non-JavaScript response with the useful page

What to do: compare initial HTML with the normal browser page. Why it matters: a retrieval client may not run scripts that insert answers, offers, links, or product data. How to do it: test every important template before hydration, the process that attaches application behavior to server HTML. Tool: a no-script browser or HTTP client plus the rendered page. Done when: the initial response contains the title, H1, primary content, core facts, and discovery links; any exception has evidence and an owner.

3. Test rendered-DOM extractability

What to do: extract main content from the rendered Document Object Model (DOM), the browser’s structured page representation, without navigation, consent text, or hidden variants. Why it matters: receiving text is not the same as identifying the correct text; template noise can corrupt retrieval. How to do it: compare extracted title, answer, publisher, dates, facts, and links with the visible source across long, sparse, and commercial pages. Tool: browser inspection, text extraction, and page source. Done when: the record preserves primary meaning and facts without visual position or undocumented selectors.

4. Inspect the accessibility tree

What to do: review the accessibility tree: roles, names, states, headings, landmarks, and controls. Why it matters: semantics distinguish headings from decoration, buttons from icons, and primary content from navigation. How to do it: test critical templates for one H1, ordered headings, named controls, landmarks, descriptive links, and meaningful image alternatives. Tool: AmICited Accessibility Tree checker and browser inspection. Done when: the score meets the threshold, no critical control lacks a name, and the main region and next action are identifiable.

5. Validate structured-data coverage and truthfulness

What to do: compare visible facts with structured data , standardized markup such as Schema.org JSON-LD. Why it matters: markup clarifies entities, offers, authorship, and dates, but inaccurate markup communicates the wrong fact. How to do it: validate applicable schemas and reconcile names, URLs, prices, currency, availability, dates, ratings, and identifiers with source systems. Tool: validator, rendered HTML, and source records. Done when: critical templates have valid applicable markup, values match visible facts, and every material warning is resolved or explained.

6. Review llms.txt as a guide, not a gate

What to do: check whether /llms.txt accurately summarizes the organization and links to canonical public resources. It is a proposed plain-text guide, not access control or a guaranteed ranking signal. Why it matters: a concise map reduces ambiguity; a stale one misdirects agents. How to do it: verify status, Markdown structure, description, destinations, canonical URLs, and owner. Tool: AmICited llms.txt review and direct fetch. Done when: the file, if used, has no broken/private links and a maintenance owner; absence is an improvement finding, not proof of invisibility.

7. Sample self-contained passages

What to do: review independently retrieved definitions, answers, facts, comparisons, steps, and limitations. Why it matters: retrieval can separate a paragraph from its heading; “it works better for them” then loses its subject and comparison. How to do it: read at least 20 passages without their title or previous paragraph. Tool: extraction output and editorial review. Done when: each names its subject, answers an identifiable question, preserves conditions or units, and avoids unresolved references.

8. Confirm canonical entity clarity

What to do: verify that the site states who the organization is, what it offers, and how its brands, products, people, locations, and profiles relate. A canonical entity is the primary real-world thing a name refers to. Why it matters: inconsistent names, old logos, conflicting descriptions, and disconnected profile URLs make it easy to merge two entities or split one entity into several. How to do it: compare the homepage, About page, contact details, structured data, social profiles, author pages, legal name, and important third-party profiles. Tool: entity fact sheet, rendered pages, and structured-data output. Done when: one approved fact sheet resolves official name, aliases, canonical URL, logo, description, ownership, primary offers, and same-entity profiles, with conflicts logged for correction.

9. Test response time and fetch reliability

What to do: measure status, Time to First Byte (TTFB), redirects, timeouts, and response consistency under normal and relevant crawler user agents. TTFB is the interval from request start until the first response byte arrives. Why it matters: intermittent 403s, 429s, 5xx responses, long redirect chains, or slow origins make content unreliable even when a single browser visit succeeds. How to do it: run the repeatable sample defined in the threshold table from more than one network location when geography matters, then reconcile failures with logs and Core Web Vitals. Tool: request monitor, CDN/origin logs, and AmICited Web Vitals. Done when: critical URLs satisfy the reliability and latency thresholds or have a severity, cause, owner, and retest date.

10. Assess WebMCP and commerce protocols where they create value

What to do: test WebMCP and commerce readiness only where the business model supports agent actions. WebMCP is an emerging way for a website to expose callable tools, such as search, booking, or adding an item to a cart. Agentic commerce covers agent-assisted product discovery and transactions, including protocols such as ACP or UCP. Why it matters: readable content supports answers; explicit tools and commerce data support reliable action without screen scraping. How to do it: map valuable user tasks, inspect declared tools or protocols, validate input and output descriptions, and verify confirmation, authentication, permission, price, inventory, and error behavior. Tool: AmICited WebMCP and Agentic Commerce checks plus a controlled test environment. Done when: applicable capabilities are detected and safely testable, or the check is marked not applicable with an approved business-model reason and review trigger.

Tools in AmICited

Open https://app.amicited.com/accessibility for the consolidated audit and the AI Accessibility and Agent Readiness feature explanation. The product reports independent readings rather than hiding different failure types inside one blended score.

  1. Use the summary to check the Agent Accessibility Score and record each component, not just the headline state.
  2. Use Robots.txt & Sitemaps to check robots.txt and sitemap coverage , then compare the stated rule with a live crawler-style fetch.
  3. Use the file review to review llms.txt and open every listed destination.
  4. Use the page checker to inspect a page’s accessibility tree for each critical template.
  5. Use the readiness checks to check WebMCP readiness and check agentic commerce readiness when those capabilities apply.
  6. Open https://app.amicited.com/audit/web-vitals to check Core Web Vitals and connect field performance with crawler-condition request tests.

Decision rules

These are operational acceptance thresholds, not claims about ranking algorithms. Tighten them for revenue-critical journeys or regulated content, and record any alternative before testing so the result is not adjusted after the fact.

CheckPass or acceptableFinding thresholdDefault action
Policy ownershipEvery relevant crawler has allow, block, or conditional status, rationale, approver, and review dateAny live rule has no owner or documented intentMajor; escalate the business decision within 2 working days
Stated versus effective accessLive behavior matches the approved policy on every critical URLAllowed crawler receives a 401, 403, 429, 5xx, challenge page, or materially different contentCritical on critical URLs; Major elsewhere
Non-JavaScript responseTitle, H1, primary content, core facts, and crawlable discovery links are presentAny required element exists only after JavaScript, or initial HTML is an empty application shellCritical for primary content; Major for supporting content
Rendered extractionExtracted title, answer or offer, facts, dates, and primary links match the visible pageWrong variant, hidden text, navigation noise, or missing qualifying context changes meaningCritical if facts change; Major if extraction is incomplete
Accessibility treeAmICited score 80–100 and no unnamed critical control or broken main-content outline50–79 is Major; below 50 is Critical; any unusable purchase, booking, login, or lead control is Critical regardless of scoreRepair semantics and retest the affected template
Structured dataZero syntax errors; material properties match visible content and source recordsAny invalid required property or conflicting price, availability, date, identity, rating, or canonical URLCritical for deceptive/conflicting facts; Major for missing applicable coverage
llms.txtIf present: HTTP 200, readable Markdown, accurate summary, zero broken/private links, named ownerMissing file is Advisory; invalid, stale, redirected, or misleading file is MajorCreate or correct after access and extraction blockers
Passage qualityAt least 20 sampled passages; all identify subject and retain conditions, units, and answerOne ambiguous passage is Major for that page; repeated template-wide ambiguity is Critical to the content patternCorrect the pattern, then resample 20 passages
Entity clarityApproved fact sheet matches critical pages and machine-readable identityConflicting official name, canonical URL, ownership, product relationship, or same-entity referenceMajor; Critical when the conflict changes who provides the offer or advice
Fetch reliability25 requests per critical template over at least 2 test periods: 100% valid 2xx after expected redirects, no challenge pages, and at least 98% valid responses across the wider sampleAny critical-URL failure, or wider sample below 98% valid responsesCritical for critical URLs; Major for wider reliability
TTFBMedian at or below 800 ms and 95th percentile at or below 1,800 ms in the test environmentMedian above 800 ms is Major; any repeated timeout or 95th percentile above 1,800 ms is Critical for affected critical URLsDiagnose CDN, origin, caching, redirects, or regional routing
RedirectsZero unexpected hops; no more than one intentional same-site hop before a 200 responseLoop, cross-domain surprise, crawler-specific redirect, or two or more avoidable hopsCritical for loop or wrong destination; Major for excess hops
WebMCPApplicable tools are declaratively exposed, accurately described, permissioned, and testedJavaScript-only detection is unverified; missing applicable tool or unsafe action is a findingMajor for missing applicable capability; Critical for unsafe execution
Agentic commerceApplicable protocol is advertised and test flow preserves price, inventory, consent, confirmation, and error handlingUnsupported capability is honestly absent, or an advertised flow changes terms or acts without confirmationNot applicable is acceptable; unsafe or misleading flow is Critical

Scores never override concrete evidence. An accessibility score of 85 does not excuse an unlabeled checkout button, and a robots allow does not outweigh a challenge page returned to the real request. “Not checked” is unknown, not a pass and not a zero.

Deliverable: the agent-readiness findings register

Hand over one findings register, not a presentation and a separate AI backlog. Add each finding to the P2 priority list with these fields:

ID and title:
Affected URLs/templates:
Check and observed condition:
Expected condition/threshold:
Evidence: timestamp, user agent, status, capture or log reference
Business consequence:
Severity: Critical | Major | Advisory
Recommended action:
Owner and approver:
Effort and dependency:
Due date and retest date:
Policy decision, if relevant:
P5 measurement impact:
Status: Open | Accepted risk | Fixed | Verified

Critical means the failure prevents reliable access, materially changes extracted meaning, or permits an unsafe action. Major means access or understanding is degraded but a representative client can still recover the main content. Advisory means a useful improvement lacks evidence of present failure. Accepted risk requires the named business owner, rationale, affected scope, expiry or review date, and a way to detect changed conditions.

Deduplicate by root cause: one CDN challenge affecting conventional and AI crawlers is one item with multiple evidence records.

What goes wrong

Treating the phase as optional. Prompt tracking and content rewrites cannot compensate for failed retrieval. Make P4 an entry condition for an interpretable baseline.

Blocking by default and retroactively calling it policy. A rule with no decision owner, rationale, or review date is configuration, not policy. Present the discovery-versus-control trade-off and obtain an explicit decision.

Adding llms.txt and declaring completion. The file cannot override robots rules, CDN blocks, empty initial HTML, misleading markup, weak passages, or timeouts. Treat it as one guide within the wider evidence set.

Testing only a friendly user agent or the homepage. Edge controls vary by path, geography, rate, and identity. Test every high-value template and user agent in the policy matrix.

Confusing appearance with extractability. A polished page can expose hidden duplicates, meaningless control names, or a blank no-script response. Preserve source, DOM, accessibility-tree, and extracted-text evidence separately.

Treating unknown as failure or success. Retest timeouts and unchecked crawlers; never convert missing evidence into a convenient score.

Installing experimental agent capabilities without a use case. WebMCP or commerce protocols should expose valuable, permissioned actions. Shipping an unsafe or inaccurate action is worse than marking the capability not applicable.

Handoff to baseline measurement

The baseline measurement phase receives the merged priority list, test evidence pack, crawler-policy record, representative URL set, and readiness note. The P4 owner must identify any limitation that changes interpretation: blocked crawler families, inaccessible templates, intermittent regions, missing passages, or a recent fix whose effect has not propagated.

P5 may proceed when critical URLs are intentionally accessible to the crawler families included in measurement, valid primary content is extractable, and no unresolved Critical finding would make a zero or low score uninterpretable. It may proceed with annotations when a deliberate block excludes a known crawler or a Major issue affects a bounded template. It should wait when access intent is unknown, critical pages fail fetches, or extracted content materially contradicts the visible source.

The handoff is complete when the P5 owner can answer three questions without reopening the audit: which agents were intended to have access, which pages and facts they could reliably retrieve, and which known limitations must appear beside the baseline.

FAQ

Frequently asked questions

Should we allow every AI crawler?
Not automatically. The business owner must weigh discoverability and citation opportunities against content licensing, competitive reuse, server cost, privacy, and contractual duties. Record an explicit decision for each crawler family and test that implementation matches it.
Is an llms.txt file required to pass the audit?
No. llms.txt is a useful discovery aid, not proof that crawlers can fetch or extract the site. A missing file is an improvement finding; blocked pages, failed fetches, or unusable primary content are more severe.
Can a site pass technical SEO checks and still fail agent readiness?
Yes. Search crawlers may receive server-rendered HTML while an AI user agent receives a challenge page, or the page may rely on JavaScript, ambiguous entities, and interactions that retrieval systems cannot reliably interpret.
Do WebMCP and agentic commerce apply to every business?
No. Test WebMCP when agents could perform useful actions such as search, booking, quoting, or account tasks. Test commerce protocols when products can be discovered and purchased. Mark either check not applicable with a business-model reason.
Where do agent-readiness findings go?
Merge them into the priority list established in P2. Use the same severity, owner, due date, evidence, and dependency fields so technical, measurement, and AI-accessibility work compete in one backlog.
Find out whether AI agents can use your site
Run the accessibility audit, preserve the evidence, and turn every failed condition into an owned priority.

← All Academy tutorials

Ready to put it into practice?

Free check · 7-day trial · no credit card