Public methodology

The AgentView Standard

Agent-readability standards change constantly. AgentView's scoring is versioned and public so a score change is always explained by a documented standards update, never a mystery.

Current standard

Current standard: 2026-07-23.1

Added three agent-discovery signals — sitemap, AGENTS.md, and a robots.txt Content-Signal directive — and rebalanced the scored set to keep it at 100.

The checks

Each check measures a distinct part of whether an AI agent can reach, parse, and cite your site.

The checks form a funnel: an agent must Reach you before it can Read you, and Read you before it can Understand you.

CheckWeightWhat it checks
ReachCan AI agents get to you at all?
Agent reachability28Does an agent-style request reach your content, or hit a WAF/CAPTCHA challenge?
Robots retrieval policy12Are AI retrieval bots allowed in robots.txt? (Blocking them removes citation eligibility.)
Cloudflare AI-bot exposure8Could a Cloudflare edge setting block AI crawlers by default?
Sitemap3Is a /sitemap.xml (or robots.txt Sitemap: directive) present so crawlers can enumerate your pages?
Content-SignalBonusDoes robots.txt declare usage preferences for search, AI answers, and training? (Emerging; shown, not scored.)
ReadCan they read you cleanly once they're in?
Markdown negotiation20Do you serve compact text/markdown to agents, with Vary: Accept?
Token efficiency10How many tokens of markup does an agent pay to read your HTML vs your actual words?
Static rendering12Is your main content in the server HTML, or only after JavaScript runs?
Content substanceBonusDo your first ~380 words carry concrete facts an AI can quote? (Shown but not scored.)
UnderstandCan they parse your entity and use you as a tool?
Structured data quality4Does your JSON-LD ground a real entity with a stable @id and sameAs list?
llms.txt2Is a machine index present? (Honest hygiene; near-zero real consumption today.)
MCP discovery1Do you publish a forward-looking MCP server card?
AGENTS.mdBonusDo you publish an emerging plain-Markdown agent guide? (Shown, not scored.)

The ten scored checks sum to 100; content substance, Content-Signal, and AGENTS.md are bonus signals that appear in the report but do not affect the score.

Two checks (agent reachability, Cloudflare exposure) depend on the datacenter IP we probe from, so they are excluded from the public leaderboard score.

Changelog

This is the honest record of what moved between versions, newest first.

2026-07-23.1

Added three agent-discovery signals — sitemap, AGENTS.md, and a robots.txt Content-Signal directive — and rebalanced the scored set to keep it at 100.

  • addedSitemap — checks for a real /sitemap.xml (or a Sitemap: directive in robots.txt) so indexing and training crawlers can enumerate your pages. Scored in the Reach pillar; weight 3.
  • addedAGENTS.md — checks for an emerging plain-Markdown agent guide. Shipped as a bonus signal (shown, not scored) — adoption is still early.
  • addedContent-Signal — checks whether robots.txt declares content-usage preferences (search / AI answers / AI training). Bonus signal, reflecting Cloudflare's emerging Content Signals convention.
  • changedRebalanced the scored set to keep it at 100 while adding the sitemap check: structured data 6→4 and llms.txt 3→2 (both low-lever hygiene). No site's underlying readiness changed — only the standard.

2026-07-16.4

Added a content-substance signal and upgraded structured-data grading to check entity quality, not just presence.

  • addedContent substance — measures whether your first ~380 words carry concrete, quotable facts (the amount an AI typically extracts) rather than vague marketing copy. Shipped as a bonus signal: it appears in the report but does not affect the 0–100 score.
  • changedStructured data now grades quality, not just presence: a full pass requires a grounded Organization entity with a stable @id and a sameAs list of verified profiles. Present-but-thin JSON-LD earns partial credit.

2026-07-16.3

Added a static-vs-rendered check for pages that only appear after JavaScript runs.

  • addedStatic rendering — flags pages whose main content is built in the browser, which an agent that reads raw HTML sees as an almost-blank shell.

2026-07-16.2

Expanded from five checks to eight and reframed robots by citation cost.

  • addedToken efficiency — the markup tax an agent pays to read your HTML versus your actual words.
  • addedCloudflare AI-bot exposure — detects when a Cloudflare edge setting could block AI crawlers by default.
  • addedMCP discovery — checks for a forward-looking MCP server card.
  • changedRobots.txt now classifies bots by citation cost: blocking retrieval bots removes citation eligibility, while blocking training-only bots costs nothing. Control tokens with no crawler behind them are no longer alarmed on.
  • changedMarkdown negotiation now credits serving Markdown even when Vary: Accept is missing, and reads token-delta headers when present.

2026-07-16.1

Initial AgentView standard — five deterministic agent-readability checks.

  • addedAgent reachability, Markdown content negotiation, robots.txt retrieval-bot policy, Schema.org structured data, and llms.txt presence.