# Agent-Adoption Check: search.google.com

**Score:** 33 / 100
**Level:** L1 — Basic Web Presence
**Scanned:** 2026-05-02 (scanner v0.8.0, spec v1.0.0, profile b2b-saas)
**Permalink:** https://respectarium.com/agent-adoption-check/search-google-com

> This is one implementation of the open Agent-Adoption Specification. The specification is open; anyone can build their own implementation.

## Next Level Gates

To advance to L2 (AI-Aware), pass:

- **content-signals** — Content-Signal (Cloudflare) or AIPREF Content-Usage directives declare what AI systems may do post-fetch — train, search-index, or generate answers.
  - Current: Skipped — dependency robots-txt-exists did not pass

## Categories

### Discoverability — score 0 (0 pass · 2 fail · 1 neutral)

> Can agents find the signals they need to navigate your site? Covers robots.txt, sitemap, and Link headers — the first three signals any crawler looks for.

- [FAIL] **robots-txt-exists** (weight 7) — Non-2xx response (404)
  - robots.txt is the first file crawlers and agents check for access rules; silence defaults to blanket-allow. Per RFC 9309.
  - Spec: https://www.rfc-editor.org/rfc/rfc9309
- [FAIL] **sitemap-exists** (weight 4) — No valid sitemap found at any probed path
  - An XML sitemap is the route map agents use to find your pages. Without one they link-walk and miss deep or orphaned content.
  - Spec: https://www.sitemaps.org/protocol.html
- [NEUTRAL] **link-headers** (informational) — Homepage returned no Link header — v1 does not penalize
  - Link: response headers expose related resources — API catalogs, service docs, alternates — before an agent parses HTML. Per RFC 8288.
  - Spec: https://datatracker.ietf.org/doc/html/rfc8288

### Access Control — score 0 (0 pass · 0 fail · 1 informational · 3 neutral)

> Are you telling AI systems what they may and may not do with your content? Per-bot rules, usage signals (Content-Signal / AIPREF), and cryptographic bot authentication.

- [NEUTRAL] **ai-bot-rules** (weight 7) — Skipped — dependency robots-txt-exists did not pass
  - Per-bot robots.txt rules (GPTBot, ClaudeBot) or Content-Signal directives declare who may train on or cite your content. Missing rules allow all.
  - Spec: https://www.rfc-editor.org/rfc/rfc9309, https://developers.cloudflare.com/bots/additional-configurations/content-signals/, https://datatracker.ietf.org/doc/draft-ietf-aipref-vocab/, https://platform.openai.com/docs/bots/, https://docs.anthropic.com/en/docs/agents-and-tools/usage-policies, https://developers.google.com/search/docs/crawling-indexing/google-common-crawlers
- [NEUTRAL] **content-signals** (informational) — Skipped — dependency robots-txt-exists did not pass
  - Content-Signal (Cloudflare) or AIPREF Content-Usage directives declare what AI systems may do post-fetch — train, search-index, or generate answers.
  - Spec: https://developers.cloudflare.com/bots/additional-configurations/content-signals/, https://datatracker.ietf.org/doc/draft-ietf-aipref-vocab/, https://datatracker.ietf.org/doc/draft-ietf-aipref-attach/
- [FAIL] **web-bot-auth** (informational) — No Web Bot Auth directory published at /.well-known/http-message-signatures-directory
  - If you operate an AI crawler, a signing-key directory lets targets verify it's you. Most sites don't publish — presence-only check. IETF draft.
  - Spec: https://datatracker.ietf.org/doc/html/draft-meunier-http-message-signatures-directory-05, https://datatracker.ietf.org/doc/draft-meunier-web-bot-auth-architecture/
- [NEUTRAL] **robots-allow-all** (informational) — Skipped — dependency robots-txt-exists did not pass
  - A blanket-allow posture (wildcard User-agent, Allow: /, no cross-bot blocks) declares that every crawler is welcome. Informational — no pass/fail.
  - Spec: https://www.rfc-editor.org/rfc/rfc9309

### Content Readability — score 76 (3 pass · 1 fail · 1 informational · 7 neutral)

> Once an agent is inside, can it actually read what it fetches? Covers markdown availability, page size, rendering strategy, and llms.txt.

- [NEUTRAL] **llms-txt-exists** (informational) — No llms.txt found at primary or fallback path
  - An llms.txt file gives agents a curated entry point into your docs — sitemap-equivalent, but sized for context windows. Per llmstxt.org.
  - Spec: https://llmstxt.org/
- [NEUTRAL] **llms-txt-valid** (informational) — llms-txt-valid cannot evaluate without llms.txt body
  - A well-formed llms.txt (H1 title, summary blockquote, linked sections) parses cleanly; a malformed one is skipped silently — worse than no file. Per llmstxt.org.
  - Spec: https://llmstxt.org/
- [NEUTRAL] **llms-txt-size** (informational) — llms-txt-size cannot evaluate without a discovered llms.txt
  - llms.txt must fit in an agent's context window alongside the user's question. Under 50K is comfortable; over 100K gets truncated mid-section.
  - Spec: https://llmstxt.org/, https://agentdocsspec.com/spec/#llms-txt-size
- [NEUTRAL] **llms-txt-has-optional-section** (informational) — llms-txt-has-optional-section cannot evaluate without llms.txt body
  - Reports the shape of your llms.txt — Optional section, H2 count, link count — so you can tell at a glance whether agents get a skeleton or a full map.
  - Spec: https://llmstxt.org/
- [NEUTRAL] **markdown-url-support** (weight 8) — No sitemap URLs available to sample — cannot evaluate markdown support
  - A .md twin alongside each HTML page (page.md ↔ page.html) gives agents an agent-readable fetch path — no HTML, ad wrappers, or modals.
  - Spec: https://llmstxt.org/, https://developers.cloudflare.com/fundamentals/reference/markdown-for-agents/, https://nextjs.org/docs/app/building-your-application/routing/route-handlers, https://agentdocsspec.com/spec/#markdown-url-support
- [FAIL] **markdown-negotiation** (weight 5) — Server ignored Accept: text/markdown — returned HTML instead
  - Accept: text/markdown negotiation serves HTML to humans and agent-readable markdown to agents from one URL — no duplicate-URL strategy.
  - Spec: https://llmstxt.org/, https://developers.cloudflare.com/fundamentals/reference/markdown-for-agents/
- [PASS] **rendering-strategy** (weight 10) — Server-side rendering confirmed
  - Classifies the site as server-rendered, hydrated, or client-rendered (SPA) — what agents see without running JavaScript. A pure SPA reads as blank.
  - Spec: https://agentdocsspec.com/spec/#rendering-strategy
- [NEUTRAL] **page-size-html** (weight 6) — No sitemap URLs available to sample — cannot evaluate page size
  - Measures how much markdown each page feeds into an agent's context window. Under 50K fits cleanly; over 100K truncates mid-page — pages have context budgets too.
  - Spec: https://agentdocsspec.com/spec/#page-size-html
- [PASS] **http-status-codes** (weight 6) — Correct HTTP 404 returned for non-existent path
  - Soft-404s (HTTP 200 on a missing page) make agents cache garbage as canonical content. An honest 4xx tells agents the URL is dead — drop it.
  - Spec: https://agentdocsspec.com/spec/#http-status-codes
- [NEUTRAL] **redirect-behavior** (weight 4) — No sitemap URLs available to sample — cannot evaluate redirect behavior
  - Same-domain HTTP 3xx redirects work for agents. JavaScript redirects break agents without JS; cross-domain jumps read as tracking.
  - Spec: https://agentdocsspec.com/spec/#redirect-behavior
- [FAIL] **agents-md-detection** (informational) — AGENTS.md not found at /AGENTS.md — HTTP 404 response
  - AGENTS.md is a coding-agent convention. ETH Zurich research (2026) found it often hurts those agents; we track presence to test the effect on websites. Informational.
- [PASS] **cache-header-hygiene** (informational) — Homepage response carries 1 of Cache-Control / ETag / Last-Modified: cache-control
  - Cache-Control, ETag, and Last-Modified headers let agents re-fetch only what changed — missing headers force full re-downloads. Informational.
  - Spec: https://www.rfc-editor.org/rfc/rfc7234

### Agent Endpoints — score 0 (0 pass · 2 fail · 3 informational · 1 neutral)

> Do you expose what agents need to call you programmatically? Covers MCP Server Cards, A2A Agent Cards, Agent Skills, and OAuth discovery — the surfaces that turn a site into an API for agents.

- [FAIL] **api-catalog** (informational) — No api-catalog published at /.well-known/api-catalog
  - A /.well-known/api-catalog (RFC 9727) points agents at your OpenAPI specs and developer docs in one fetch — the discovery surface for programmatic integration.
  - Spec: https://www.rfc-editor.org/rfc/rfc9727, https://www.rfc-editor.org/rfc/rfc9264
- [NEUTRAL] **oauth-discovery** (weight 6) — no OAuth surface detected — scan inconclusive
  - OAuth discovery metadata at /.well-known/oauth-authorization-server lets agents locate your auth endpoints without reading docs. Per RFC 8414.
  - Spec: https://www.rfc-editor.org/rfc/rfc8414, http://openid.net/specs/openid-connect-discovery-1_0.html
- [FAIL] **oauth-protected-resource** (weight 7) — No OAuth Protected Resource metadata at /.well-known/oauth-protected-resource
  - Protected Resource metadata identifies which authorization server protects your API. Paired with oauth-discovery, agents complete auth without reading docs. Per RFC 9728.
  - Spec: https://www.rfc-editor.org/rfc/rfc9728
- [FAIL] **mcp-server-card** (weight 10) — No MCP Server Card found at any well-known path
  - An MCP Server Card advertises your Model Context Protocol endpoint to Claude, Cursor, and other MCP clients — the standard discovery path.
  - Spec: https://github.com/modelcontextprotocol/modelcontextprotocol/pull/2127, https://github.com/modelcontextprotocol/modelcontextprotocol/pull/2525, https://modelcontextprotocol.io/community/server-card/charter
- [FAIL] **a2a-agent-card** (informational) — No A2A Agent Card published at /.well-known/agent-card.json
  - An A2A Agent Card at /.well-known/agent-card.json describes your service to other agents — capabilities, skills, calling conventions. Per A2A v1.0.
  - Spec: https://a2a-protocol.org/latest/specification/, https://a2a-protocol.org/latest/topics/agent-discovery/
- [FAIL] **agent-skills** (informational) — No Agent Skills index found at primary or fallback path
  - An Agent Skills index at /.well-known/agent-skills/ exposes your capabilities as discrete, individually-addressable skills. Agent Skills v0.2.
  - Spec: https://github.com/cloudflare/agent-skills-discovery-rfc, https://agentskills.io/, https://schemas.agentskills.io/discovery/0.2.0/schema.json

## Methodology

- Checks evaluated: 25
- Scan duration: 29.1s
- Outbound HTTP probes: 18 (success 1, not-found 17)
- Browser-rendered fetches: 2
- Scanner version: 0.8.0
- Spec version: 1.0.0
- Profile: b2b-saas

## Citation

Respectarium. (2026-05-02T04:25:19.896Z). Agent-Adoption Check: search.google.com [Scan]. https://respectarium.com/agent-adoption-check/search-google-com