13 min read
Before the fetch: how AI agents discover what they can do with your site
Long before an agent's harness fetches your page, it already asked two other questions elsewhere - is it allowed here, and what else is available. A tour of robots.txt, Content Signals, llms.txt, .well-known, MCP, A2A, and the newer ARD discovery layer.