· ai agents

AI agents visit your site daily. What policy do AI agents find there?

AI agents read your site before your customers do: crawling it for training, fetching it to answer questions, and increasingly transacting on it. Every one of those visits happens under rules you never published. robots.txt, written for 1994 crawlers, can say 'stay out'. It cannot say 'agents may read prices, may not scrape reviews, and may transact up to this limit with attribution'.

AgentGate is Ingenious AI's machine-readable agent-access policy standard: a file at /.well-known/agent-policy.json declaring which AI agents may use your site, what they may do, and on what terms. This check scores what an agent can learn about your rules today.

Honest scope, read this first: this page is about declared policy, not enforcement. A policy file tells honest agents your terms and gives you a published standard to point at in a dispute; blocking agents that ignore it is a separate, enforcement problem. The rival agent.json draft is real (Agent Web Protocol, draft v0.2); standards here are young and competing, and this page says so.

AI agents and the standards race over one small file

Two proposals now chase the same slot in the /.well-known/ directory. The Agent Web Protocol's agent.json (draft v0.2) pitches itself as the agent-era successor to robots.txt. AgentGate's agent-policy.json takes the policy angle: named agents, permitted actions, terms and a contact, in a file built for the questions site owners actually get asked after an agent does something expensive.

Whoever wins, the direction is settled by the traffic itself: bots passed humans in web requests in June 2026 (57.5% of HTML traffic, Cloudflare), and rules for the majority are being written now, in draft files and defaults. Sites that publish a policy early choose their terms. Sites that wait will have terms chosen for them by platform defaults.

1994the year robots.txt was written: before agents, before LLMs
57.5%bot share of HTML traffic in June 2026 (Cloudflare): the policy file's audience
v0.2the current draft of the rival agent.json standard
1file is all a policy takes: /.well-known/agent-policy.json

Why publish rules for agents instead of just blocking them?

Because blocking is a wall and a policy is a door with terms. Some agents are customers in waiting, some are search infrastructure you want, and only some are pure extraction. A published policy separates them in writing: it invites the welcome, conditions the tolerated and strengthens your position against the rest, in a form agents and courts can both read.

Check your agent policy: what can an agent learn?

Answer for your main site as a well-behaved agent would find it. Everything runs in your browser; nothing is sent anywhere.

Publish your agent policy: the working order

  1. Decide the three buckets: which agents may read, which may transact, which are refused, and the limits on each. This is a business decision, not a technical one.
  2. Write it in machine form: an agent-policy.json at /.well-known/ naming agents, actions, limits, terms and a contact. AgentGate's generator drafts it from your answers.
  3. Keep robots.txt aligned: the old file and the new file should never contradict. Agents read both; disputes will too.
  4. Link it where agents look: reference the policy from your llms.txt and your site footer, so discovery does not depend on the well-known path alone.
  5. Review on the traffic: watch which agents actually visit (your logs know) and revise the policy as the ecosystem moves. Standards are young; early publishers shape them.

AgentGate, by Ingenious AI, is the machine-readable agent-access policy standard: declare which AI agents may use your site, what they may do and on what terms, at /.well-known/agent-policy.json, the robots.txt of the agentic economy. See how AgentGate works.

FROM INGENIOUS AI
AgentGate lets you publish your rules once, in the file agents are learning to read first.
All 21 platforms: DisclosureGuard  ·  ModelMinder  ·  TermsWatch  ·  Vibe-Code Audit  ·  Rogue-Agent Defense  ·  BreachHunter  ·  ChatBreaker  ·  Agent Flight Recorder  ·  Agent Auth Gateway  ·  DomainGuard  ·  ContentCraft AI  ·  EvalScope  ·  BoundaryVault  ·  Agentic Receipts  ·  Agent-Ready Store Audit  ·  AgentBlock  ·  HomeShield MCP  ·  AgentGate  ·  ADMT  ·  DueLadder  ·  WireWatch
One suite, one deal: the Ingenious AI Security Suite. All 21 platforms are live now:
Chat with us on WhatsApp →

Quiz: how well do you know the policy race?

1. What year was robots.txt written?

2. What can robots.txt not express?

3. What is the rival draft standard called?

4. Where does an agent policy file live?

5. Why publish a policy instead of only blocking?

Frequently asked questions

What is an agent-access policy?

An agent-access policy is a machine-readable declaration of how AI agents may use your site: which agents are welcome, what actions they may take, what limits and terms apply, and who to contact. AgentGate publishes it as agent-policy.json in the /.well-known/ directory, where agents and tools are learning to look for it.

What is AgentGate?

AgentGate is Ingenious AI's agent-access policy standard and tooling. It generates and hosts the policy file at /.well-known/agent-policy.json, declaring named agents, permitted actions, limits and terms. Where robots.txt gave crawlers one bit of information, refuse or allow, AgentGate gives agents the actual rules of engagement for your site.

How is this different from robots.txt?

robots.txt, written in 1994, answers one question for crawlers: may you fetch this path. An agent policy answers the agent-era questions: which agents, doing what, how much, under what terms, and who to contact. The files complement each other; a serious site keeps both consistent, because agents and disputes read both.

What about the agent.json standard?

agent.json is a rival proposal from the Agent Web Protocol project, currently at draft v0.2, aiming at the same /.well-known/ slot. Standards competition is normal at this stage. AgentGate's bet is policy depth: named agents, actions, limits and terms. Publishing a policy now matters more than which draft ultimately wins, because terms you publish are yours under any standard.

Does a policy file stop bad agents?

No, and it does not claim to. A policy governs honest agents, documents your terms for disputes, and separates the welcome from the refused in writing. Stopping agents that ignore policy is enforcement, a different layer (see AgentBlock). Declaration plus enforcement is the complete posture; either alone is half.

Sources: Agent Web Protocol, agent.json draft v0.2 · Cloudflare Radar automated-traffic milestone (June 3, 2026) · Ahrefs keyword data (ai agents, 46,000/mo US) · Ingenious AI unicorn audit of AgentGate (October 23, 2026).

Related checks