AI Training Opt-Out Kit

Free AI training opt-out kit. Get ready-to-paste robots.txt rules, noai/noimageai meta tags and server headers to block AI training crawlers, plus a directory of per-company opt-out links.

Use these to tell AI companies not to use your website's content for training. They're requests that well-behaved crawlers honour, not a hard block. Add all three layers for the best coverage.

1. robots.txt (add to the top of your robots.txt)

Blocks the known AI training and dataset crawlers while leaving normal search engines free to index you.

# Block AI training & dataset crawlers (keep normal search crawlers allowed)
User-agent: GPTBot
Disallow: /
User-agent: OAI-SearchBot
Disallow: /
User-agent: ChatGPT-User
Disallow: /
User-agent: Google-Extended
Disallow: /
User-agent: Applebot-Extended
Disallow: /
User-agent: CCBot
Disallow: /
User-agent: ClaudeBot
Disallow: /
User-agent: anthropic-ai
Disallow: /
User-agent: Claude-Web
Disallow: /
User-agent: Bytespider
Disallow: /
User-agent: meta-externalagent
Disallow: /
User-agent: FacebookBot
Disallow: /
User-agent: Amazonbot
Disallow: /
User-agent: Diffbot
Disallow: /
User-agent: Omgilibot
Disallow: /
User-agent: Timpibot
Disallow: /
User-agent: cohere-ai
Disallow: /
User-agent: PerplexityBot
Disallow: /
User-agent: Applebot
Disallow: /
User-agent: ImagesiftBot
Disallow: /
User-agent: Scrapy
Disallow: /

2. HTML meta tags (add inside <head>)

Signals 'no AI' and 'no image AI' training at the page level, honoured by a growing number of crawlers.

<meta name="robots" content="noai, noimageai">
<meta name="googlebot" content="noml">

3. Server header — nginx

Applies the noai signal site-wide via HTTP headers (nginx server block).

add_header X-Robots-Tag "noai, noimageai" always;

3. Server header — Apache (.htaccess)

Header set X-Robots-Tag "noai, noimageai"

Per-company opt-out directory

Company	Covers	How to opt out
OpenAI (ChatGPT)	Web crawling for training & ChatGPT browsing	Block GPTBot, OAI-SearchBot and ChatGPT-User in robots.txt	Official ↗
Google (Gemini / Vertex)	AI-model training (separate from Search)	Block the Google-Extended token in robots.txt	Official ↗
Anthropic (Claude)	Web crawling for training	Block ClaudeBot and anthropic-ai in robots.txt	Official ↗
Common Crawl	Open dataset used to train many models	Block CCBot in robots.txt	Official ↗
Apple	Apple Intelligence training	Block Applebot-Extended in robots.txt	Official ↗
Meta (Llama)	Web crawling for AI training	Block meta-externalagent & FacebookBot in robots.txt	Official ↗
Creators (art, images)	Opt out of many training datasets at once	Register your work with Spawning's Have I Been Trained	Official ↗

The robots.txt above already blocks these crawlers. Want to confirm it's working? Use the AI Crawler Access Checker.

What the AI training opt-out kit does

This kit gives you everything needed to tell AI companies not to use your website for training: ready-to-paste robots.txt rules that block the known AI training crawlers, noai / noimageai meta tags, server headers, and a directory of per-company opt-out links. It's the mirror image of our AI Crawler Access Checker, here you're closing the door rather than opening it.

How the opt-out works

AI crawlers are supposed to check your robots.txt and honour page-level signals before taking your content. This kit produces those signals in three layers, robots.txt, HTML meta tags and HTTP headers, so the widest range of crawlers respects your choice. They're requests, not a hard wall: well-behaved crawlers from the major AI companies comply, but not every bad actor will.

The three layers

robots.txt — blocks GPTBot, Google-Extended, CCBot, ClaudeBot, Bytespider, Applebot-Extended, meta-externalagent and more, while leaving normal search crawlers free.
Meta tags — noai and noimageai signal 'no training' at the page level.
Server header — an X-Robots-Tag applies the signal site-wide.

Will it affect my Google ranking?

No. The robots.txt block here targets AI training crawlers, not search crawlers, so Googlebot and other search engines keep indexing you normally. Your search visibility is unaffected.

For artists and photographers

Add the meta tags and robots.txt above, then register your work with Spawning's Have I Been Trained (linked in the directory) to propagate opt-outs across many image-training datasets at once.

Good to know

Everything is generated in your browser, copy the parts you need.
After adding the robots.txt, confirm it with the AI Crawler Access Checker.
Opt-out standards are evolving, revisit periodically as new crawlers and tokens appear.

Good to know

AI Training Opt-Out Kit, frequently asked questions

How do I stop AI companies from training on my website?

Add three layers of signals: a robots.txt block for the known AI training crawlers (GPTBot, CCBot, Google-Extended, ClaudeBot, Bytespider and more), noai/noimageai meta tags in your pages, and an X-Robots-Tag server header. This kit generates all three ready to paste, plus a directory of per-company opt-out links.

Does blocking AI crawlers hurt my search ranking?

No, if you use the training-crawler block provided here. It targets AI training and dataset crawlers (like GPTBot, Google-Extended and CCBot) while leaving normal search crawlers such as Googlebot free to index you, so your search visibility is unaffected.

Do these opt-outs actually work?

robots.txt and the noai signals are honoured by well-behaved crawlers from the major AI companies, but they are requests, not a technical wall. Some bad actors ignore them. Combining robots.txt, meta tags and server headers gives you the strongest, most widely-respected opt-out available today.

What's the difference between this and blocking search?

AI training crawlers and search crawlers are different bots. This kit blocks the training ones only. If you also want to stop appearing in AI answers, block the AI search bots too, use the AI Crawler Access Checker to see the full picture.

I'm an artist, how do I opt out of image datasets?

Beyond the robots.txt and noimageai tags here, register your work with Spawning's 'Have I Been Trained', which propagates opt-outs across many training datasets. The directory in this kit links to it and other official opt-out pages.

Is it free?

Yes, free with no signup. Everything is generated in your browser, copy what you need.

Keep going

AI Training Opt-Out Kit

1. robots.txt (add to the top of your robots.txt)

2. HTML meta tags (add inside <head>)

3. Server header — nginx

3. Server header — Apache (.htaccess)

Per-company opt-out directory

What the AI training opt-out kit does

How the opt-out works

The three layers

Will it affect my Google ranking?

For artists and photographers

Good to know

AI Training Opt-Out Kit, frequently asked questions

Related tools