What the server log analyzer does
Upload your nginx or Apache access log and this tool shows, in seconds, how much of your traffic comes from AI crawlers versus search bots and real humans. It classifies each request by user-agent, GPTBot, ClaudeBot, PerplexityBot, CCBot, Bytespider, Googlebot and more, gives you an AI-traffic share, lists your most-crawled pages, and flags crawl-budget waste. Everything runs in your browser, so your log (which can contain IPs and URLs) is never uploaded.
Why analyse your logs
Your access log is the ground truth of who actually visits your site, no sampling, no JavaScript tracking gaps. As AI crawlers grow into a real share of web traffic, logs are the only reliable way to see them. And logs reveal problems analytics can't: crawlers wasting their budget on the wrong URLs.
Crawl-budget waste (the ?_rsc= trap)
Modern frameworks generate prefetch URLs, for example Next.js React Server Component requests ending in ?_rsc=. If crawlers spend their limited budget hammering those instead of your real pages, large parts of your site can go uncrawled and unindexed. This exact trap has silently derailed big migrations. The analyzer counts how many such requests are in your log and which paths they hit, so you can block *?_rsc=* in robots.txt if needed.
How to use it
- Download your access log from your server (nginx:
/var/log/nginx/access.log; Apache:/var/log/apache2/access.log). - Upload it here. It's parsed instantly on your device.
- Read the report: AI-traffic share, per-bot counts, top pages, and any crawl-budget waste.
Good to know
- Supports the standard Common and Combined Log Formats; unrecognised lines are skipped.
- Nothing is uploaded, logs stay private on your machine.
- Pair it with the AI Crawler Access Checker to control which of these bots you allow.