Bot protection that doesn't break your SEO
We show you which bots hit your site, stop the bad ones at the edge and check that Googlebot, Bingbot and ad crawlers still get through. Start with a free report for any site behind Cloudflare.
Get a free bot report How it works
Verified search and ad crawlers are never blocked. We check it after every change.
- 66.249.66.1GoogleGooglebot/2.1Allowed, verified
- 198.51.100.xHome broadbandChrome, WindowsHuman
- 167.99.x.xDigitalOceanGooglebot/2.1Blocked, fake
- 157.55.39.10Microsoftbingbot/2.0Allowed, verified
- 203.0.113.xProxy networkChrome, LinuxChallenged
- 192.0.2.xMobile carrierSafari, iPhoneHuman
What is bot protection for a website?
Bot protection separates automated traffic from people and decides what each bot may do. Search engines and ad crawlers should get in. Scrapers, fake crawlers and request floods should not.
Doing that well needs data. Switching on a blanket "block bots" setting often stops Googlebot too, and nobody notices until pages drop out of search.
What bots do to small sites
Server load
One hosting network sending requests all night can slow a store down for real customers, even behind a CDN.
Junk in analytics
Direct sessions from data centers inflate traffic and wreck conversion rates. See bot traffic in Google Analytics 4.
Blocked Googlebot
An anti-bot switch or a country block that catches Google's IPs stops crawling. Read what Bot Fight Mode blocks.
Scrapers and AI crawlers
Price scrapers and AI training bots copy your content. Decide which to allow with our AI crawler guide.
How it works
- Free reportYou create a read-only token for one site. We break down every request Cloudflare saw.
- RulesWe write firewall rules that skip verified crawlers first, then stop the bad traffic.
- CheckA few days later we confirm that search and ad crawlers still get through and the load is gone.
The token steps and what we can and cannot see are on How it works.
Free tools
- Bot traffic checkFree report on bots, fake crawlers and blocked search engines.
- Verify GooglebotPaste an IP and see if it is a real Google crawler.
- Googlebot IP rangesOfficial lists for Google, Bing and OpenAI, ready to download.
- Bot user agentsExact user agents of search, ad, AI and SEO crawlers.
Services
- Protection setup$100One-off: traffic analysis, rules, and a check that search crawlers are allowed.
- Monitoringfrom $30/moDaily checks, alerts on new bot waves or blocked crawlers, rule updates.
Prices by traffic volume are on the pricing page.
What we found on a real store
An online store on Cloudflare's Free plan saw its server stall several times a week. The report showed:
- Over 50%of all requests in one week came from a single hosting network using a Linux Chrome user agent.
- Googlebot blockedby the anti-bot settings switched on to fight it, and a separate AI-bot rule blocked Bingbot.
- About 70%of GA4 sessions were bots, so the conversion rate looked several times lower than it was.
After a rule set that skips verified crawlers and blocks the network at the edge, the bot traffic stopped reaching the server and Googlebot was crawling normally again.
FAQ
Is it safe to give you a Cloudflare token?
The token is read-only, limited to one site, can be restricted to our IP address and expires on its own. It cannot change settings or open your pages' content, orders or customer accounts. Firewall events do include visitor IP addresses, which we use only for the report.
Will you block Google by mistake?
Every rule set starts with a skip for verified search and ad crawlers, and we check after each change that Googlebot, Bingbot and AdsBot get normal responses.
My site is not on Cloudflare. Can you help?
Yes, with your server's access logs. For blocking we usually suggest moving the site behind Cloudflare: the free plan is enough to start.
How much does it cost?
The report is free. A one-off setup is $100, monitoring starts at $30 a month and depends on your traffic volume. See pricing for details.