Question

Does Cloudflare Bot Fight Mode block AI crawlers?

It can. If your bot security layer issues challenges, crawler requests may fail even when robots.txt is permissive.

How to verify quickly

1. Check Cloudflare Security Events for service field and action around crawler requests.
2. Look for repeated 403 or challenge outcomes on /robots.txt, sitemap, and core pages.
3. Cross-check origin logs to confirm requests reached the server.

Practical fix path

If policy needs AI-search visibility, tune edge security settings first, then validate crawler access again with logs.

Use the 403 troubleshooting flow