Cloudflare AI Blocking Hits Googlebot

A report from an SEO community suggests that Cloudflare’s AI bot blocking settings may be preventing legitimate search crawlers, including Googlebot and Bingbot, from accessing websites.

The issue appears to involve Cloudflare’s tools designed to block AI training crawlers. Some users have reported that enabling these settings may also block search engine bots, potentially affecting website indexing.

User Reports Googlebot Being Blocked

An SEO professional raised the concern on Reddit after noticing that activating Cloudflare’s AI Training blocking feature resulted in Googlebot and Bingbot receiving 403 Forbidden responses when attempting to access the website’s sitemap.

The user explained that the sitemap became accessible again immediately after disabling the AI Training block, creating uncertainty over whether the feature was incorrectly blocking legitimate search crawlers.

They questioned whether website owners would need to choose between allowing AI training access or risking search engines being unable to crawl and index their content.

Google’s John Mueller responded to the discussion, asking the user to send further details privately so the issue could be investigated.

Another Reddit user suggested that Cloudflare may have been blocking fake Googlebot traffic, where malicious bots imitate Google’s crawler user agent. However, the original poster stated that the issue appeared to involve Cloudflare’s own dashboard classifications rather than spoofed bots.

Cloudflare’s Upcoming AI Bot Changes

The concerns come as Cloudflare prepares changes to how it handles AI-related crawlers.

According to Cloudflare’s documentation, from 15 September 2026, new default settings will block certain bots classified as AI training or AI agents on websites displaying advertisements.

However, the company has also highlighted that some crawlers have mixed purposes, meaning they can both collect information for AI training and perform search indexing. These mixed-use crawlers may also be affected when AI training blocks are enabled.

Cloudflare stated that customers would have the option to adjust these settings before the new defaults take effect.

Potential Impact on Website Owners

It remains unclear whether the reported Googlebot blocking is caused by a temporary issue, incorrect configuration or an unintended interaction between Cloudflare’s security features.

Website owners using Cloudflare’s AI crawler controls may want to review their settings carefully to ensure legitimate search engine bots are still able to access important pages, including XML sitemaps.

As AI-related web controls become more common, balancing content protection with maintaining search visibility is likely to become an increasingly important consideration for website owners and SEO professionals.