AI Search Shows 3X More 404s Than Google

AI-powered search tools are directing users to far more broken links compared to Google. A recent study reviewing 16 million URLs found that ChatGPT is particularly prone to generating non-existent links.

The findings support Google’s earlier warnings that fabricated, or “hallucinated”, links could become a widespread problem across AI-driven platforms.

According to research by Ahrefs, AI assistants are almost three times more likely than Google Search to lead users to 404 error pages.

This report comes just six months after Google’s John Mueller highlighted concerns about the growing risks of AI link hallucinations.

ChatGPT Leads In URL Hallucination Rates

Among all AI assistants tested, ChatGPT was found to generate the highest number of fake URLs. According to the research, around 1% of links users clicked through ended up on 404 error pages, compared with just 0.15% on Google.

The issue appears even larger when considering every URL mentioned by ChatGPT, not only those clicked. In this case, 2.38% of its links led to broken pages. By contrast, only 0.84% of Google’s top results pointed to errors.

Other AI tools showed varied performance. Claude ranked second with 0.58% broken links from clicks, followed by Copilot at 0.34%, Perplexity at 0.31%, and Gemini at 0.21%. Mistral performed best with only 0.12% broken links, though it was also the least effective in driving traffic to websites.

Why Does This Happen?

The study identified two key reasons why AI systems often generate false links.

The first is that some web pages once existed but have since been removed or relocated. If an AI tool relies on outdated data instead of checking the web in real time, it can end up pointing users towards pages that no longer exist.

The second issue is that AI occasionally fabricates links altogether. These links may look plausible, following common URL patterns, but in reality, the pages have never existed.

Ryan Law from Ahrefs highlighted this with examples from their own website. AI assistants had generated links such as “/blog/internal-links/” and “/blog/newsletter/”, which appear credible but are in fact non-existent pages.

Limited Impact on Overall Traffic

At first glance, the issue might appear serious, but for most websites the impact is minimal. Currently, AI assistants account for just 0.25% of overall website traffic, whereas Google is responsible for a far larger share at 39.35%.

This suggests that fabricated URLs affect only a very small slice of an already limited traffic source. However, the challenge could become more pronounced as reliance on AI tools for research and information gathering continues to grow.

The research also revealed another concern: around 74% of newly published web pages now contain AI-generated content. If such content includes non-existent links, there is a risk that web crawlers will index them, which could spread the problem more widely across the internet.

Mueller’s Prediction Proves Accurate

The results echo predictions made by Google’s John Mueller back in March. At the time, he anticipated a “modest rise in clicks on hallucinated links” over the following 6 to 12 months.

Mueller advised website owners to put their energy into creating stronger 404 error pages rather than trying to capture occasional, unintended traffic.

His recommendation to gather evidence before making any major adjustments now seems well-founded, especially given that Ahrefs’ study shows the actual traffic impact to be quite limited.

Mueller also suggested that the issue may naturally decline as AI systems become more sophisticated in handling URLs. Whether this proves true remains to be seen.

Looking Forward

At present, the best approach for most websites is to concentrate on two key actions. Firstly, ensure that 404 error pages are useful and provide guidance to visitors who end up on broken links. Secondly, put redirects in place only for fake URLs that actually attract a noticeable amount of traffic.

By doing this, site owners can manage the issue effectively without making unnecessary changes, since it remains a relatively minor concern for the majority of websites.