AI Agents Pay to Browse

Google is testing a new way of paying publishers when their content helps power AI answers. But the system does not show publishers exactly how their monthly payment is calculated.

A different approach is now being tested on an independent website. Instead of waiting for a platform to decide what content is worth, the site sets its own price.

The idea is simple. An AI agent requests a page and receives a price. If it wants the content, it pays before accessing it.

Google’s AI Contribution Pilot

Google’s AI contribution pilot gives selected publishers an earnings figure through Google Search Console.

Publishers are paid when Google determines that their content made a significant contribution to an AI-generated answer. This can happen through AI Overviews, AI Mode or the Gemini app.

However, publishers do not receive a detailed calculation showing how Google arrived at the payment.

The system also distinguishes between content used to create an answer and content that is simply linked afterwards. A page can qualify when it helps generate the response. Being listed as a source after the answer has already been created does not qualify.

The pilot is still limited. Google has reportedly approached dozens of publishers, with smaller publishers showing more interest than some larger organisations.

A Different Model for AI Crawlers

An independent test takes the opposite approach.

The website charges AI agents one cent per page. The agent has to pay before it can access the protected content.

Several test payments have already been completed. Each transaction was settled on a blockchain and has a public transaction record.

However, there is an important limitation. The payments have so far been made by the site’s own agents. They are not payments from search crawlers.

The transactions also use testnet USDC. This means the tokens have no real monetary value.

The experiment is therefore about testing the payment process rather than generating income.

The basic system works like this:

An agent requests a page.
The website returns a payment requirement.
The agent checks the price against its spending limits.
It makes the payment.
The website releases the content.

This type of system is becoming more relevant as companies develop tools that allow AI agents to make payments.

What Cloudflare Is Building

Cloudflare is also working on infrastructure designed for AI agents and online payments.

During its Agents Week in August, the company announced its Wallets product and opened reservations for wallet handles.

The handle itself is free to reserve. However, reserved handles are not currently usable as normal wallets.

Cloudflare’s system separates wallets for people from wallets used by their AI agents.

Account Wallets are controlled by people. They can be funded and given spending limits.

Virtual Wallets are designed for agents. They allow agents to spend within the limits set by the person controlling the account.

Cloudflare is also developing a Monetization Gateway. This is intended to let websites charge AI agents for accessing online resources.

The product has been announced but remains limited to users who can access it through the relevant waitlist.

The independent test therefore uses another method to demonstrate the same general concept.

Why Should Websites Charge AI Agents?

The traditional relationship between websites and search engines was fairly straightforward.

Search engines crawled websites and displayed their pages in search results. Users then clicked through to the websites.

That traffic had value for publishers.

AI search changes the process.

An AI system can visit a website, extract useful information and use it to answer someone’s question. The user may never visit the original site.

This creates a different relationship between publishers and AI companies.

Some smaller websites have responded by blocking AI crawlers. This prevents their content from being accessed, but it can also limit their visibility in AI systems.

Large publishers have taken another route. Some have negotiated licensing agreements with AI companies.

Google’s contribution pilot offers a third model. Instead of negotiating individual deals, Google decides when content has contributed to an AI answer and pays the publisher accordingly.

There is also growing interest in pay-per-use models.

Cloudflare has said it is moving towards systems where websites can receive payment based on actual use rather than simply counting crawls.

That distinction matters.

A crawler downloading a page does not necessarily mean the content was useful. But proving that an AI system actually used the information is difficult for the website owner.

A publisher therefore has to trust the platform to report how the content was used and what it was worth.

A direct payment model works differently. The website can set the price and confirm whether the payment was made.

How the x402 System Works

The experiment uses the x402 protocol.

HTTP has had a 402 Payment Required status code since 1997. It was originally reserved for future use.

x402 gives that status code a practical role in online payments.

When an agent requests a protected page, the server can return a 402 response. The response includes information about the required payment.

The agent then authorises the payment and sends the request again with the payment information.

A payment facilitator verifies the transaction and settles it on the blockchain.

Once the payment is confirmed, the website provides the requested content.

The process also creates a settlement record that can be checked later.

The test site currently charges $0.01 per page using testnet USDC on Base Sepolia.

USDC uses six decimal places. Therefore, the payment amount of 10000 represents 0.01 USDC.

The agent does not need to create an account with the website. It simply needs a compatible wallet and the required funds.

This is different from relying on robots.txt.

A website’s robots.txt file asks crawlers to follow certain rules. A payment requirement can actually prevent access until the transaction has been completed.

Building the Test Website

The experiment is running on a Cloudflare Worker.

The site uses Hono and x402 middleware. The implementation is relatively small and has been running since early August.

There are three main areas.

The landing page explains the experiment and remains free.

The research page is protected and costs one cent per crawl.

A separate earnings page records the payments. It shows information such as timestamps, payer addresses and blockchain transaction hashes.

A normal request to the paid page receives a 402 Payment Required response.

Human visitors see a different interface. The page explains that the content has been priced for AI agents and provides a preview option.

The preview allows people to read the article without making a payment.

There is also another version of the system that can charge only requests coming from recognised AI crawler user agents.

The test included services such as GPTBot, ClaudeBot and PerplexityBot.

However, user-agent strings can be copied or faked. They therefore cannot prove who is actually making a request.

Cryptographic verification provides a stronger way of confirming crawler identity.

Setting Spending Limits

The payment system also needs controls on the buyer side.

An AI agent should not be allowed to spend unlimited amounts simply because a website requests payment.

The test agent therefore has two spending limits.

The maximum price for one request is $0.05.

It also has a daily allowance of $0.25.

The agent checks the requested price before authorising a payment.

If the price is above the permitted limit, the transaction is rejected.

This is an important part of the experiment.

Giving an AI agent access to a wallet is not the same as giving it unlimited spending power.

The human still needs to control how much the agent can spend.

The spending checks happen before the payment is signed. If the request exceeds the limits, the agent stops without creating a payment signature.

That is broadly similar to the approach Cloudflare is taking with its virtual wallets.

Claude Can Pay Automatically

The most interesting part of the experiment is the connection with Claude Code.

Claude Code supports hooks that can run around tool calls.

A hook can detect when Claude attempts to access a page and receives a 402 response.

The hook then checks the payment request.

If the amount falls within the agent’s budget, the hook makes the payment and retries the request.

The content is then returned to Claude along with the payment receipt.

Claude can continue working without the user manually approving every small payment.

The system still has a limit.

If the requested price is higher than the spending cap, the payment is rejected. Claude can then tell the user that the page requires payment.

This creates a useful balance between automation and control.

The agent can purchase information when it needs it. But it cannot increase its own spending limit.

Should Websites Start Charging AI Agents?

For most websites, there is probably no reason to start charging AI crawlers immediately.

The technology is developing quickly, but the major crawlers used by website owners do not all support this type of payment system yet.

AWS has made agent payments available through its AgentCore platform. Cloudflare is also developing tools that support agent payments.

That does not mean that GPTBot, ClaudeBot, PerplexityBot or Googlebot will automatically pay when they encounter a paid page.

If a website requires payment and the crawler cannot make that payment, the content simply remains inaccessible.

That could affect how often the site appears in AI-generated answers.

There is also a need for caution when looking at transaction numbers in the wider x402 market.

Large numbers of blockchain transactions do not necessarily mean that AI agents are actively buying content.

Automated scripts, testing systems and transactions between accounts controlled by the same organisation can all create blockchain activity.

Transaction data alone cannot always show whether an autonomous AI agent was responsible.

The test website provides a good example of why this matters. The payments are genuine blockchain settlements, but the person running the experiment controls both sides.

What Could Websites Charge For?

The bigger question may not be whether every website should charge AI agents.

Instead, website owners could consider which resources have enough value to justify a payment.

General information may still be better left freely accessible. Being cited by an AI system can help people discover a website.

Original research could be different.

Datasets, specialist reports and useful online tools may provide something an AI agent needs to complete a specific task.

Charging for that individual request could make more sense than placing the entire website behind a subscription.

It also gives the publisher control over the value of the resource.

Google’s AI contribution pilot takes a platform-led approach. Google determines when content has made a significant contribution and then calculates a payment.

A direct payment model moves that decision closer to the website owner.

The publisher sets the price.

The agent decides whether the resource is worth buying.

The transaction provides a record of what was paid.

The Future of AI Payments

The infrastructure for AI payments is moving forward.

Cloudflare is developing its Monetization Gateway. Other companies are also working on systems that allow agents to make payments during tasks.

The x402 Foundation is another sign that interest in this area is growing. The organisation launched under the Linux Foundation with members including Visa, Mastercard, Google, AWS, Stripe and Cloudflare.

However, widespread adoption will depend on AI agents and websites supporting the same payment standards.

For now, the technology remains experimental for many publishers.

The test shows that an AI agent can request a page, receive a price, check its budget, make a payment and access the content automatically.

The next step is moving from testnet transactions to real money.

Once agents begin paying real USDC for useful information, the model will face a much more important test.

The bigger question will then be whether AI agents are willing to pay for individual pieces of information and whether website owners can turn that demand into a sustainable source of revenue.

For publishers, it may be worth thinking about that question now, before agent payments become another standard option in website dashboards.