Getting Products Into ChatGPT Isn’t Enough

AI-powered shopping is growing quickly, with Shopify reporting a 15-fold increase in orders coming through AI-powered search since January 2025.

That growth is encouraging for retailers looking to make their products visible through platforms such as ChatGPT. But getting a product discovered is only the first step.

The bigger question is what happens afterwards.

When an AI agent is responsible for completing a purchase rather than a human shopper, can the retailer’s website, product data, checkout system and refund process cope with the way machines operate?

Research and testing from software quality assurance company QAwerk suggests that many ecommerce systems may not yet be prepared for that shift.

AI Shopping Works Differently From Human Shopping

A traditional online shopper moves through a website at a relatively unpredictable pace. They might browse several products, leave the site, return later, change their mind or abandon their basket altogether.

An AI agent behaves very differently.

It can send structured requests to APIs, compare products against specific requirements and make purchasing decisions within seconds. It can also send multiple requests in rapid succession.

That creates a problem for systems designed primarily around human behaviour.

Rate limits and bot protection, for example, are often designed to identify automated activity. But an AI shopping agent is legitimate automation, meaning the very systems intended to protect an ecommerce site could potentially interfere with a genuine purchase.

Session management can create similar problems. A website may expect a shopper to maintain one continuous session, while an agent could retrieve product information, end the session and return later to complete the transaction.

According to QAwerk founder Konstantin Klyagin, this type of issue is often overlooked because conventional quality assurance tends to focus on whether a system produces the correct result. Less attention is given to whether the infrastructure can support a non-human user operating at machine speed.

Product Visibility Doesn’t Guarantee a Sale

This distinction matters because much of the current focus around AI commerce is centred on getting products discovered.

Retailers are working on structured product information, catalogue feeds and emerging commerce protocols such as Google’s Universal Commerce Protocol (UCP) and OpenAI’s Agentic Commerce Protocol (ACP).

Shopify has already seen significant growth in AI-driven shopping activity, while major retailers and marketplaces have begun experimenting with selling through AI platforms.

But being included in an AI platform’s results does not automatically mean the transaction will succeed.

A product might be correctly identified, selected and added to a purchasing journey, only for the process to fail because the underlying systems cannot agree on price, stock levels or order status.

In other words, AI visibility and AI commerce are two different problems.

Conflicting Data Could Cause Problems

QAwerk’s testing has highlighted how inconsistencies between different parts of an ecommerce system can become particularly problematic when an AI agent is involved.

In one project involving a platform called Pridefit, the team discovered that two parts of the purchasing funnel were maintaining separate versions of the same plan information.

The differences were relatively small, including variations in pricing and product attributes. A human customer might not even notice the discrepancy, or could simply refresh the page and try again.

An AI agent has much less room to work around such inconsistencies.

For example, an agent could retrieve a product from one data source showing that it is available at a particular price, while the checkout system checks another source and returns a different price, SKU or stock status.

The agent may then be unable to determine which version is correct.

QAwerk addressed the problem by removing the duplicated information and creating a central source that the different parts of the funnel could rely on.

This illustrates a broader issue that could become increasingly important as agentic commerce grows: all systems involved in a transaction need to agree on the same state.

Where Agentic Commerce Could Break Down

The problem is not necessarily an AI agent choosing the wrong product.

Instead, failures could occur because different systems interpret the transaction differently.

Potential examples include:

A product feed showing a variant as available while checkout reports it as out of stock.
A payment request timing out and being sent again to an API that cannot safely handle duplicate requests.
A refund being processed by the retailer without the updated status reaching the AI agent.
Different systems displaying inconsistent prices or product details.
Session requirements preventing an agent from returning to an existing transaction.

A human shopper can often recognise that something has gone wrong and take another route.

An AI agent needs the underlying systems to provide clear, predictable responses.

Three Tests Retailers Should Run

Before spending more time connecting products to another AI platform, retailers should make sure their existing infrastructure can cope with machine-led transactions.

1. Test Checkout Under Machine-Like Traffic

A checkout system should not only be tested with individual human-style sessions.

Retailers should also assess how it performs when multiple API requests arrive rapidly and concurrently.

A system may have successfully handled millions of normal customer visits but still struggle when an automated shopping agent starts making several requests in quick succession.

This is particularly important for retailers adopting UCP, ACP or other agentic commerce technologies.

2. Check Product Data at Source

Product information should be checked from the perspective of the systems consuming it, rather than simply looking at what appears on the webpage.

A product page can look completely consistent to a person while different systems underneath are using conflicting information.

Retailers should therefore check whether product feeds, APIs, catalogue data and checkout systems are all working from reliable and consistent information.

3. Test Returns and Refunds

The purchasing journey does not end when an order is placed.

Returns and refunds also need to work reliably when initiated by an automated system.

A human customer can contact support if a refund appears stuck or an order status is unclear. An AI agent may not have the same ability to interpret or resolve an inconsistent response.

Making sure these processes update correctly the first time will become increasingly important as more transactions involve software rather than people.

AI Commerce Needs More Than Another Protocol

The growth of UCP, ACP and other commerce standards is likely to make it easier for AI systems and retailers to communicate.

However, adopting another protocol will not solve problems that already exist within a retailer’s own infrastructure.

The same product information still needs to be accurate. APIs still need to respond reliably. Inventory systems still need to match checkout. Orders and refunds still need to maintain a consistent status.

These foundations matter regardless of which AI shopping platform ultimately becomes dominant.

SEO Gets Customers to the Door

There is an important lesson here for SEO and digital marketing teams.

Technical SEO and product optimisation can help make a product discoverable to AI systems. Structured data, accurate feeds and clear product information can all increase the chances of appearing in an AI-driven shopping journey.

But visibility is only one part of the process.

If an AI agent finds the product but cannot successfully complete the transaction, the visibility work has not delivered its full value.

Retailers therefore need to think beyond “Can AI find my product?” and start asking “Can AI successfully buy it?”

That requires SEO, ecommerce, development and QA teams to work together rather than treating AI visibility as a standalone marketing task.

Ecommerce Testing Is Likely to Change

As AI agents become more common, quality assurance may need to develop alongside them.

Traditional testing will still be necessary for the human customer experience. However, retailers may also need dedicated testing for AI agents that can interact with websites and APIs at a much higher speed.

That means checking whether an agent can:

Understand the available product information.
Retrieve consistent data.
Move through the purchasing process.
Handle API responses correctly.
Complete a transaction without unexpected errors.
Receive accurate order and refund updates.

This is similar to how ecommerce testing had to evolve as mobile shopping became mainstream. A new type of user requires systems to be tested in a new way.

What Retailers Should Focus On

The rush to appear in ChatGPT, Gemini and other AI platforms is understandable. AI-powered shopping is developing quickly, and retailers do not want to miss the opportunity.

But discovery should not come before infrastructure.

Getting a product into an AI platform’s results may be relatively straightforward. Ensuring that the product can move from discovery to purchase, payment, delivery and potentially a refund is considerably more complicated.

For retailers preparing for the next stage of AI commerce, the priority should therefore be broader than visibility.

Make the product discoverable, but make sure the systems behind it are ready to complete the sale.