AI is changing the way people discover, consume, and interact with websites. And, as usual, the shift is not only about technology. It is also about control.
Who can access your content?
What can they do with it?
Can they use it to train a model?
Can they summarize it somewhere else?
Can they act on behalf of a user?
And most importantly: what does your business get in return?
Long story short?
Website owners need more control over how AI bots access and use their content.
Cloudflare recently announced new AI traffic options that give website owners more granular control over different types of AI crawlers. Instead of treating all AI bots the same, Cloudflare now separates them into three main categories: Search, Agent, and Training.
And for eCommerce businesses, this is worth paying attention to.
AI traffic is not one single thing anymore
Until recently, the discussion around AI bots was mostly focused on one concern: model training.
A bot crawls your website, collects your content, and uses it to improve an AI model. Your product descriptions, buying guides, blog posts, FAQs, category copy, technical information, images, and other valuable content can become part of someone else’s system.
Without sending traffic back.
Without attribution.
Without compensation.
Without asking nicely.
But today, AI usage is broader than that.
Cloudflare’s new approach separates AI traffic into three practical use cases:
- Search: bots that index your content so people can find it later
- Agent: bots or AI agents that act on behalf of a user, often in real time
- Training: bots that collect content to train or fine-tune AI models
Of course this distinction matters. Because not every bot has the same value for your business.
Search can still help users discover your webshop.
An agent might help a customer compare products or complete a task.
Training, on the other hand, may use your content without bringing any immediate value back.
So the question becomes less “Should we block AI?”. The question should be:
Which AI use cases make sense for our business, and under which conditions?
Why this matters for eCommerce
For webshops, the copy is not just “content”. But a part of the buying experience.
Product descriptions help users choose the right item.
Category pages guide users through complex assortments.
Buying guides reduce doubt.
FAQs prevent support overload.
Delivery and return information builds trust.
Blog posts support SEO and long-term visibility.
All of this takes resources such as time, budget, research, UX thinking, SEO work, technical implementation. So when AI crawlers access this information, there should be a clear strategy behind it.
Because the impact can be real:
- Your content may be used in AI-generated answers without users visiting your website.
- Your organic traffic may decrease if users get answers directly from AI tools.
- Your product information may appear out of context.
- Your brand may lose control over how its expertise is presented.
- Your competitors may indirectly benefit from content you invested in.
This does not mean every AI bot is bad. It means access should be intentional.
Search visibility vs. content protection
This is where things become tricky.
Most website owners still want to be visible in search engines. For an eCommerce business, discoverability is crucial. If people cannot find your products, they cannot buy them.
But AI has blurred the line between classic search and AI-powered answers.
In the past, the deal was quite simple: search engines crawled your website and sent traffic back. Today, some AI-powered systems can use your content to answer the user directly, reducing the need to click through.
That creates a difficult balance.
Block too much, and you may hurt discoverability. Allow too much, and you may lose control over your content.
Cloudflare’s new controls are interesting because they give website owners a more nuanced setup. Search, Agent, and Training can be treated differently, instead of being bundled together under one generic “AI bots” label.
For eCommerce teams, this is exactly the kind of nuance that is needed.
What we would look at first
When working on eCommerce platforms, we usually start by understanding the business logic behind the technical decision.
Because bot control is not only a security setting. It affects SEO, marketing, analytics, content strategy, and sometimes even conversion.
So before deciding what to allow or block, we would look at questions like:
Which content is business-critical?
Not every page has the same value.
A homepage, product page, category page, blog post, help article, landing page, and checkout step all serve different purposes. Some pages are meant to attract traffic. Others are meant to convert. Others should probably not be accessed by bots at all.
How?
By mapping the website structure and identifying which pages should be open, restricted, protected, or monitored more closely.
Which bots bring value?
Some crawlers support search visibility, SEO tools, ad verification, monitoring, or useful integrations. Others may only extract value without giving anything back.
The goal is not to block everything blindly.
The goal is to understand what each type of bot is doing and whether that activity supports your business objectives.
How does this affect SEO?
Search traffic is still highly valuable for most webshops. Any change to crawler access should be handled carefully, especially when search-related bots are involved.
This means working together across development, SEO, content, and marketing.
A small technical change can have a large commercial impact if it affects indexation, rankings, or visibility.
What should AI agents be allowed to do?
AI agents introduce a new type of interaction.
They may browse a webshop on behalf of a user, compare products, check availability, or even complete actions in the future. For eCommerce, this could become a new buying channel.
But it also raises questions.
Should agents access product pages?
Should they access pricing?
Should they interact with cart or checkout flows?
Should they be treated like users, bots, or something in between?
These are not just technical questions. They are UX and business questions too.
A practical setup is better than a generic one
As with checkout, tracking, SEO, or performance, the best setup is rarely the default one.
A good configuration depends on the business model, the platform, the content strategy, and the traffic mix.
For some websites, allowing search and blocking training may be the right starting point.
For others, agent traffic may become useful if it supports product discovery.
For content-heavy brands, stronger protection may be needed.
For B2B platforms, the answer may depend on whether product data is public, gated, or connected to customer-specific pricing.
What matters most is that the decision is intentional.
Not accidental.
Not forgotten in a dashboard.
Not decided only after traffic drops.
Where development, marketing and tracking meet
This topic sits exactly at the intersection of multiple teams.
From a development point of view, bot rules need to be configured correctly and tested.
From an SEO point of view, visibility needs to be protected.
From a marketing point of view, content value and attribution matter.
From a tracking point of view, teams need to understand what traffic is human, automated, useful, or potentially harmful.
That is why AI bot management should not be treated as a one-time technical checkbox.
It should become part of the broader eCommerce maintenance and optimisation process.
How?
By:
- reviewing crawler activity
- checking Cloudflare or hosting-level bot settings
- validating robots.txt and content signals
- monitoring organic traffic changes
- checking server logs where needed
- aligning decisions with SEO and marketing goals
- protecting sensitive or high-value areas of the website
- documenting the chosen setup for future changes
Our take
AI will not stop crawling the web.
And eCommerce websites will not stop needing visibility.
So the real challenge is finding the right balance between being discoverable and being protected.
We believe website owners should have more control over how their content is accessed and used. Especially when that content is the result of real work: UX research, SEO strategy, copywriting, product data enrichment, technical implementation, and years of business knowledge.
For webshops, this is not only about blocking bots.
It is about protecting the value of the buying experience.
Because your website is not just a database for someone else’s AI model. It is your sales channel, your brand experience, your knowledge base, and often one of your most important business assets.
So yes, AI traffic deserves attention.
And the sooner eCommerce teams start treating it strategically, the better prepared they will be for what comes next.
Are you wondering how AI bots interact with your webshop? Let’s have a chat.
