Cloudflare is testing AI search signals, Pay Per Use models, and reporting tools designed to help website owners understand and potentially benefit when answer engines use their content without sending visitors back to the website. Businesses now have to answer a harder question: how does my business benefit when AI uses my knowledge before the customer ever visits my website?

For years, the online economy worked through a clear exchange. Businesses allowed search engines to crawl their websites, search sent visitors back, and those visits created value through sales, leads, subscriptions, ads, analytics, or trust. AI search is weakening that model by delivering answers before the click.

That shift affects publishers, ecommerce companies, consultants, software companies, healthcare providers, local businesses, and any organization whose website helps customers make decisions. Website traffic still matters, but it may no longer be enough to measure online discovery.

In short, AI search is separating discovery from the website visit. Cloudflare's announcement shows how companies are beginning to build the controls, reporting, and compensation models website owners may need as AI answers become a primary way people find information.

AI discovery is the way AI systems find, interpret, and use a business's online information to answer customer questions, shaping online discovery before a customer visits the website or even when no visit happens.

AI discovery is becoming business infrastructure for an Internet where AI answers can use company knowledge before a customer ever visits the website.

For decades, online discovery was built around a click. A customer question created website value through a search result and a visit.

Customer question
→ search result
→ website visit
→ business value

The visit carried the measurable business outcome. Publishers earned ad impressions or subscriptions. Ecommerce companies made sales. Consultants received inquiries. Local businesses earned enough trust for someone to call, book, or visit.

AI answers can break that sequence.

Customer question
→ AI answer
→ customer decision

The website may still provide the information behind the answer, but the customer may never reach the page. That missing visit is the economic problem. Without the visit, the business may lose the ad impression, subscription opportunity, lead form, sale, analytics signal, retargeting path, or direct customer relationship that once made discovery measurable.

Cloudflare is testing search signals that help AI systems find fresher content with less unnecessary crawling by AI bots, along with payment and reporting experiments that connect AI content use back to website owners. The old web economy depended on search sending visitors back to the website. AI answers can use the same information without the visit.

Cloudflare says more than 50% of online traffic is now non-human, and it argues that search is being rebuilt around AI answers. It also cites a 2025 Pew Research Center study finding that when Google shows an AI summary, users clicked traditional search results 8% of the time and clicked links inside the summary 1% of the time.

The click data shows why website traffic is becoming a less complete way to measure discovery. AI answers may use a company's information before the company ever sees the customer. Without traffic, attribution, or reporting, the business has fewer ways to know whether its website content is helping customers make decisions, supporting sales, or creating value the company can measure.

Cloudflare is trying to give website owners more than a yes-or-no choice about AI crawlers. Blocking AI crawlers can make a business harder for AI search systems to find. Allowing AI access can make a website's content more discoverable, but it can create traffic, and value problems for the site owner.

More than 20% of the web sits behind Cloudflare's network. That gives the company a view into which pages have changed, which pages are attracting human and bot traffic, and which pages answer engines may need to revisit. Cloudflare also says more than 50% of crawl traffic from good bots goes to re-fetching pages that have not changed.

Cloudflare's search research program is designed to test whether better signals can improve AI search while reducing repeated crawler traffic. The program would combine customer-shared signals, such as content freshness, with Cloudflare's view of traffic patterns. An answer engine could use those signals to find fresher content instead of repeatedly checking pages that have not changed. That saves compute for the answer engine and reduces the volume of requests the website has to serve.

For website owners, the same system could reduce unnecessary bot requests and provide more visibility into AI search activity. Cloudflare says customers could see what users are asking and how their content appears in AI results. That reporting gives businesses another way to evaluate content that may influence customers without producing a website visit.

Cloudflare says it is not sharing customer content and is not using the program to train foundation models. The company plans to publish what it learns, including whether the signals improve content discovery and reduce unnecessary crawling, and it plans to make the capability more widely available later this year.

The key point: Cloudflare is treating AI discovery as a control and measurement problem, not only a crawling problem. Its proposed model connects crawler permissions, freshness signals, AI search visibility, and usage data so website owners can make more specific decisions than simply allowing or blocking AI systems.

Cloudflare is also trying to move beyond Pay Per Crawl, the earlier model that allowed publishers to charge AI companies for crawling their content. Pay Per Crawl gave website owners a way to charge for crawler access. With Pay Per Use, compensation would be tied to content appearing in or supporting an AI answer.

Crawling alone does not show how much value a page creates. A page could be crawled once and then help answer thousands of user questions, while another page could be crawled repeatedly and never appear in an answer. That is the gap Cloudflare is trying to address. Payment is designed to follow the value the work delivers rather than the number of times a crawler happens to fetch it.

Ceramic.ai shows one version of that model on Cloudflare's network. Ceramic has built a pay-per-query system in which participating publishers can be paid when their content appears in Ceramic's AI search results. In that model, compensation is connected to visible use in an AI search result.

The Cloudflare and Ceramic program also includes reporting for answer engine optimization. Participating customers would be able to see the top queries that lead to their content appearing in search results, the specific webpage and snippet, average search result ranking position, and related information. That gives website owners a way to see which AI search questions surface their expertise instead of measuring only website visits.

You.com is testing a different approach. Instead of paying when content appears in a search result, an AI agent can pay on demand for a specific piece of premium content it needs for an answer, without an upfront commitment.

For businesses, these experiments begin to turn answer engine optimization into a measurement system. Instead of asking only how many people visited a page, a business can begin asking which AI queries surfaced its expertise, which pages were used, and whether that exposure created any return.

Pay Per Use remains experimental. Cloudflare says there is still a lot to learn, including how the model holds up at Internet scale. Cloudflare's approach is one of several emerging attempts to address this challenge, and the long-term business models for AI discovery are still evolving.

Businesses have long allowed search crawlers because search visibility brought visitors back to their websites. AI creates a different problem because the same website content may now support search results, AI-generated answers, AI products, agents, or training systems.

Cloudflare's crawler-policy change is meant to separate traditional search crawling from other AI uses. Starting September 15, 2026, Cloudflare's default settings will block mixed-use crawlers from pages that host ads unless the site owner changes the setting. Mixed-use crawlers are crawlers that combine traditional search crawling with crawling for AI agents, training, or other AI services.

The policy reflects this tension for website owners. Businesses still want to be visible in search, but they may not want the same crawler access to cover every AI use of their content. A page that helps a search engine index a website is not the same as a page used to train an AI model, support an AI agent, or generate an AI answer that may not send the user back.

Google Extended gives website owners one layer of separation. They can keep their pages eligible for Google Search while opting out of having their content used for some AI training and AI products, including Gemini Apps and Vertex API. Googlebot is different because it crawls for Search, including Search features such as AI Overviews and AI Mode.

That distinction matters because AI features are becoming part of the search experience itself. Cloudflare is responding from a different angle by pushing crawler operators to separate search, agent, and training purposes more clearly, while also testing reporting and payment models for AI content use.

For businesses, the result is a more complicated discovery decision. Search crawling, AI answers, AI agents, and AI training can no longer be treated as one bundled choice. Neither approach fully answers the business question yet: how does my business benefit when AI uses my knowledge?

Publishers make the AI search value problem easy to see. Their business model is often tied directly to traffic, subscriptions, and advertising, so fewer website visits have an obvious financial effect. The same problem can reach businesses whose websites do not look like media products.

Before customers buy, book an appointment, compare products, contact a business, install software, or trust a company, they ask questions. Business websites already contain many of the answers to those questions.

That makes AI discovery relevant to almost any business that publishes useful information online. A manufacturer may publish specifications, manuals, product pages, and distributor information. An ecommerce company may publish product descriptions, comparisons, policies, and support content. A software company may publish documentation, integrations, pricing pages, and security details. A healthcare provider may publish service information, patient education, and appointment guidance. A consultant or local business may publish expertise that helps a customer decide whether to make contact.

In each case, the website is doing more than attracting visits. It is holding the knowledge customers use to make decisions.

If AI answers rely on that knowledge, businesses need to know how their information is being found, interpreted, and used. Website visits have traditionally mattered because they are where businesses capture and measure value through ad impressions, subscriptions, sales, leads, analytics, retargeting, and customer relationships. A useful AI answer may still influence a customer, but if the customer never visits the website, the business may lose the traffic, data, and direct relationship that made online discovery valuable in the first place.

AI search and AI answers are beginning to change the website economy. The website may still be the source of the knowledge, but it may no longer be the place where every customer interaction happens. Businesses may need to rethink what a website is for when people, AI search systems, and AI agents can use its information to compare options, answer questions, and influence purchases before a visitor ever arrives.

Businesses still need AI systems to find their content. They also need ways to benefit when that content is used. The answer may look different for a publisher, an ecommerce company, a software business, a healthcare provider, a consultant, or a local service business. For now, many of the models are still experimental.

That is why Cloudflare's announcement should not be read only as a publisher story. Smarter crawling, content controls, AEO reporting, compensation experiments, and AI discoverability tools are all early attempts to answer the same business question: how does my business benefit when AI uses my knowledge?

Q: How can a business benefit when AI uses its knowledge?
A: A business can benefit from AI discovery if answer engines use its website content in ways that create measurable value, such as visibility, trust, leads, subscriptions, sales, referrals, or compensation. The challenge is that AI answers may use a company's knowledge before the customer ever visits the website, making traffic alone a less complete measure of discovery.

Q: What is Cloudflare trying to change with AI search?
A: Cloudflare is trying to give website owners more control, visibility, and possible compensation when AI systems use their content. Its AI search work includes signals for fresher content, tools to reduce unnecessary crawling, reporting on AI search visibility, and Pay Per Use experiments tied to content use.

Q: Why is website traffic becoming a less complete measure of discovery?
A: Website traffic measures visits, but AI discovery may happen before a visit or without a visit at all. If an AI answer uses a company's product page, service page, documentation, or article to answer a customer question, the business may influence the decision even when analytics never record a click.

Q: How are Cloudflare and Google handling AI access differently?
A: Google gives site owners some separation through Google Extended, which can opt content out of some AI training and AI product uses without removing it from Google Search. Googlebot still supports Google Search features, including AI Overviews and AI Mode. Cloudflare is pushing for more granular crawler separation, AI search reporting, and economic models that work beyond Google's search system.

Q: Does AI discovery affect businesses outside publishing?
A: Yes. Publishers are the clearest example because their business models often depend on traffic, advertising, and subscriptions. The same discovery problem can affect manufacturers, ecommerce companies, software firms, healthcare providers, consultants, and local businesses whose websites contain information AI systems may use to answer customer questions.

Q: Is Pay Per Use already a proven business model?
A: No. Pay Per Use is still experimental. Cloudflare is testing the idea with partners such as Ceramic.ai and You.com to see whether payment can follow content use in AI search results or AI agent access to premium content instead of crawler activity alone.

Q: What should businesses do now about AI search?
A: Businesses should keep investing in website traffic while also tracking whether AI systems can find, understand, and accurately use their expertise. The next measurement question is not only how many people visited the website, but whether AI used the company's knowledge in a way that created value.

Cloudflare's announcement shows that AI discovery is moving from a search visibility issue to a business infrastructure issue. SEO rankings and website visits still matter, but AI answers are creating another layer of discovery where company knowledge can influence customers before a click happens.

The key point is that website value may need to be measured beyond traffic. A website can still shape an AI answer, recommendation, or customer decision because its product pages, service pages, documentation, articles, and policies can provide the information an answer engine uses, even when no one lands on the page. That creates a measurement gap for businesses that have treated visits as the main proof that their content is working.

Business owners, marketers, publishers, ecommerce operators, SaaS companies, consultants, healthcare providers, and local businesses should care because their websites are becoming source material for AI systems. The more useful, original, and decision-oriented that content is, the more important it becomes to know whether AI can understand it, surface it accurately, and return value to the company.

The pressure to make those decisions is growing as non-human traffic and AI crawling become larger parts of the web. Cloudflare says more than 50% of online traffic is already non-human, and AI search can require repeated web requests to answer a single user prompt. That makes crawler permissions, content structure, AEO reporting, and payment models business decisions rather than technical details.

Businesses still need website traffic, but they also need to decide whether traffic alone is enough to measure discovery. In a web where AI may become the customer's first stop, and sometimes their only stop, companies may need to track whether AI systems can find their expertise, use it accurately, and return value to the business even when no website visit happens.

In short, the website is becoming both a destination for people and a source layer for AI answers. Cloudflare's approach does not fully settle the business value issue, but it shows that the economic layer of AI discovery is beginning to be built.

The next measure of online discovery may be whether AI used your knowledge in a way that created value for your business, even when the customer never clicked.

Editor’s Note: This article was created by Alicia Shapiro, CMO of AiNews.com, with writing support, AEO/GEO/SEO optimization, image concept development, and editorial structuring support from ChatGPT, an AI assistant. All final editorial decisions, perspectives, and publishing choices were made by Alicia Shapiro.