The News, Explained
Cloudflare announced a system for continuously evaluating Verified bot behavior on October 16, 2026. A bot is software that requests web pages without someone manually directing every request. The new system observes whether bots follow a site’s published crawling preferences and respond when asked to reduce requests. Cloudflare is using it internally now, with customer availability presented as a future plan. Source
From identification to ongoing observation
Cloudflare sets out three broad criteria for Verified bots: identify themselves accurately and securely, perform only their stated work, and avoid abusive behavior. The new system examines whether continuing requests uphold the principle of avoiding harm to sites. It adds evidence about behavior after verification. Source
The observations are intended to produce an explainable “trust band.” This is a range describing how consistently a bot respects site preferences and which observed behaviors contributed to the assessment. Cloudflare says it focuses on sustained patterns rather than isolated mistakes. Its current internal phase is building that history and refining how observations become useful guidance. Source
Crawl paths and request rates are separate signals
The first signal is compliance with robots.txt, a file through which a site communicates crawling preferences, including which paths bots should visit. The system observes whether actual traffic consistently follows that guidance over time. Cloudflare also says bots should respect other signals through which a site communicates that access is unwelcome or requests should slow down. Source
The second signal is behavior after an HTTP 429 response, the server’s message that there are too many requests. It tells the visitor to reduce the request rate and sometimes provides guidance on how long to wait before retrying. Slowing down or pausing increases trust in the assessment, while repeatedly ignoring that feedback lowers it. Source
| Signal | What the site communicates | Behavior observed |
|---|---|---|
| robots.txt | Preferences about crawl paths and scope | Whether requests consistently follow the published guidance |
| HTTP 429 | Feedback to reduce an excessive request rate | Whether the bot slows down or pauses after receiving it |
Separating these signals makes room to assess both respect for crawl boundaries and responsiveness to server load. Cloudflare explains that a bot can remain on allowed paths while still placing a substantial burden on a site through the volume of its requests. Source
Internal evaluation now, customer controls later
The current stage is internal evaluation. Cloudflare plans to publish scores on Radar, its public data service, and expand the measurements. It also plans to make the system available to all Cloudflare customers, allowing them to select the trust threshold used in handling Verified bot traffic. A threshold is the boundary that determines which assessment level is used as a handling criterion. Source
The announcement gives no customer release date, numeric scoring range or default threshold. The distinction is between ongoing internal observation today and the planned public scores and customer choice of thresholds. Source
OYOPICK’s Take
We see value in considering a bot’s identity alongside its actual behavior. Observing compliance with crawl guidance separately from responses to excessive-request feedback could help a site owner judge more specifically why a bot deserves trust. It offers a basis for considering how well a bot respects that particular site’s signals, beyond a blanket choice to allow or block automated collection.
For future threshold selection to be useful, the observations behind each band will need to be understandable to site owners. Explaining whether a bot earned its assessment through crawl-path compliance, request-rate adaptation or both would give the choice more context. We hope the public scoring stage provides that explanation and gives bot operators feedback they can use to improve behavior.