Blocking Google-Extended will not remove you from AI Overviews.

Blocking Google-Extended is not a switch for leaving AI Overviews. It covers different uses of your content. Choose a rule based on the product you want to control.

Our historical sample found 89 blocks among 1,020 publisher files. That is a count of public rules. It does not show that those publishers misunderstood the token or wanted to leave Search.

What Google-Extended controls

Google's current crawler documentation describes two uses covered by Google-Extended:

• Training future Gemini models for the products Google names.
• Grounding in Gemini Apps and Grounding with Google Search on Vertex AI. Grounding means supplying retrieved information when a model answers a question.

This is broader than training alone. It is still separate from Google Search inclusion and ranking.

Four controls that do different jobs

Google updated its content-control guidance on October 21, 2026. Choose the control that matches your aim:

The Search generative AI control covers AI Overviews, AI Mode and generative AI features in Discover. Google says excluding a site removes its links and content from those features. It is not a ranking or inclusion signal for other parts of Search.

That choice is private. Reading your public robots.txt cannot reveal its value. Before changing it, compare the available Search Console reports. AI impressions show visibility, not the clicks or revenue you would have received under a different setting.

Changes to page instructions require Google to recrawl and process them. A new rule is not an instant confirmation that a result has changed.

What our sample actually tells you

The sample describes 1,020 robots.txt files from a historical frame of 16,076 domains listed by two ad networks. Of those files, 89 blocked Google-Extended at the root.

We cannot infer the owner's reason. Some publishers may want exactly this separation: limit the specified Gemini uses while keeping Google Search available.

We also did not test vendor compliance. A public rule states a preference; it is not a record of what a crawler did afterward.

Check your own rule before changing it

1. Open your site's /robots.txt file.
2. Find the Google-Extended group, if one exists.
3. Decide whether your aim concerns Gemini training, the named grounding uses, Search crawling or Search previews.
4. Read the current documentation for that specific control.
5. Test the resulting file and inspect the page's indexing instructions before publishing.

Use our free public-site analysis to inspect visible crawler rules. It cannot verify private account settings, crawler compliance or your site's actual appearance in an AI answer.

Scope of this check

We checked this explanation again on October 22, 2026, including the separate Search Console control. The earlier wording described the token too narrowly as training-only and suggested motives the sample could not establish. The historical count is unchanged. This is not a fresh census of crawler policies.

Begin with the outcome you actually want

“Block AI” is too broad to describe a configuration. You might want to limit a named training use while staying discoverable in Search. You might want a page to stay searchable but reduce the text shown in a preview. You might want confidential information unavailable to anyone without authorization.

Those goals need different controls and different checks. Write the intended outcome first, then identify the product, content and scope. A public token with an AI-related name is not automatically the switch for every AI feature made by the same company.

For a chosen planning example, a publisher wants public research discoverable through ordinary Search but does not want to permit the uses governed by Google-Extended. That is not an internally inconsistent aim. The publisher must examine the distinct controls and record the choice instead of block all Google crawling by accident.

For another example, the concern is a private customer report. A crawler preference is not access control. The report needs appropriate authentication and server authorization, not merely a robot rule announcing that the private path exists.

Crawler permission and indexing instructions are separate

Google's robots introduction explains that crawl controls are not the general mechanism for keeping an HTML page out of Search. A URL can be known through other evidence even when its body is not crawled.

Google's page-instruction reference describes meta tags, headers and text-level preview controls. The crawler needs access to read the relevant page-level instruction. Blocking the fetch and then expecting a newly added meta instruction to be read creates a practical conflict.

Keep those two layers in the implementation review. Inspect the root robots response and the actual page response, including relevant headers. A setting saved in a content-management screen does not prove the deployed response has the intended instruction.

A public scan can report which instructions appeared in its sampled response. It cannot prove that Google has processed the change or predict exactly when a result will disappear. The appropriate Search tools and later observations are another part of verification.

A product token does not need to appear as a separate request

Google describes Google-Extended as a robots control token rather than a distinct HTTP user-agent string for a separate crawler. That matters when you inspect logs. Not finding requests literally named Google-Extended does not prove the preference is unused.

Conversely, a request carrying a familiar crawler name does not by itself establish that it is genuine. User-agent text can be copied. If the decision depends on who actually requested content, use the operator's verification guidance and the available network evidence.

Keep the public policy reading separate from observed requests. A parsed block is evidence of a declared rule. A verified log entry is evidence of a request under the conditions recorded. An edge access decision is evidence of enforcement. None automatically supplies the other two.

This is especially important when a dashboard labels all three as an “AI block.” Ask whether the number counts rules, attempted requests, successful responses or blocked requests. The unit changes the meaning.

A private Search setting cannot be recovered from robots.txt

The separate Search generative-AI setting is managed through Search Console, not encoded as a simple public Google-Extended declaration. Its current official documentation also describes property inheritance. The effective value for a child property can therefore require inspecting its parent relationship.

A public checker should say it cannot see that private setting. It should not label a site opted in or out solely because Google-Extended is allowed or blocked. It should not ask for account credentials to guess the answer either.

For your own record, inspect the effective setting in the authenticated property you intend to manage. Save the date and relevant scope. If a parent and child property differ, understand the effective configuration before drawing a conclusion about the whole domain.

Do not turn this into a universal recommendation to exclude or include. The commercial and editorial trade-off depends on the site and its actual evidence. This guide distinguishes the controls; it does not estimate the traffic or revenue under an unobserved alternative choice.

What the historical count does not establish

The eighty-nine blocks in the sample tell us that the resolved root rule was present for that token under the method. They do not tell us whether the owner wrote it, copied a template, intentionally kept Search available or misunderstood a product name.

A shared comment in many files may suggest common source text. It still does not establish that nobody reviewed the template. An unusual rule may be deliberate. A missing explicit group may interact with a wildcard rule. Read the effective policy instead of count names alone.

The sample also does not observe downstream model training, answer generation or compliance with every rule. A robots file is not a receipt proving that a company's internal use changed. Public research needs to stop at what the collected evidence can establish.

Keep the historical sampling frame and observation date visible. A current explanation of controls can accompany an older count, but it does not turn that count into today's adoption rate. Rerunning a parser over cached files is a method check, not a fresh web census.

A decision table for a small publisher

Write one row per intended outcome. Include the affected content, relevant product, current control, desired change, source and a verification step. Keep an unresolved column for facts requiring private account access.

For a public article remaining in Search, the verification step may involve checking its deployed access and indexing instructions. For a Search AI participation choice, it involves the authenticated effective setting and subsequent relevant reporting. For a named training preference, it involves the resolved public directive and the current operator definition.

Do not attach a green “protected from all AI” badge to any of these rows. Each has a narrower scope. A copied robot rule cannot make a public article confidential or guarantee that every operator on the internet honors the preference.

The table is valuable because it makes mistakes visible before deployment. It can reveal that a proposed broad Googlebot block would affect more than the intended product, or that the desired outcome concerns previews rather than crawling at all.

Test a change without changing every layer at once

Save the original robots file, page instructions and effective private settings relevant to the decision. Record the date. Then make the smallest configuration change that matches the stated aim.

Inspect the live response after publishing. Check that the file is plain text, served at the intended root and not replaced by an HTML error template. Verify that unrelated routes and essential resources remain accessible under the intended policy.

If you change a private setting, confirm its effective value in the correct property. Do not infer success from a public scanner that has no access to that setting. Allow for the documented processing and caching behavior before interpreting external appearance.

Keep a rollback path for accidental broad blocks. A narrow intended preference should not silently remove public content from ordinary discovery or break the site's own resources. If the result differs from the goal, restore the known configuration and investigate the specific mismatch.

Measure effects without inventing a counterfactual

A later change in impressions or visits is an observation. It does not by itself prove that the crawler or Search setting caused it. Query demand, rankings, publication changes, seasonality and search presentation can move at the same time.

Use complete comparable periods and preserve the relevant report type. AI-feature visibility, ordinary search clicks, on-site sessions and ad revenue describe different stages. Do not call impressions revenue or turn a visibility count into visits you would otherwise have received.

If a setting excludes the site from a feature, it changes eligibility under the documented rule. Estimating the business effect still needs actual measurement and careful assumptions. An ordinary before-and-after comparison may remain uncertain.

The same applies to training preferences. A public block can be verified as a declaration without proving a measurable revenue gain. Keep the policy decision and the economic evaluation separate so each can be discussed honestly.

Maintain the explanation as products change

Record the official source and date when a product's scope changes. Keep definitions in a shared source table for articles and tools so a corrected explanation does not leave an old tooltip telling a different story.

Review the user's intended outcome after a meaningful provider update, a site migration or an ownership change. The right rule for an earlier publishing strategy may need reconsideration. Do not silently assume the original template remains the desired policy forever.

For readers, the useful end point is simple: which use you are controlling, which setting expresses that choice and what evidence verifies its actual scope. Inspect your public signals for the part that can be seen, then use the official authenticated tools for the private part.

Give the choice an owner inside the publishing team

A site can have several people changing templates, hosting rules and Search settings. Assign responsibility for the policy and the deployment check. Otherwise a theme update can restore a default that no longer matches the publisher's decision.

Keep a short change record with the purpose, source, scope, date and reviewer. This is especially useful when a vendor manages part of robots.txt. Ask which system generates the final file and which settings can overwrite it. Inspect the deployed response rather than rely only on the saved editor value.

Also state what the policy does not replace. It is not a copyright determination, a contract with every reader or an access-control system for confidential records. Those questions need their own appropriate review.

Clear ownership makes a narrow technical preference more reliable over time. It avoids the same product-name confusion returning in a later template, translation or scanner result.