Google asks sites to wave its agent through to the cart
Published 2026-11-18 ·
primary source
What it means
Google now tells sites how to let its agent past the firewall, all the way to add-to-cart and booking, from a pool of IPs that many users share. That settles who gets in, not who gets counted: GA4's bot exclusion is invisible and names no agent, Shopify tightened its own cart and checkout bot filter a day earlier, so the same agent cart can count in one report and vanish from the other. Match the published IP list in your own logs and label the traffic yourself.
On 17 November Google published a guide called Allow Google-Agent traffic. Google-Agent is the name its agents use when they browse a site and act for a user. The guide asks your firewall, CDN and bot service to skip bot challenges for it, past the homepage and into search, filters, add-to-cart and booking. That decides whether the agent gets in. It does not decide whether your analytics counts what it does there, and Google’s guide does not mention analytics.
What Google asks for
Three checks identify the agent, and Google says to skip bot challenges when they pass:
- the
User-Agentheader containsGoogle-Agent; - the IP address is in Google’s published list,
user-triggered-agents.json; - where present, the request signature verifies. Google is testing
Web Bot Auth, signed HTTP requests (RFC 9421) from the identity
agent.bot.goog. We fetched its key directory today: five Ed25519 keys.
The troubleshooting list is where the detail is. The allow rule should run before data-centre ASN
blocks, managed bot rules and “category-level AI agent block policies”. It should skip in-page
CAPTCHAs on search and form submission too, and cover the XHR, GraphQL and api.* calls a cart makes.
And per-IP rate limits should exempt the agent’s ranges, because concurrent requests from many users
share one egress pool.
robots.txt is not on the list. Google’s user-triggered fetchers page says fetchers that act on a user’s request “generally ignore robots.txt rules”, Google-Agent among them.
What we checked
We downloaded the agent’s IP list and Google’s four other crawler and fetcher lists today.
| List | IPv4 prefixes | IPv4 addresses |
|---|---|---|
user-triggered-agents.json (Google-Agent) | 12 | 70,016 |
user-triggered-fetchers.json | 544 | 17,408 |
user-triggered-fetchers-google.json | 252 | 8,064 |
common-crawlers.json | 170 | 5,088 |
special-crawlers.json | 136 | 4,352 |
The agent’s pool is twice the size of the other four lists combined, 70,016 addresses against
34,912, and 65,536 of them sit in one block, 136.122.0.0/16. None of its 20 prefixes overlaps any
other Google list. So an IP match tells agent traffic apart from Googlebot and the other fetchers
without trusting the user agent string.
That string has a trap. In the mobile version, (compatible; Google-Agent; …) comes last, like most
bot tokens. In the desktop version it sits inside the (KHTML, like Gecko …) brackets, before
Chrome/. A log filter or a regex that expects the bot token at the end of the string catches the
phones and misses the desktops. Match Google-Agent anywhere.
The read: who counts the agent’s cart
Your firewall and your analytics make two separate bot decisions. Allowing Google-Agent at the edge does not tell GA4 or Shopify anything. And each tool makes its call where you cannot see it:
- GA4 excludes known bots using Google’s research and the IAB bots list. You cannot turn it off, and you cannot see what it removed. The page does not name Google-Agent. We could not test what GA4 does with an agent session, because only Google’s agent sends one.
- Shopify improved its bot detection for cart and checkout activity on 16 November, a day before Google’s guide. Human-filtered sessions may dip slightly. Orders and sales are not affected. Shopify does not say which agents its filter calls bots.
So the same add-to-cart can show in one report and not in the other. The order, if one follows, counts everywhere, because orders are outside both filters. A cart-to-order rate then moves for a reason no campaign caused. That is the conversion rate denominator problem again: the number of orders holds, and the number it is divided by changes.
One IP now means many people. Google says its agent’s requests from different users share an egress pool. A rate limit is the obvious casualty, but the same goes for anything else that scores by IP: a click-fraud rule that flags many carts from one address, a lead form that drops repeat submissions per IP. Each of those reads one busy agent address as one suspicious visitor.
Nothing in the visit says which campaign sent the user. An agent that a user sends to your site arrives from Google’s infrastructure, not from your ad. If a sale closes there, the touch that started it may live in a Google product your reports do not see, and even GA4’s AI Assistant channel only counts clicks that arrive with an AI referrer. We read nothing in Google’s pages on how, or whether, that hand-off is reported.
What to do
- Allow the agent with all three checks, not the user agent alone, and refresh the IP list from
user-triggered-agents.jsonon a schedule. Google’s guide says to keep IP rules synced with the latest ranges. - Label agent traffic yourself. Match the IP list in server logs, or in your
server-side tagging setup, and send an
agentflag with the event. Then you see the agent carts instead of guessing what GA4 or Shopify filtered. - Take the agent’s ranges out of per-IP rules in fraud scoring and form deduplication, or key those rules on something other than the IP.
Google-Agent is the AI visitor that robots.txt does not stop. The training bots and search bots of OpenAI, Anthropic and Perplexity are another matter, and blocking one of them does less than most sites think.
Sources
- Google's crawling documentation changelog, entry dated 17 November 2026: 'Added guidance on allowing Google-Agent', a technical guide on how to ensure that Google-Agent has access to your website.
verified 2026-11-18 - Google's Allow Google-Agent traffic page (last updated 2026-11-17) says Google-Agent is the name used by agents hosted on Google infrastructure to navigate the web and perform actions upon user request, and that each request has a User-Agent header containing Google-Agent and comes from an IP address in user-triggered-agents.json.
verified 2026-11-18 - The same page says Google is experimenting with the Web Bot Auth protocol using the https://agent.bot.goog identity, signing Google-Agent requests with HTTP Message Signatures (RFC 9421), with a Signature-Agent header set to g="https://agent.bot.goog".
verified 2026-11-18 - The same page gives three checks for a service that does not verify Google-Agent itself (the User-Agent contains Google-Agent, the IP is in user-triggered-agents.json, and the signature verifies when provided), and says: for verified requests, skip bot challenges.
verified 2026-11-18 - The page's troubleshooting list says the allow rule should evaluate before broad cloud or data center ASN blocks, managed bot rules and category-level AI agent block policies, and should skip interactive CAPTCHAs and JavaScript challenges, including in-page CAPTCHA widgets on search or form submission.
verified 2026-11-18 - The same list says the allow rule should cover actions beyond homepage loads, including on-site search, filtering, detail pages, add-to-cart, booking, and underlying XHR, GraphQL or api.* endpoints, and that per-IP rate limits should account for Google-Agent ranges because concurrent requests from multiple users share the same egress pool.
verified 2026-11-18 - Google's user-triggered fetchers page says these fetchers generally ignore robots.txt rules because the fetch was requested by a user. It lists Google-Agent's mobile user agent with '(compatible; Google-Agent; +https://developers.google.com/crawling/docs/crawlers-fetchers/google-agent)' after the Safari token, and its desktop user agent with 'compatible; Google-Agent' inside the '(KHTML, like Gecko …)' parentheses, before the Chrome token.
verified 2026-11-18 - user-triggered-agents.json, fetched 18 November 2026 (creationTime 2026-11-17T15:45:49), lists 20 prefixes: 12 IPv4 prefixes covering 70,016 addresses, 65,536 of them in 136.122.0.0/16, and 8 IPv6 prefixes.
verified 2026-11-18 - user-triggered-fetchers.json, fetched 18 November 2026, lists 544 IPv4 prefixes covering 17,408 addresses. None overlaps a prefix in user-triggered-agents.json.
verified 2026-11-18 - user-triggered-fetchers-google.json, fetched 18 November 2026, lists 252 IPv4 prefixes covering 8,064 addresses. None overlaps a prefix in user-triggered-agents.json.
verified 2026-11-18 - special-crawlers.json, fetched 18 November 2026, lists 136 IPv4 prefixes covering 4,352 addresses. None overlaps a prefix in user-triggered-agents.json.
verified 2026-11-18 - common-crawlers.json, fetched 18 November 2026, lists 170 IPv4 prefixes covering 5,088 addresses. None overlaps a prefix in user-triggered-agents.json.
verified 2026-11-18 - The Web Bot Auth key directory at agent.bot.goog, fetched 18 November 2026, returned five Ed25519 (EdDSA) verification keys.agent.bot.googverified 2026-11-18
- GA4's Known bot-traffic exclusion page says traffic from known bots and spiders is automatically excluded, identified using a combination of Google research and the IAB's International Spiders and Bots List, and that you cannot disable the exclusion or see how much known bot traffic was excluded.
verified 2026-11-18 - Shopify's changelog, 16 November 2026: Shopify Analytics improved how it detects bot traffic in cart and checkout activity, reports with a human-based filter might show a small dip in sessions, and orders, sales and customer data are not part of the change.
verified 2026-11-18