Guide · Updated November 9, 2026
How to rank in Claude
To rank in Claude, get your pages into the results of the web searches Claude runs, then give it something worth quoting. Allow Claude-SearchBot and Claude-User, publish honest comparison pages with current prices, and get onto the lists Claude already cites. Blocking ClaudeBot only affects training.
7 days free. No card required.
Slibur tracks
ChatGPT
Claude
Google AI
Perplexity
Grok

How Claude searches the web and picks the pages it cites
Claude answers from two places: what it learned in training, and what its web search finds when a question needs fresh facts. Anthropic documents both, and the details shape how you get in.
What Claude already knows
Anthropic lists a knowledge cutoff for each model: June 2026 for Claude Haiku 5.5, Sonnet 5.5 and Opus 5.5. A product, a price or a brand that appeared after that month is unknown to the model until a search turns it up.
When Claude decides to search
Its API documentation says Claude searches when a request depends on information that is current, changing or outside its training data, such as prices or products that may have changed. Stable facts and concepts it answers directly. In the newer claude.ai experience there is no search switch at all: Claude searches when it helps.
What a search hands Claude
In the API, every result arrives with a URL, a title and a page age, the date the site was last updated. Citations are always on, and each one carries up to 150 characters of the passage Claude relied on. When someone pastes a link into claude.ai, web fetch reads that page in full.
Anthropic’s help article on web search (opens in a new tab) does not name the engine behind Claude’s text results; it says image results are powered by Bing, and that Claude may use a location inferred from the person’s IP address for local questions. Its API documentation (opens in a new tab) adds that Anthropic runs the searches and hands Claude the results, which it can filter before reading them.
Two consequences for you. First, a product built on Claude can limit its searches to a list of domains or exclude some, so the same question can draw on different sites in different apps. Second, Claude quotes passages, not whole pages: the sentence that states your price or your fit is the unit that gets cited. Optimizing for Claude means being found by its search and being quotable once you are.
Anthropic’s crawlers: the token that lets Claude find you
Anthropic runs three robots and one hosted browser, each with its own user-agent token. Two of them decide whether Claude can find and read your pages.
Scroll sideways to see every column.
| Token | What Anthropic says it does | If you block it | For ranking in Claude |
|---|---|---|---|
| Claude-SearchBot | Crawls the web to improve the relevance and accuracy of Claude’s search results. | Your content is not indexed for search, which Anthropic says may reduce your visibility in user search results. | Allow it. This is the token that decides whether Claude’s search can find you. |
| Claude-User | Visits a page when a person asks Claude something that needs it. | Claude cannot retrieve your page to answer a user’s question, which may reduce your visibility in user-directed search. | Allow it, or Claude cannot open your page even when a buyer asks about it. |
| ClaudeBot | Gathers web content that may go into training Anthropic’s models. | Your future pages are left out of training datasets. | Your choice. Blocking it does not take you out of Claude’s search. |
| Claude-Agent | A real browser on Anthropic’s servers that completes tasks for a user, for example in Claude Duet. It adds the Claude-Agent token to an ordinary browser user agent. | People cannot have Claude complete tasks on your website through it. Your rules for the three robots above do not cover it. | Not part of search. Refuse it only if you do not want agents acting on your site. |
A robots.txt that keeps you in Claude’s search
Copy it to the root of each subdomain. Leave out the last group if you are happy for your pages to be used in training.
robots.txt for Claude
# Stay in Claude’s search and in answers to user questions User-agent: Claude-SearchBot Allow: / User-agent: Claude-User Allow: / # Optional: keep your pages out of model training User-agent: ClaudeBot Disallow: /
What else Anthropic documents
A general “no AI training” signal. Since October 30, 2026, Anthropic has begun honoring a robots.txt group for the token __usage_ai-training, from a draft in the IETF’s aipref work (opens in a new tab). A rule that names ClaudeBot wins over it. Only 2 of the 38 big sites below use it so far: LinkedIn and X.
Firewalls count too. Anthropic says blocking its IP addresses is not a reliable way to opt out. To get in, the firewall matters as much as robots.txt: a bot filter can keep Claude away even when the file lets it in. Its published list (opens in a new tab) had 14 IPv4 ranges, dated , when we read it.
A user block holds. Anthropic says its robots follow robots.txt, Claude-User included. OpenAI and Perplexity say their user-triggered fetchers may not.
The same three jobs on ChatGPT and Perplexity
Scroll sideways to see every column.
| Job | Claude | ChatGPT | Perplexity |
|---|---|---|---|
| Indexes pages for search answers | Claude-SearchBot | OAI-SearchBot | PerplexityBot |
| Reads a page when a user asks | Claude-User. Anthropic says its bots honor robots.txt | ChatGPT-User. OpenAI says robots.txt rules may not apply | Perplexity-User. Perplexity says it generally ignores robots.txt |
| Collects content for training | ClaudeBot | GPTBot | None listed. PerplexityBot is not used for training |
The other two sides, step by step, are in our guides on how to rank in ChatGPT and how to rank on Perplexity.
Which big US sites let Claude in
The review sites and publications you want to be listed on have robots.txt files of their own. We read the files of 50 large US sites on November 9, 2026, and applied the matching rules of RFC 9309 to each Anthropic token. 38 could be read.
13 of 38
shut out Claude-SearchBot, the token Claude’s search depends on
22 of 38
shut out ClaudeBot, the training crawler
9
block training and still let Claude’s search in
15
still name anthropic-ai or Claude-Web, tokens missing from Anthropic’s current list
Scroll sideways to see every column.
| Site | Kind of site | Claude-SearchBot | Claude-User | ClaudeBot |
|---|---|---|---|---|
| g2.com | Reviews and directories | Open | Open | Open |
| capterra.com | Reviews and directories | Open | Open | Open |
| trustpilot.com | Reviews and directories | Open | Open | Open |
| softwareadvice.com | Reviews and directories | Open | Open | Open |
| tripadvisor.com | Reviews and directories | Open | Open | Shut out |
| glassdoor.com | Reviews and directories | Open | Open | Shut out |
| yelp.com | Reviews and directories | Shut out | Shut out | Shut out |
| youtube.com | Communities | Open | Open | Open |
| github.com | Communities | Open | Open | Open |
| medium.com | Communities | Open | Open | Shut out |
| linkedin.com | Communities | Open | Shut out | Shut out |
| reddit.com | Communities | Shut out | Shut out | Shut out |
| quora.com | Communities | Shut out | Shut out | Shut out |
| x.com | Communities | Shut out | Shut out | Shut out |
| techradar.com | News and buying guides | Open | Open | Open |
| tomsguide.com | News and buying guides | Open | Open | Open |
| nerdwallet.com | News and buying guides | Open | Open | Open |
| forbes.com | News and buying guides | Open | Open | Shut out |
| businessinsider.com | News and buying guides | Open | Open | Shut out |
| healthline.com | News and buying guides | Open | Open | Shut out |
| webmd.com | News and buying guides | Open | Open | Shut out |
| nytimes.com | News and buying guides | Shut out | Shut out | Shut out |
| wsj.com | News and buying guides | Shut out | Shut out | Shut out |
| cnet.com | News and buying guides | Shut out | Shut out | Shut out |
| theverge.com | News and buying guides | Shut out | Shut out | Shut out |
| wired.com | News and buying guides | Shut out | Shut out | Shut out |
| zdnet.com | News and buying guides | Shut out | Shut out | Shut out |
| investopedia.com | News and buying guides | Shut out | Shut out | Shut out |
| bloomberg.com | News and buying guides | Shut out | Shut out | Shut out |
| etsy.com | Retail | Open | Open | Open |
| walmart.com | Retail | Open | Open | Open |
| target.com | Retail | Open | Open | Open |
| costco.com | Retail | Open | Open | Open |
| ebay.com | Retail | Open | Open | Shut out |
| amazon.com | Retail | Shut out | Shut out | Shut out |
| en.wikipedia.org | Reference | Open | Open | Open |
| britannica.com | Reference | Open | Open | Open |
| consumerreports.org | Reference | Open | Open | Open |
Shut out: the rules that apply to that token forbid an ordinary page. Open: they allow it, though some sections may still be off limits. Twelve sites could not be read: ten answered with a bot check, an error or nothing, and two sent us to an international edition. We fetched each file with an ordinary browser user agent; a site can serve crawlers a different one, and a firewall can block what robots.txt allows.
Review sites mostly let Claude in. Of the seven we could read, only Yelp shuts out Claude-SearchBot. G2, Capterra, Trustpilot and Software Advice let all three tokens in; Tripadvisor and Glassdoor block training only. News sites are the opposite: 8 of 15 shut out Claude’s search, among them The New York Times, CNET, Wired and Investopedia.
Pick your targets with this in mind. 16 sites let all three tokens in, from Wikipedia to Walmart. Reddit, Quora and X shut out every one, LinkedIn lets the search crawler in but not Claude-User, and Amazon closes its doors to all of them. Before you pitch a site for a mention, read its robots.txt. That minute is part of how to rank in Claude: a page its search cannot index is a page Claude is unlikely to quote.
What Claude cited for 12 buyer questions
On November 9, 2026, we put 12 questions a US buyer might ask to Claude Haiku 5.5 through Anthropic’s API, with Anthropic’s own web search tool, a US location and one run each. No question named a brand.
Scroll sideways to see every column.
| Question | Searches | Results back | Pages cited | Two of the sites cited |
|---|---|---|---|---|
| “What is the best CRM for a 10-person sales team?” | 1 | 10 | 4 | pipelinecrm.com, superoffice.com |
| “Best project management software for a small marketing agency” | 1 | 10 | 7 | avaza.com, teamwork.com |
| “Which payroll service is best for a US startup with 20 employees?” | 2 | 19 | 6 | ifeeltech.com, equalpaytoday.org |
| “Best running shoes for flat feet under $150” | 1 | 9 | 4 | runrepeat.com, marathonsports.com |
| “Best office chair for back pain under $500” | 2 | 19 | 4 | btod.com, aboutdarwin.com |
| “Best accounting software for freelancers” | 2 | 20 | 5 | lessaccounting.com, smallbusinesssoftwareguide.com |
| “Which email marketing platform is best for a small online store?” | 2 | 19 | 6 | drip.com, sender.net |
| “Best mattress for side sleepers with shoulder pain” | 2 | 20 | 8 | sleepfoundation.org, aarp.org |
| “Best robot vacuum for pet hair” | 1 | 10 | 7 | tomsguide.com, consumerreports.org |
| “How do I choose a web hosting provider for a small business website?” | 0 | 0 | 0 | No search: answered from training |
| “Best Italian restaurant in downtown Chicago for a group dinner” | 1 | 9 | 4 | tripadvisor.com, theinfatuation.com |
| “Best invoicing app for contractors” | 1 | 9 | 5 | joist.com, servicebusinessacademy.org |
In all: 16 searches, 154 results back and 60 pages cited, about 39 in 100 of what the searches returned. One run on one model is a snapshot: claude.ai, another model or another day can answer the same question differently.
Seven things the answers show
It searched for 11 of 12. The exception was the how-to question about web hosting. Claude treated it as stable knowledge, the case in which its API documentation says it answers without searching.
It added the year on its own. 9 of its 16 searches ended in “2026”, though no question mentioned a year. The searches were short rewrites of the question, such as “best CRM for small sales team 2026”.
Second searches went after prices. 5 answers ran a second search. 3 of those asked for pricing, and two named the leaders outright: “Gusto vs Rippling vs ADP pricing comparison” and “Klaviyo vs Mailchimp pricing small ecommerce”.
Lists and comparisons won. 59 of the 60 cited pages had a list or comparison title (best, top, a number, vs, alternatives or compared), and 47 carried 2026 in the title or the address.
Vendors were cited for their own categories. In 7 of the 11 answers, at least one cited page was a “best of” list published by a company that makes a product in that category: Pipeline CRM and SuperOffice on CRM, Drip and Sender on email marketing, Joist on invoicing.
Few household names. The 60 citations came from 58 different sites and no site appeared in two answers. None was a .edu or .gov page, and only five of the sites were among the 38 big ones in the table above.
Blocking training did not block citations. Of the 57 cited sites whose robots.txt we could read, 5 shut out ClaudeBot and were cited anyway, Forbes and Tripadvisor among them. None shut out Claude-SearchBot or Claude-User, and none of the 154 results came from the 13 big sites that do.
How to rank in Claude: 7 steps
Each step comes from Anthropic’s documentation or from the answers above. None of them is a trick, and none can force Claude to name you.
Open the two doors Claude uses
Allow Claude-SearchBot and Claude-User in robots.txt, on every subdomain you want found. Then look past robots.txt: a firewall or bot filter can turn Anthropic away before it reads the file, and Anthropic publishes the IP ranges of its crawlers so you can let them through. If you want out of model training, block ClaudeBot; it is a separate token. The free AI crawler checker lists, token by token, what your robots.txt allows for Claude.
Rank for the searches Claude runs
Claude turns a buyer’s question into a short web search, often with the current year added, and sometimes into a second one that compares the leaders on price. Write those searches down for your main questions and make sure that a page of yours, or a page that lists you, ranks for them. Through the API every query comes back in the response; on claude.ai you see when Claude is searching.
Publish the comparison Claude is looking for
Nearly every page Claude cited in our sample was a ranked list or an “X vs Y” page. If you sell in a category, publish an honest “best X for Y” or “X vs Y” page on your own site: name the rivals, give real prices and say who each option suits, yourself included. Claude cited pages like that from vendors in most of our answers.
Write facts Claude can quote
A Claude citation carries up to 150 characters of the passage it leans on. Put each fact you want repeated (the price, who it is for, a key spec, the city you serve) in one short sentence under a heading that matches the question. Keep it in the HTML itself: Anthropic does not say whether its crawler runs scripts, so text that only appears after JavaScript is a risk you do not need.
Keep prices and dates current
Each search result reaches Claude with its page age, and its follow-up searches ask for prices. Update prices and plan names when they change, show the date of the last real update, and take down pages that still quote old figures.
Get onto the pages Claude already cites
Ask Claude the questions your buyers ask and list the third-party pages it cites. Check that each one lets Claude-SearchBot in, then earn a place on the open ones: an accurate listing, a product for the reviewer, data the author can use. A famous site that shuts Anthropic out is a weaker target than a niche one that lets it read. Doing this for dozens of questions is what AI citation tracking is for.
Ask the same questions every week
Claude writes each answer fresh and searches again each time, so a single check proves little. Keep a fixed list of questions, ask them on a schedule, and count how often you are named or cited over several weeks, not in one reply. Before you trust a rise or a fall, read how often to measure and why answers move.
See what AI can read on your website
We run 18 checks on what helps AI assistants find, read and cite your site, and tell you what to fix first.
Ready as soon as you sign up, included in the free trial.
No account? Free GEO audit tool: the same 18 checks, without signing up.
Claude readiness checklist, scored out of 100
Ten minutes with your robots.txt, your CDN and a few chats with Claude. Add up the points of every line you pass.
- 0115
Claude-SearchBot may crawl every subdomain you want found
Open /robots.txt on each one, or run the free crawler checker.
- 0210
Claude-User may fetch your pages
Same file. Without it, Claude cannot open your page for a buyer’s question.
- 0310
Your firewall or CDN lets Anthropic’s published IP ranges through
Check the bot rules in your CDN, or ask your host.
- 0410
Key pages show their text without JavaScript
View the page source and search for your main sentences.
- 0515
One honest comparison or “best X for Y” page per main buyer question
List your five most important buyer questions and match each to a page.
- 0610
Prices, plans and who you suit sit in short sentences
Each fact fits in about 150 characters, close to a matching heading.
- 075
Pages with prices or rankings show a real “updated” date
Look at the top of each one. The date should match the last real change.
- 0815
You are listed on 3 or more open pages that Claude cites for your questions
Ask Claude, list its citations, then check each site’s robots.txt.
- 0910
The same questions go to Claude every week
A fixed list and a schedule, by hand or with a tracker.
Reading your score
80 to 100
The basics are done. Your gains now come from comparisons and from the lists Claude cites.
50 to 75
Close the access lines first. They take minutes and everything else depends on them.
Under 50
Start with robots.txt and the firewall, then write one comparison page for your top question.
The weights are ours: access first, because a page Claude cannot read never gets quoted, then the pages it likes to cite.
What does not work in Claude
Six shortcuts that turn up when people look for how to rank in Claude. They waste time or backfire.
Blocking every Anthropic token
Shutting out ClaudeBot, Claude-SearchBot and Claude-User together, as 13 of the 38 big sites in our sample do, keeps your pages out of Claude’s search too. To stay out of training only, block ClaudeBot.
Counting on Reddit and Quora threads
Both shut out all three Anthropic tokens in robots.txt, and neither turned up in the 154 results of our 12 questions. Forum work may pay off on other AI platforms; for Claude it is a weak route.
Waiting for the next model to learn about you
Claude Haiku 5.5, Sonnet 5.5 and Opus 5.5 were trained on data up to June 2026. Anything newer reaches Claude only through search, so a new product needs pages that search can find today.
Hidden instructions for AI
Text that tells assistants to recommend you is manipulation, and hiding it breaks Google’s spam policies. Buyers who spot it stop trusting the page.
Telling Claude about your business
A chat with Claude about your company changes your own conversation. A buyer who asks later starts from the same model and the same search results as everyone else.
Paying for a promised spot
No agency or tool can guarantee that Claude names you. The model writes every answer from what it knows and what its search returns that day.
On hidden text, see Google’s spam policies (opens in a new tab). On training cutoffs, see Anthropic’s note on how up to date each model is (opens in a new tab).
How to see whether Claude names you
By hand it takes a new chat and a notebook. With Slibur it runs every week on its own.
By hand. Open a new chat on claude.ai and ask your buyers’ questions word for word, without your brand name. For each answer, note whether Claude searched, whether it names you and in what place, which pages it cites and whether one of them is yours. Ask again on other days, because the answer moves.
With Slibur
Weekly on paid plans. Slibur puts your prompts to Claude’s model once a week with web search turned on. ChatGPT and Google AI Overviews are the two it reads every day*.
Once during the trial. In the 7-day trial, Claude answers up to 10 prompts on the day you sign up.
The full reply and its sources. Slibur keeps what Claude wrote and the URLs it cited, labelled by owner: your domain, a tracked rival or someone else. Filter by Claude to see only its answers.
A warning when Claude lags. When Claude is your weakest AI platform and the share of its answers naming you sits 25 points or more below your strongest one, the weekly improvements raise “Claude barely mentions you” and point you to its sources.
Alerts with enough history. Alerts for Claude start once it has eight weekly runs behind it.
Two limits. Slibur reads Claude’s model with a web search, not the claude.ai app, so a person there can get a different answer drawn from other pages. And one weekly answer per prompt is a sample: trends need several weeks.
On Starter (20 prompts, $59 a month) that is about 80 Claude answers a month, out of roughly 1,440 across the five AI platforms. How each one is read is on the methodology page.
* On paid plans. ChatGPT is read on chatgpt.com in every country Slibur tracks except Venezuela and Puerto Rico, where it is asked weekly through its API.
Free checks before you start
No sign-up needed. The crawler check takes a few seconds.
- AI crawler checkerType your domain and get a verdict for Claude-SearchBot, Claude-User and ClaudeBot, with the exact rule behind each one.Use it free
- Free GEO audit tool18 tests on your site, from robots.txt and the sitemap to the text a crawler reads when scripts do not run (step 4).Use it free
- GEO audit checklistA wider audit for every AI platform, scored out of 100, with the source behind each check.Use it free
See what Claude says about you, free for 7 days
Your first Claude answers arrive the day you sign up, together with those of ChatGPT, Google AI Overviews, Perplexity and Grok, so you know which pages each one cites before you touch your site.
7 days free, no card. Sign up with your email or Google. When the trial ends, tracking stops and nothing is charged.
Up to 10 prompts. We suggest prompts and competitors from your website, and you edit the list before the first run.
All five AI platforms on day one. ChatGPT, Google AI Overviews, Claude, Perplexity and Grok answer your prompts as soon as your email is confirmed.
ChatGPT and Google AI again every day while you keep visiting your dashboard. Claude, Perplexity and Grok are asked once, at sign-up.
One “Run now” and the 18-point site audit. After the trial, our free GEO audit tool still audits any site, without an account.
On paid plans only: team invites and the weekly email summary with alerts.
* On paid plans. ChatGPT is read on chatgpt.com in every country Slibur tracks except Venezuela and Puerto Rico, where it is asked weekly through its API.
Slibur Starter
Track 20 prompts on 5 AI platforms
$59/ month (USD)
Or $602 billed yearly · save $106
Start your 7-day free trial7 days free. No card required.
- Up to 20 prompts, about 1,440 answers a month
- ChatGPT, Google AI Overviews, Claude, Perplexity, Grok
- ChatGPT and Google AI every day*, the other three every week
- Competitor ranking (up to 25 rivals) and every cited URL
- Weekly improvements, plus an email summary with alerts
- 18-point AI-readiness site audit
- Up to 2 team members
Need more prompts? Growth is $129 a month for 75 and Scale $299 for 250. Compare all plans
Sources for this guide
What this guide says about Claude comes from Anthropic’s articles on its crawlers (opens in a new tab), on its hosted browsers (opens in a new tab), on web search (opens in a new tab) and on training data (opens in a new tab), from its web search tool documentation (opens in a new tab) and from its crawler IP list (opens in a new tab). The ChatGPT and Perplexity columns come from OpenAI’s (opens in a new tab) and Perplexity’s (opens in a new tab) crawler pages, and the matching rules from RFC 9309 (opens in a new tab). All were read on , the same day as the robots.txt files and the 12 answers. Anthropic does not publish how Claude ranks one source above another, so the steps on how to rank in Claude are practical advice built on what it does publish.
Questions about ranking in Claude
About Slibur itself? Read the full FAQs or see every plan and price.
How long before Claude starts citing my pages?
Anthropic gives no timeline. Answers built from search can change as soon as Claude’s search picks up your updated page, while what the model knows on its own changes only with a new model, and the current ones stop at June 2026. Compare the same questions week by week for at least a month before you judge a change.
Which search engine does Claude use?
Anthropic’s help article on web search, read on November 9, 2026, does not name the provider of its text results. It does say image results are powered by Bing. Its API documentation says Anthropic runs the searches and passes the results to Claude. The practical answer: allow Claude-SearchBot and be easy to find for the searches your buyers trigger.
If I block ClaudeBot, will Claude stop citing my site?
No. Anthropic says ClaudeBot collects content for training, while Claude-SearchBot indexes pages for search and Claude-User fetches them for users. In our 12-question sample, 5 of the cited sites block ClaudeBot and were cited all the same.
How do I get mentioned by Claude?
Be on the pages Claude reads for your buyers’ questions and give it a reason to pick you: a comparison that says who you suit, current prices, and listings on the review sites it cites. Getting recommended by Claude follows from the same work, since the brands it suggests are the ones its sources back.
Is SEO for Claude different from SEO for Google?
It builds on it. Claude’s searches read like ordinary searches, and pages that rank for them get read. Three things differ: Anthropic’s own tokens decide access, Claude quotes short passages instead of listing ten links, and the page that wins is often a comparison written by someone else. A page can rank on Google and stay invisible to Claude if robots.txt shuts out Claude-SearchBot.
Does Claude use Reddit?
Reddit’s robots.txt, as it was served to us on November 9, 2026, disallows every crawler, Anthropic’s three included, and no Reddit page came up in the 154 search results of our 12 buyer questions. That can change, but today Reddit is a poor route into Claude’s answers.
Can a small brand rank in Claude?
Our sample says yes. The 60 pages Claude cited came from 58 different sites, most of them niche review blogs and vendors, and only five belonged to the 38 big US sites we checked. What they shared was a list or comparison title that matched Claude’s search, not the size of the site behind it.
Will pages I write with Claude rank on Google?
That is a different question from ranking inside Claude’s answers, and Google judges the page, not the tool. Its spam policies list using generative AI to produce many pages without adding value for users as scaled content abuse. A page drafted with Claude that brings facts, prices or data of its own is not what that policy describes.
Can anyone guarantee a ranking in Claude?
No. Claude writes each answer and picks its sources question by question. What you control is whether it can read your pages, whether they are worth quoting and whether you sit on the pages it already cites. Then you measure.
Can I check what Claude says about my brand for free?
By hand, yes: a new chat, your buyers’ questions and a note of every page Claude cites. Slibur does it for you, starting with a 7-day trial with no card: up to 10 prompts, put to Claude at sign-up next to ChatGPT, Google AI Overviews, Perplexity and Grok. Once the trial ends, paid plans start at $59 a month.
See where you rank in Claude today
Learning how to rank in Claude starts with knowing where you stand. Give Slibur your domain and the questions your buyers type, and on day one you get Claude’s answer, the pages it cites and how four other AI platforms compare.
7 days free. No card required.
