PerplexityBot
又称: Perplexity crawler, Perplexity-User
一句话解释
PerplexityBot is Perplexity's crawler, used to index pages for the search product behind its answers. Perplexity-User is the separate agent that fetches a page live when a user's question requires it. Both can be allowed or blocked by name in robots.txt.
Why this one is worth allowing
Perplexity displays its sources prominently beside the answer rather than burying them. For a brand that means a citation here is more visible than the equivalent citation elsewhere — closer to a link in a results page than a footnote.
That makes it a relatively high-return engine to be indexed by, and a relatively costly one to be absent from.
Its source preferences differ
Engines do not agree with each other. Analyses of AI citations find Perplexity leaning toward community platforms and B2B review sites on commercial questions, where other engines lean toward reference sites or editorial publications.
The practical implication is that a Perplexity visibility problem often has a different cause than a ChatGPT one, and a blended visibility score would hide that entirely.
The usual robots.txt caveats
Two agents, both needing their own rule. Named groups replace the wildcard group rather than extending it, so repeat your disallows. And verify at the network layer too — a permissive robots.txt sitting behind a CDN bot rule that returns 403 to non-browser agents is the most common way to be accidentally invisible.
常见问题
- Should I allow PerplexityBot?
- For most brands, yes. Perplexity is unusually citation-forward — sources are displayed prominently next to the answer rather than tucked away — so being in its index is worth more visible attribution per answer than on some other engines.
- How do I block PerplexityBot?
- A named group in robots.txt with User-agent: PerplexityBot and Disallow: /. As with every named group, repeat any disallow rules from your wildcard group, and note that Perplexity-User is a separate agent requiring its own rule.
- Does Perplexity cite different sources than ChatGPT?
- Substantially, yes. Published analyses put the overlap between engines' cited sources at roughly a tenth, and Perplexity leans noticeably toward community platforms and B2B review sites on commercial questions. Visibility has to be measured per engine.
相关术语
- AI crawler
An AI crawler is an automated bot that fetches web pages on behalf of an AI company, either to build a training corpus, to maintain a search index, or to retrieve a page live while answering a user's question. Each is identified by a named user agent you can allow or block in robots.txt.
- GPTBot
GPTBot is OpenAI's crawler for gathering publicly available web content used in training its models. It is one of three OpenAI agents: GPTBot for training, OAI-SearchBot for the search index behind ChatGPT's answers, and ChatGPT-User for fetching a page live when a user's question requires it.
- ClaudeBot
ClaudeBot is Anthropic's web crawler, used to gather publicly available content for Claude. A separate agent, Claude-User, fetches a specific page live when a user's request requires it. Both are identified by name in robots.txt and can be allowed or disallowed independently.
- robots.txt
robots.txt is a plain-text file at your site root that tells automated crawlers which paths they may fetch. Each block names a user agent and lists allow and disallow rules. It is advisory rather than enforced, and it controls crawling rather than indexing or access.
相关指南
- Generative Engine Optimization (GEO): the complete guide
GEO is the practice of getting your brand named and cited inside AI answers. What it is, how it differs from SEO, and the levers that measurably work.