PerplexityBot is the web crawler operated by Perplexity AI, identified by the user-agent string PerplexityBot. It fetches and indexes web content to power Perplexity's AI search engine. Unlike some AI crawlers that only collect training data, PerplexityBot serves a dual role -- it both builds Perplexity's index and retrieves content in real time when users perform searches.
Why It Matters
Perplexity AI is the fastest-growing AI search engine, and PerplexityBot is the gateway to appearing in its results. For website owners focused on AI visibility, understanding this crawler is essential.
Why PerplexityBot matters:
- Direct search impact. Unlike GPTBot (which only collects training data), blocking PerplexityBot immediately removes your site from Perplexity's search results. There is no separate "search bot" -- PerplexityBot handles everything.
- Source-first citations. Perplexity prominently displays source links alongside its answers. This makes it one of the strongest AI platforms for driving referral traffic back to your website. Every cited source gets a clickable link.
- Rapid indexing. Perplexity can discover, index, and cite new content within hours of publication -- significantly faster than traditional search engines and most other AI platforms.
- Growing user base. Perplexity's audience is expanding rapidly, particularly among researchers, professionals, and users who want AI-generated answers backed by verifiable sources.
- YouTube integration. Perplexity cites YouTube content in 16.1% of responses, making it unique among AI search platforms in how heavily it uses video content.
For most businesses pursuing AI visibility, allowing PerplexityBot is a high-value decision with clear traffic benefits.
How It Works
PerplexityBot operates differently from pure training crawlers because it performs both indexing and real-time retrieval.
- User-agent string. PerplexityBot identifies itself with the user-agent
PerplexityBot. Target this in your robots.txt to control access. - Indexing mode. PerplexityBot periodically crawls the web to build and update Perplexity's search index. This is similar to how Googlebot maintains Google's index.
- Real-time retrieval. When a user asks Perplexity a question, the system may dispatch PerplexityBot to fetch fresh content from relevant pages. This means your latest content can be cited immediately -- even before it is formally "indexed."
- Citation behavior. When Perplexity uses your content in an answer, it includes a numbered citation with a direct link to your page. Users see these citations inline and can click through to the original source.
- Robots.txt compliance. PerplexityBot respects robots.txt directives. Block it with
User-agent: PerplexityBotandDisallow: /. Allow specific sections with targeted Allow rules.
PerplexityBot vs. other AI crawlers
| Feature | PerplexityBot | GPTBot | OAI-SearchBot | |---|---|---|---| | Purpose | Index + real-time search | Training only | Real-time search only | | Blocking removes from search? | Yes | No | Yes | | Source citations | Always, with links | N/A | Sometimes | | Crawl speed | Hours | Days to weeks | Real-time per query |
Manage all your AI crawler access in one place with our robots.txt for AI crawlers guide.
Key Points
- PerplexityBot (user-agent:
PerplexityBot) is Perplexity AI's dual-purpose crawler -- it indexes content and retrieves it in real time for search - Blocking PerplexityBot directly removes your site from Perplexity search results
- Perplexity provides prominent source citations with clickable links, driving meaningful referral traffic
- New content can be discovered and cited within hours of publication
- Control access through robots.txt directives
- Check your current Perplexity visibility with our Perplexity visibility guide
Frequently Asked Questions
Should I allow PerplexityBot to crawl my site?
If AI search visibility matters to your business, yes. PerplexityBot powers both indexing and real-time search on Perplexity AI. Blocking it removes your site from Perplexity results entirely. Since Perplexity provides source citations with direct links, allowing PerplexityBot can generate measurable referral traffic. Configure it in your robots.txt for AI crawlers.
Does PerplexityBot drive traffic to my website?
Yes. Perplexity is one of the best AI platforms for referral traffic because it always includes source citations with clickable links. When your content is used to answer a question, users see your site listed as a source and can click through directly. Track this traffic in GA4 by filtering referrals from perplexity.ai.
How does PerplexityBot differ from GPTBot?
GPTBot is a training-only crawler -- it collects data for future GPT models but does not power real-time search. PerplexityBot handles both indexing and live retrieval. Blocking GPTBot does not affect ChatGPT search results (that is OAI-SearchBot's job), but blocking PerplexityBot directly removes you from Perplexity's search results.
Are you visible on Perplexity AI?
Check if PerplexityBot can access your site and whether Perplexity is citing your content.
Free scan -- no signup required