ExaSearchBot user agent
Operated by Exa
ExaSearchBot is Exa's crawler, reported as indexing public pages so they can be found and cited through Exa's AI search API.
The ExaSearchBot bot at a glance
- ExaSearchBot is operated by Exa and indexes web pages so an AI search product can show and cite them in answers.
- The robots.txt token for ExaSearchBot is ExaSearchBot.
- Blocking an AI search crawler has no effect on Google or Bing rankings.
- ExaSearchBot robots.txt support: Not documented.
- Recommendation for Generative Engine Optimization (GEO): Allow.
What is ExaSearchBot?
Exa provides search results to AI applications and agents, so its index reaches many AI tools at once.
Exa has not published a crawler page for this name.
Should you block ExaSearchBot?
Allow ExaSearchBot unless it causes a measurable load problem.
| Effect on search rankings | Blocking an AI search crawler has no effect on Google or Bing rankings. |
|---|---|
| Effect on AI visibility | Blocking removes your pages from that AI search index, so its answers can no longer cite or link to your website. |
| Follows robots.txt | Not documented |
| Recommendation | Allow |
How to block or allow ExaSearchBot
Put the robots.txt rule in the file at the root of every domain and subdomain, for example https://example.com/robots.txt. The token ExaSearchBot is matched case-insensitively. robots.txt alone may not stop ExaSearchBot, so add the Nginx, Apache or Cloudflare rule as well.
robots.txt: block
# Block ExaSearchBot
User-agent: ExaSearchBot
Disallow: /robots.txt: allow
# Allow ExaSearchBot
User-agent: ExaSearchBot
Allow: /robots.txt: partial
# Keep ExaSearchBot out of private areas only
User-agent: ExaSearchBot
Disallow: /wp-admin/
Disallow: /cart/
Disallow: /checkout/
Disallow: /my-account/
Allow: /Nginx
# Nginx: return 403 to ExaSearchBot
if ($http_user_agent ~* "(ExaSearchBot)") {
return 403;
}Apache
# Apache .htaccess: return 403 to ExaSearchBot
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} (ExaSearchBot) [NC]
RewriteRule .* - [F,L]Cloudflare WAF
# Cloudflare > Security > WAF > Custom rule, action: Block
(http.user_agent contains "ExaSearchBot")How to verify real ExaSearchBot traffic
Exa publishes no IP list or reverse DNS pattern for ExaSearchBot, so the user agent cannot be verified. Anyone can send the same string. Rate-limit unusual volume instead of trusting the name.
How to find ExaSearchBot in your server logs
Search the web server access log for the token. The command below counts ExaSearchBot requests per URL on Nginx or Apache.
grep -i "ExaSearchBot" access.log | awk '{print $7}' | sort | uniq -c | sort -rn | head -20For a full report per AI bot, upload the log to the AI Log File Analyzer, a free tool by Justin Hà.
ExaSearchBot FAQ
ExaSearchBot uses the robots.txt token "ExaSearchBot". Exa does not publish a full user-agent string, so match the token inside the User-Agent header.
Exa has not documented how ExaSearchBot treats robots.txt. Add a robots.txt rule and back it with a server rule if the traffic matters to you.
Blocking an AI search crawler has no effect on Google or Bing rankings. Blocking removes your pages from that AI search index, so its answers can no longer cite or link to your website.
Add two lines to the robots.txt file at the root of each domain and subdomain: "User-agent: ExaSearchBot" followed by "Disallow: /". The token is case-insensitive. Changes can take up to 24 hours to be picked up.
Not reliably. Exa publishes no IP list or reverse DNS pattern for ExaSearchBot, so anyone can send the same user agent. Rate-limit unusual volume instead of trusting the name.
Search your server access log for "ExaSearchBot", for example with: grep -i "ExaSearchBot" access.log. The AI Log File Analyzer by Justin Hà counts visits per AI bot from an uploaded log file.
Sources: Public bot registry. Last reviewed by Justin Hà.
See which of these bots actually reach your website
The AI Log File Analyzer by Justin Hà counts visits from GPTBot, ClaudeBot, PerplexityBot and other AI crawlers in your server log, for free. The GEO Technical Package sets up robots.txt, llms.txt and crawler access for AI search.
