Bots by company
01.AI bots and crawlers
01.AI builds the Yi language models in China.
01.AI bots at a glance
- 01.AI runs 1 bot listed in this directory.
- They belong to: AI Training Crawlers.
- Each 01.AI bot has its own robots.txt token, so each purpose can be allowed or blocked separately.
- Website: www.01.ai
Every 01.AI bot
| Bot | Group | Purpose | robots.txt | For GEO |
|---|---|---|---|---|
| YiBot | AI Training Crawlers | AI trainingReported | Not documented | Your choice |
Block or allow all 01.AI bots
The rules below name every 01.AI token in this directory. To block training but keep AI search, delete the search and assistant tokens from the block rule.
robots.txt: block
# Block all 01.AI bots
User-agent: YiBot
Disallow: /robots.txt: allow
# Allow all 01.AI bots
User-agent: YiBot
Allow: /robots.txt: partial
# Keep all 01.AI bots out of private areas only
User-agent: YiBot
Disallow: /wp-admin/
Disallow: /cart/
Disallow: /checkout/
Disallow: /my-account/
Allow: /Nginx
# Nginx: return 403 to all 01.AI bots
if ($http_user_agent ~* "(YiBot)") {
return 403;
}Apache
# Apache .htaccess: return 403 to all 01.AI bots
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} (YiBot) [NC]
RewriteRule .* - [F,L]Cloudflare WAF
# Cloudflare > Security > WAF > Custom rule, action: Block
(http.user_agent contains "YiBot")See which of these bots actually reach your website
The AI Log File Analyzer by Justin Hà counts visits from GPTBot, ClaudeBot, PerplexityBot and other AI crawlers in your server log, for free. The GEO Technical Package sets up robots.txt, llms.txt and crawler access for AI search.
