AI Training Crawlers

MistralAI-Training user agent

Operated by Mistral AI

AI trainingOfficial docs Your choice

MistralAI-Training is Mistral's crawler that collects web content to build training datasets for Mistral models.

The MistralAI-Training bot at a glance

  • MistralAI-Training is operated by Mistral AI and collects public web pages that may be used to train AI models.
  • The robots.txt token for MistralAI-Training is MistralAI-Training.
  • Blocking an AI training crawler has no effect on Google or Bing rankings.
  • MistralAI-Training robots.txt support: Yes.
  • Recommendation for Generative Engine Optimization (GEO): Your choice.

What is MistralAI-Training?

MistralAI-Training gathers public pages that may be used to train Mistral’s language models.

It is separate from MistralAI-Index and MistralAI-User, so training can be blocked on its own.

Should you block MistralAI-Training?

Blocking MistralAI-Training is a business decision: it protects your content but can lower how well AI products know your brand.

Effect on search rankingsBlocking an AI training crawler has no effect on Google or Bing rankings.
Effect on AI visibilityBlocking keeps your future pages out of that company's training data, so its models may know less about your brand over time. It does not remove you from that company's AI search, which uses a separate bot.
Follows robots.txtYes
RecommendationYour choice

How to block or allow MistralAI-Training

Put the robots.txt rule in the file at the root of every domain and subdomain, for example https://example.com/robots.txt. The token MistralAI-Training is matched case-insensitively.

robots.txt: block

# Block MistralAI-Training
User-agent: MistralAI-Training
Disallow: /

robots.txt: allow

# Allow MistralAI-Training
User-agent: MistralAI-Training
Allow: /

robots.txt: partial

# Keep MistralAI-Training out of private areas only
User-agent: MistralAI-Training
Disallow: /wp-admin/
Disallow: /cart/
Disallow: /checkout/
Disallow: /my-account/
Allow: /

Nginx

# Nginx: return 403 to MistralAI-Training
if ($http_user_agent ~* "(MistralAI\-Training)") {
    return 403;
}

Apache

# Apache .htaccess: return 403 to MistralAI-Training
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} (MistralAI\-Training) [NC]
RewriteRule .* - [F,L]

Cloudflare WAF

# Cloudflare > Security > WAF > Custom rule, action: Block
(http.user_agent contains "MistralAI-Training")

How to verify real MistralAI-Training traffic

Mistral AI publishes no IP list or reverse DNS pattern for MistralAI-Training, so the user agent cannot be verified. Anyone can send the same string. Rate-limit unusual volume instead of trusting the name.

How to find MistralAI-Training in your server logs

Search the web server access log for the token. The command below counts MistralAI-Training requests per URL on Nginx or Apache.

grep -i "MistralAI-Training" access.log | awk '{print $7}' | sort | uniq -c | sort -rn | head -20

For a full report per AI bot, upload the log to the AI Log File Analyzer, a free tool by Justin Hà.

MistralAI-Training user agent string

The full user agent Mistral AI publishes for MistralAI-Training. Version numbers such as Chrome/W.X.Y.Z change over time, so match the token, not the whole string.

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; MistralAI-Training/1.0; +https://docs.mistral.ai/robots)

MistralAI-Training FAQ

MistralAI-Training identifies itself with the robots.txt token "MistralAI-Training". A typical request header reads: Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; MistralAI-Training/1.0; +https://docs.mistral.ai/robots)

Yes. Mistral AI states that MistralAI-Training follows robots.txt rules written for the "MistralAI-Training" token.

Blocking an AI training crawler has no effect on Google or Bing rankings. Blocking keeps your future pages out of that company's training data, so its models may know less about your brand over time. It does not remove you from that company's AI search, which uses a separate bot.

Add two lines to the robots.txt file at the root of each domain and subdomain: "User-agent: MistralAI-Training" followed by "Disallow: /". The token is case-insensitive. Changes can take up to 24 hours to be picked up.

Not reliably. Mistral AI publishes no IP list or reverse DNS pattern for MistralAI-Training, so anyone can send the same user agent. Rate-limit unusual volume instead of trusting the name.

Search your server access log for "MistralAI-Training", for example with: grep -i "MistralAI-Training" access.log. The AI Log File Analyzer by Justin Hà counts visits per AI bot from an uploaded log file.

Sources: Mistral AI crawler documentation. Last reviewed by Justin Hà.

See which of these bots actually reach your website

The AI Log File Analyzer by Justin Hà counts visits from GPTBot, ClaudeBot, PerplexityBot and other AI crawlers in your server log, for free. The GEO Technical Package sets up robots.txt, llms.txt and crawler access for AI search.

Zalo logo Chat Zalo WhatsApp logo Chat WhatsApp