AI Search Crawlers

LinerBot user agent

Operated by Liner

AI searchAI assistantReported Allow

LinerBot is a crawler reported for Liner, an AI search and research assistant.

The LinerBot bot at a glance

  • LinerBot is operated by Liner and indexes web pages so an AI search product can show and cite them in answers, and fetches a page when a person asks an AI assistant about it or shares its link.
  • The robots.txt token for LinerBot is LinerBot.
  • Blocking an AI search crawler has no effect on Google or Bing rankings.
  • LinerBot robots.txt support: Not documented.
  • Recommendation for Generative Engine Optimization (GEO): Allow.

What is LinerBot?

LinerBot reads pages that Liner’s AI search can cite in answers.

Liner has not published a crawler page.

Should you block LinerBot?

Allow LinerBot unless it causes a measurable load problem.

Effect on search rankingsBlocking an AI search crawler has no effect on Google or Bing rankings.
Effect on AI visibilityBlocking removes your pages from that AI search index, so its answers can no longer cite or link to your website.
Follows robots.txtNot documented
RecommendationAllow

How to block or allow LinerBot

Put the robots.txt rule in the file at the root of every domain and subdomain, for example https://example.com/robots.txt. The token LinerBot is matched case-insensitively. robots.txt alone may not stop LinerBot, so add the Nginx, Apache or Cloudflare rule as well.

robots.txt: block

# Block LinerBot
User-agent: LinerBot
Disallow: /

robots.txt: allow

# Allow LinerBot
User-agent: LinerBot
Allow: /

robots.txt: partial

# Keep LinerBot out of private areas only
User-agent: LinerBot
Disallow: /wp-admin/
Disallow: /cart/
Disallow: /checkout/
Disallow: /my-account/
Allow: /

Nginx

# Nginx: return 403 to LinerBot
if ($http_user_agent ~* "(LinerBot)") {
    return 403;
}

Apache

# Apache .htaccess: return 403 to LinerBot
RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} (LinerBot) [NC]
RewriteRule .* - [F,L]

Cloudflare WAF

# Cloudflare > Security > WAF > Custom rule, action: Block
(http.user_agent contains "LinerBot")

How to verify real LinerBot traffic

Liner publishes no IP list or reverse DNS pattern for LinerBot, so the user agent cannot be verified. Anyone can send the same string. Rate-limit unusual volume instead of trusting the name.

How to find LinerBot in your server logs

Search the web server access log for the token. The command below counts LinerBot requests per URL on Nginx or Apache.

grep -i "LinerBot" access.log | awk '{print $7}' | sort | uniq -c | sort -rn | head -20

For a full report per AI bot, upload the log to the AI Log File Analyzer, a free tool by Justin Hà.

LinerBot FAQ

LinerBot uses the robots.txt token "LinerBot". Liner does not publish a full user-agent string, so match the token inside the User-Agent header.

Liner has not documented how LinerBot treats robots.txt. Add a robots.txt rule and back it with a server rule if the traffic matters to you.

Blocking an AI search crawler has no effect on Google or Bing rankings. Blocking removes your pages from that AI search index, so its answers can no longer cite or link to your website.

Add two lines to the robots.txt file at the root of each domain and subdomain: "User-agent: LinerBot" followed by "Disallow: /". The token is case-insensitive. Changes can take up to 24 hours to be picked up.

Not reliably. Liner publishes no IP list or reverse DNS pattern for LinerBot, so anyone can send the same user agent. Rate-limit unusual volume instead of trusting the name.

Search your server access log for "LinerBot", for example with: grep -i "LinerBot" access.log. The AI Log File Analyzer by Justin Hà counts visits per AI bot from an uploaded log file.

Sources: Public bot registry. Last reviewed by Justin Hà.

See which of these bots actually reach your website

The AI Log File Analyzer by Justin Hà counts visits from GPTBot, ClaudeBot, PerplexityBot and other AI crawlers in your server log, for free. The GEO Technical Package sets up robots.txt, llms.txt and crawler access for AI search.

Zalo logo Chat Zalo WhatsApp logo Chat WhatsApp