Question
How to check ClaudeBot visits
Use the exact token ClaudeBot. Do not merge it with Claude-SearchBot in one count.
Quick commands
grep -i "ClaudeBot" /var/log/nginx/access.log | tail -n 80
grep -i "ClaudeBot" /var/log/nginx/access.log | awk '{print $7}' | sort | uniq -c | sort -nr | head
grep -i "ClaudeBot" /var/log/nginx/access.log | awk '{print $9}' | sort | uniq -c | sort -nr
What to look for
- High
200ratio on core pages means crawler access is clean. - Frequent
403suggests policy or firewall blocking. - Mostly
/robots.txthits means discovery started but deep crawl is still light.