Question

How to check Claude-SearchBot visits

Do not mix tokens. Claude-SearchBot, ClaudeBot, and Claude-User can represent different behaviors in logs.

CLI quick checks

grep -i "Claude-SearchBot" /var/log/nginx/access.log | tail -n 50
grep -i "Claude-SearchBot" /var/log/nginx/access.log | awk '{print $9}' | sort | uniq -c

Practical interpretation

• If only robots.txt is hit, policy may still be under evaluation.
• If key guides return 200, discovery is likely working.
• If hits are absent, check whether wildcard rules or CDN settings block crawl access.

Classify crawler lines