Question
Should I block Amazonbot?
Amazonbot crawls the web for Alexa answers and Amazon's AI services and honors robots.txt. Block it if you do not want your content used by Amazon AI/Alexa or it adds crawl load; keep it allowed if you want to appear in Alexa-style answers.
When blocking makes sense
- You do not target Alexa/Amazon answer surfaces.
- You want to limit AI training/answer use of your content.
- Crawl volume is costing you origin resources.
Block it
User-agent: Amazonbot
Disallow: /
Confirm it is honored
grep -i 'Amazonbot' /var/log/your-access.log | awk '{print $7}' | sort | uniq -c | sort -nr | head
Related pages: should I block Bytespider, what is CCBot, AI crawler user agents.