Question
Does llms.txt help SEO?
It can help indirectly. llms.txt improves source clarity for AI answer systems, but it does not replace core SEO fundamentals.
What llms.txt can improve
- Source-page discoverability for AI answer grounding.
- Consistency between your key pages and site intent.
- Editorial control over which pages are highlighted.
What llms.txt cannot replace
- Useful original content.
- Technical crawlability and indexing.
- Strong page experience and clear information architecture.
Where teams usually see real impact
- Documentation sites: clearer discovery of setup and API reference pages.
- B2B product sites: better citation likelihood for pricing/security/policy pages.
- Q&A pages: stronger mapping between search intent and source pages.
Concrete rollout examples
- Docs portal: list quickstart, API reference, changelog, and migration pages in
llms.txt. Keep old version pages out unless still canonical. - Local-service site: include service pages, pricing, booking policy, and coverage area pages. Exclude thin tag pages.
- B2B lead-gen site: include solution pages, integration docs, case studies, and security/compliance pages.
What to measure in the first 14 days
- Whether crawler hits move from only
/robots.txtto content URLs. - Whether AI-referral landing pages match the pages listed in
llms.txt. - Whether high-intent pages show better click consistency in Search Console.
Common failure pattern
Teams often publish llms.txt but keep linking non-canonical or weak pages. The result is noisy signals, not stronger visibility. Curating fewer high-value URLs is usually better than listing everything.
Two real-world before/after patterns
- Before: a SaaS site listed 120 mixed URLs including old release pages. After: reduced to 24 canonical docs/pricing/security pages, then saw crawler hits shift from entry files to docs URLs within one week.
- Before: a local-service site listed homepage only. After: added service, area, pricing, booking, and guarantee pages; AI-answer referrals started landing on intent pages instead of only home.
30-minute rollout checklist
- Publish a concise file at
/llms.txt. - List only high-value canonical URLs, not every page.
- Validate HTTP 200 on each linked URL.
- Track crawler access and referral changes for 2-4 weeks.
curl -I https://yourdomain.com/llms.txt
curl -I https://yourdomain.com/robots.txt
curl -I https://yourdomain.com/sitemap.xml
# check whether crawler hits move to intent pages
grep -Ei 'OAI-SearchBot|PerplexityBot' /var/lib/caddy/logs/llmsfile-access.log \
| awk '{print $7}' | sort | uniq -c | sort -nr | head -n 20Related pages: where to publish llms.txt, how to track AI search referrals, and how to verify OAI-SearchBot access.