Question

Where to put llms.txt

Updated June 7, 2026

Publish the file at your root path so assistants and crawlers can find it reliably. The expected URL is simple: https://yourdomain.com/llms.txt. If the file exists only inside docs, assets, or a framework route, many tools and crawlers will miss it.

Correct URL format

Your file should be available at https://yourdomain.com/llms.txt. Avoid deep paths like /docs/llms.txt unless you also maintain a root file.

Canonical deployment pattern by stack

• Static hosting: commit llms.txt to the public root directory.
• Nginx/Caddy: serve as a plain text/markdown file from site root.
• Framework apps: place in public assets so build/deploy preserves root path.

Stack-specific examples

• Next.js: put the file in /public/llms.txt, then deploy and check the root URL.
• Vite/Astro: put it in the public directory, not inside source routes.
• WordPress: upload to the web root or use a plugin only if it serves the exact root path.
• Cloudflare Pages: include it in the output directory, then verify that no Worker route rewrites it.

Host consistency example

Pick one canonical host and keep llms.txt behavior identical there. A safe default is: www -> apex redirect, then serve /llms.txt from apex with 200.

# Check apex and www behavior together
curl -I https://yourdomain.com/llms.txt
curl -I https://www.yourdomain.com/llms.txt

Deployment checklist

• File exists at root path.
• Returns HTTP 200.
• Listed in robots.txt as a reference page.
• Contains only canonical URLs.

Common misdeploys to avoid

• Publishing only at /docs/llms.txt without a root file.
• Serving different content between apex and www.
• Returning 403 from WAF challenge rules for crawler user agents.
• Serving stale content from a CDN cache after the file was updated.
• Listing URLs that redirect several times or resolve to non-canonical hosts.

Should robots.txt mention llms.txt?

It is not required, but adding a reference can help humans and automated tools discover your policy files together. Keep robots permissions and llms.txt guidance separate: robots controls access; llms.txt maps the pages you want understood.

User-agent: *
Allow: /

Sitemap: https://yourdomain.com/sitemap.xml
# Optional reference for humans and tooling:
# llms.txt: https://yourdomain.com/llms.txt

Fast verification commands

curl -I https://yourdomain.com/llms.txt
curl -s https://yourdomain.com/llms.txt | sed -n '1,80p'
curl -I https://www.yourdomain.com/llms.txt

If apex and www return different files, unify behavior before further SEO/GEO changes. Then confirm crawler hits with OAI-SearchBot log checks.

Next step: test llms.txt quality and set update cadence.

Validate your file