Case study

LLMs File traffic baseline and checkpoint (May 2026)

This page documents real operational numbers from our own server logs. We publish both the starting baseline and the latest checkpoint so decisions stay measurable.

Two checkpoints from the same growth cycle

The 404 jump on 2026-05-28 was mostly scanner noise on fake paths. We filtered that noise before selecting content priorities.

Important correction: server logs are not the same as real visitors

After adding browser analytics, we stopped treating raw server unique IP as the growth KPI. Caddy logs include health checks, IndexNow verification, scanner probes, AI crawlers, and clients that fetch HTML without running JavaScript. Umami-style browser sessions are stricter because the page must load the analytics script and send a browser event.

The practical rule now is simple: server logs are used to find crawlability problems and page-level demand signals; browser analytics is used to judge real visitor growth. If a page gets repeated clean server hits but no browser sessions, we treat it as a discovery or crawler signal, not proof of acquired traffic.

This distinction matters because the acquisition queue can correctly identify pages worth improving while Umami still shows low real visitor volume. That is not a contradiction: one system is finding crawl and demand hints, the other is measuring browser sessions.

What pages got real visits first

Top landing behavior stayed practical and troubleshooting-heavy: Cloudflare 52x pages, AI crawler log checks, and core root files (/robots.txt, /sitemap.xml, /llms.txt).

High-intent pages such as how to choose an llms.txt generator and Bot Fight Mode troubleshooting showed recurring demand, so we expanded those first.

Example: how a demand signal becomes an edit

When OAI-SearchBot log checks and Claude-SearchBot zero-hit checks kept appearing in clean reports, we did not create broad generic SEO articles. We added narrower diagnostic pages that answer the exact operator question: what user agent to search for, which status codes matter, and how to separate a crawler-access problem from a ranking problem.

When the Cloudflare AI crawler troubleshooting hub started showing stable demand, we used it as the organizing page for 520-526, 403, WAF, and Bot Fight Mode paths. That hub is useful because a user with a crawler issue normally starts from a symptom, not from a product category.

How we measured

# Daily traffic baseline report
sudo /usr/local/bin/llmsfile-traffic-report /var/lib/caddy/logs/llmsfile-access.log /home/ubuntu/llmsfile-reports

# Opportunity view (top pages by unique IP)
sudo /usr/local/bin/llmsfile-opportunity-report /var/lib/caddy/logs/llmsfile-access.log /home/ubuntu/llmsfile-reports

# AI referral isolation
sudo /usr/local/bin/llmsfile-ai-referral-report /var/lib/caddy/logs/llmsfile-access.log /home/ubuntu/llmsfile-reports

Actions taken after the baseline

  1. Expanded Cloudflare troubleshooting coverage from 521-526 to 520-526 and added cross-links.
  2. Added feed and sitemap automation for faster update discovery.
  3. Improved log-quality filtering to remove synthetic monitor noise from growth decisions.
  4. Added bilingual intent pages where search logs showed repeated troubleshooting demand.
  5. Added an acquisition queue report to prioritize edits using clean unique-IP demand signals every day.
  6. Added a browser-traffic sanity check so server-side clean IP growth does not get mistaken for real daily visitor growth.

What this still does not prove

The uplift does not prove long-term ranking stability yet. We still need sustained indexing growth and non-zero referral sessions from AI answer surfaces.

Next focus: submit the priority URL queue in Search Console and Bing Webmaster Tools, publish a small number of relevant community posts, and keep pruning low-value content work. More pages alone will not move the site from early crawl discovery to daily visitor growth.

Open referral tracking workflow