Two sites with Cloudflare + free R2 + acceleration via selected domains: a postmortem that clears up all the pitfalls in one go

AIDO ops
, ,
1

A Serial Drama About Image Acceleration: How We Took Two Sites from “Accessible” to “Actually Fast”

Conclusion first:

  • d3v and aiya can now stably go through the Cloudflare + R2 path.
  • Both old and new images can be opened.
  • Local historical uploads can be backed up and then cleaned up—no more being held hostage by disk space.

Now the story:

Act One: We Thought It Was the Network, Turns Out It Was a “Systemic Detour”

At the beginning everyone was staring at latency:

  • China to Singapore, ping looked like 300ms.
  • SSH was so laggy it made you question life.
  • The business site felt okay because it used CF reverse proxy and an optimized domain.

So the soul-searching question arrived:

“Can SSH be accelerated like this too?”

Answer: you can tinker with it, but it’s not something you can just copy-paste from the HTTP acceleration playbook. Later we narrowed the focus to the image path first, because that’s what users feel the most.

Act Two: 404 Doesn’t Mean the Object Is Gone—It Means You’re Origin-Pulling from the Wrong Home

The most classic pitfall:

  • The link uploads.aiya.de5.net/... opened as 404.
  • But in R2, head_object showed the object was clearly there.

What does that mean?

It means “storage is fine, traffic is broken.”

Final diagnosis:

  • The SaaS custom hostname for uploads.aiya had, for a time, pointed its origin to the entrance of another bucket.
  • The origin Host/SNI and Nginx matching also had mismatches.
  • On top of that, Cloudflare cached the old 404, making it look like “it’s still broken even after it’s fixed.”

One-line summary:

An object being in the bucket doesn’t mean the user can fetch it.

Act Three: You Think It’s DNS, But It’s Actually “Multi-Layer Configuration Coupling”

This path actually had four layers:

  1. Huawei Cloud split-line routing (mainland/outside mainland)
  2. Cloudflare SaaS custom hostname
  3. Nginx origin routing/orchestration
  4. R2 bucket public/managed/custom domain

At any layer, a configuration that “looks basically fine” can send the request to the wrong bucket or the wrong Host.

So our fix wasn’t “change one thing on a whim,” but:

  • Back up first.
  • Then verify layer by layer (DNS → SaaS → origin → bucket object).
  • At every layer, use curl -I and object HEAD for fact-checking.

Act Four: Why Old Posts Always Love to “Look Like Nothing Changed”

Many people break down here:

“Didn’t I already move to S3/R2? Why do old posts still show aiya.de5.net/uploads/...?”

Because Discourse has two worlds:

  • raw: the original post text (commonly upload://...)
  • cooked: the rendered HTML (img/src/srcset lives here)

posts:rebake recomputes cooked from raw.

So if you only changed cooked, a later rebake may “revert you back to the default style.”

That’s not R2 failing—it’s normal behavior of the rendering strategy.

Act Five: What Exactly Did We Do This Time

What ultimately landed:

  • Fix uploads.aiya origin pull to the correct bucket.
  • Clear the old 404 cache.
  • Sync historically missing objects to R2.
  • Batch-replace old posts’ cooked to https://uploads.aiya.de5.net/... (so the display is unified too).

Results:

  • All the broken images you posted are fully restored.
  • Both new and old images hit the accelerated path.

A “Detour-Avoidance Checklist” for Those Who Come After

  1. Don’t suspect missing objects first—do HEAD object first.
  2. For 404, check cache status first: cf-cache-status: HIT may just be old cache.
  3. After changing origin for a SaaS custom hostname, be sure to verify actual Host matching.
  4. Change one layer at a time, verify layer by layer—don’t change multiple places at once and then pray.
  5. Back up before migrating, especially /shared/uploads/*.
  6. Writing down the “rollback path” is more important than “it finally worked this time.”

Easter Egg: The Most Real “Feels” This Time

This wasn’t a “one DNS record wasn’t filled correctly” problem.

This was “a semi-enterprise architecture stitched together out of free-tier plans”:

  • It can save money, speed things up, and run;
  • But you need SRE-level patience, and accept that it will teach you lessons at the edge cases.

Tinkering is tinkering, but in the end it was worth it:

  • Costs didn’t blow up;
  • Speed came up;
  • And the architecture finally went from “usable” to “explainable, maintainable.”

Hope this helps the next person stay up a few fewer nights.