10 Apr 2026 · 8 min read
Migrating a Warehouse Without Stopping the Business
A 200-table regulatory warehouse, a cloud target, and a business that could not miss a single Monday submission. The dual-run playbook, the metrics that kept us honest, and the graph that got the CFO to sign.
The engagement brief was one sentence with a regulator hiding in it: move the warehouse to the cloud, and the Monday submissions cannot be late, wrong, or different. Two hundred tables, nine years of undocumented business logic, and a downstream reporting estate nobody fully mapped. The following is that fictionalised-but-faithful story, told through the diagrams we actually used.
The only honest strategy: dual-run
Big-bang migrations are a bet that you understand a legacy system completely. Nine years of accreted logic said we didn’t. So we ran both — legacy and target, side by side, with a machine checking they agreed. Scroll through how the architecture evolved:
phase 0 · the incumbent
Phase 0 — respect the incumbent
The legacy warehouse feeds the Monday regulatory batch. It is slow, undocumented, and correct. That last property is the crown jewel — everything that follows exists to protect it.
Phase 1 — shadow ingestion
Change-data-capture off the legacy sources into the lakehouse. Zero risk: nothing downstream depends on it yet. This phase looks like no progress to a steering committee and is where all the real learning happened — the ingestion code was written in pairs, on purpose, so it was nobody's private kingdom.
Phase 2 — dual-run with a diff harness
New marts rebuilt the reports; the diff harness compared every figure against legacy, every day. Every mismatch was a fossil of undocumented logic — 9 years of "oh, THAT'S why" — burned down one by one. The harness, not the migration, was the actual project.
Phase 3 — the boring cutover
After 47 consecutive green days, the report feed switched sources. The legacy warehouse ran three more months as a warm fallback nobody needed. The cutover meeting lasted eleven minutes. That was the point.
The chart that got the CFO to sign
Steering committees don’t approve architectures; they approve trends. This is the one that mattered — daily mismatched figures between old and new, across the dual-run:
Two things to notice. The spike-shoulder around week 9 was a single 2013 VAT rounding decision, replicated inconsistently in eleven downstream tables — the kind of thing no discovery workshop would ever have found. And the flat zero from week 18: we didn’t cut over at first green. We waited for boring green. Forty-seven days of it.
The timeline, compressed
What actually de-risked it
Not the cloud platform, and honestly not the architecture diagram. Three habits:
- The diff harness as first-class product. We staffed it like a feature, not a test. It had an owner, a backlog, and a UI. Migrations fail on verification, not on construction.
- Pairing as knowledge insurance. Every component, two authors, from day one. When our senior engineer left in week 14 — they always leave in week 14 — the plan didn’t wobble.
- Cutover criteria agreed in week 1. “47 green days” wasn’t improvised; the number was signed off before the first pipeline existed. Deciding how you’ll know it’s safe before you’re emotionally invested is the cheapest risk management there is.
The team that ran the old warehouse runs the new one. Nobody was migrated out of a job; they were coached into a new stack on real work, week by week. That was the other deliverable — the one that wasn’t in the brief but was always the actual assignment.
Figures altered to protect the guilty. The eleven-minute meeting is real and I have witnesses.