We Treated Our Ledger Like a Database Table. It Cost Us Six Months

This story was originally published on HackerNoon at: https://hackernoon.com/we-treated-our-ledger-like-a-database-table-it-cost-us-six-months. A fintech team's production incident revealed why account balances should be derived from immutable ledger entries, not stored state. Check more stories related to finance at: https://hackernoon.com/c/finance. You can also check exclusive content about #fintech, #system-architecture, #database-design, #backend-development, #ledger-architecture, #double-entry-accounting, #idempotency, #payments-infrastructure, and more. This story was written by: @olabiisi_1. Learn more about this writer by checking @olabiisi_1's about page, and for more stories, please visit hackernoon.com. We stored account balances as a mutable column and updated them on every transfer. It worked fine until a race condition double-processed a batch of payouts, and we discovered we had no way to independently verify what the correct balances should have been. We rebuilt the system around an append-only ledger, where balances are computed from history rather than stored directly, backfilled six weeks of parallel verification, and only then retired the old column. Here's what broke, why it broke, and the architecture that replaced it.