Designing a Ledger-Backed Wallet System
The first wallet system I inherited stored balances as a single column on the user row. A background job updated it. When two payments landed in the same 200...
The first wallet system I inherited stored balances as a single column on the user row. A background job updated it. When two payments landed in the same 200...
A few years ago we shipped a payment reconciliation job that, under load, occasionally credited a merchant twice for the same settlement. Not often. Maybe one...
The first time I ran a blue-green deployment in anger, it was 6:40 on a Friday evening and the payment authorization service had been down for eleven minutes....
A few years ago I sat in a postmortem for an outage that cost us most of a business day of failed settlements. The root cause was not a bug in anything we had...
The worst rules engine I ever inherited had 1,847 rules in a single database table, evaluated top to bottom, and the order they ran in was determined by the...
Of all the operational risks I worry about as a CTO in regulated payments, the one that keeps recurring in postmortems is not a fancy distributed-systems...
For the first decade of my career, observability meant one thing: logs. When something broke, you SSH'd into a box, tailed a file, and grepped for the word...
Of all the categories of bugs I have seen ship to production over a twenty-year career, the ones rooted in date and time handling are the most insidious. They...
Settlement is the part of a payments business that nobody talks about and everybody fights about in production. It is the machinery that takes the noisy stream...
Most of the worst production incidents I have managed in fintech did not start with a dramatic failure. They started quietly, with a service that was perfectly...
Notifications look like a small feature until you operate one at scale. Then you discover that the humble "we just send the customer an email" requirement is...
Every few months a product manager or a board member asks me some version of the same question: "We encrypt the card numbers, so we are PCI compliant and safe,...