How We Run Blue-Green Deployments
The first time I ran a blue-green deployment in anger, it was 6:40 on a Friday evening and the payment authorization service had been down for eleven minutes....
9 min613Read →
AI & Technology — Page 2
The Case for Boring Technology in Fintech
A few years ago I sat in a postmortem for an outage that cost us most of a business day of failed settlements. The root cause was not a bug in anything we had...
10 min806Read →
Building a Rules Engine You Can Actually Maintain
The worst rules engine I ever inherited had 1,847 rules in a single database table, evaluated top to bottom, and the order they ran in was determined by the...
10 min504Read →
Designing Safe Database Schema Changes
Of all the operational risks I worry about as a CTO in regulated payments, the one that keeps recurring in postmortems is not a fancy distributed-systems...
11 min354Read →
Observability Beyond Logging
For the first decade of my career, observability meant one thing: logs. When something broke, you SSH'd into a box, tailed a file, and grepped for the word...
11 min396Read →
Handling Timezones and Dates Correctly
Of all the categories of bugs I have seen ship to production over a twenty-year career, the ones rooted in date and time handling are the most insidious. They...
11 min970Read →
Designing a Settlement System
Settlement is the part of a payments business that nobody talks about and everybody fights about in production. It is the machinery that takes the noisy stream...
11 min359Read →
Connection Pooling and Why It Matters
Most of the worst production incidents I have managed in fintech did not start with a dramatic failure. They started quietly, with a service that was perfectly...
12 min926Read →
Building a Notification Service That Scales
Notifications look like a small feature until you operate one at scale. Then you discover that the humble "we just send the customer an email" requirement is...
12 min781Read →
Tokenization vs Encryption for Card Data
Every few months a product manager or a board member asks me some version of the same question: "We encrypt the card numbers, so we are PCI compliant and safe,...
11 min441Read →
Circuit Breakers and Graceful Degradation
Every distributed system fails. That is not a pessimistic statement; it is an operational one. In the payments platforms I have run, the question was never...
11 min745Read →
Database Indexing for Application Developers
Most application developers treat the database as a black box. They write a query, it returns rows, and as long as the page loads in under a second nobody asks...
11 min720Read →