Why We Log in JSON

Three years ago I spent a Saturday night grepping through 40 gigabytes of plaintext logs trying to work out why a batch of settlement files had gone missing. The logs were human-readable, which sounded great in the design review and was worthless at 11pm. Every service formatted its lines differently. Some put the timestamp first, some buried it after the thread name, one heroic team had decided the log level belonged at the end. I wrote three throwaway regexes and got two of them wrong.

We moved to structured JSON logging the following quarter. It is not glamorous work, and nobody puts "migrated the log format" on a promotion packet. But it is one of the few infrastructure decisions I have never once regretted, and I want to explain why, because I still meet engineers who think JSON logs are bureaucratic overkill for a service that "just prints what it's doing."

Grep Stops Working Around The Third Service

Plaintext logging works beautifully right up until you have more than one process producing logs, and then it quietly stops working. The problem is not volume, exactly. The problem is that a log line written for a human to read is a sentence, and sentences do not have a schema. When you want to answer "show me every failed authorization for merchant 4471 in the last hour," you are asking a question about fields, and a sentence has no fields.

So you reach for regex, and regex against free text is a losing game. The message "payment declined for merchant 4471" and "merchant 4471: payment was declined" carry the same meaning and match completely different patterns. Someone edits a log string to be clearer for humans, and three dashboards break silently because their extraction rules no longer fire. I have watched an on-call engineer chase a phantom outage for twenty minutes because a log-parsing rule had been matching a substring that moved.

With JSON, the field is the field. The merchant id lives under a key called merchant_id whether the surrounding message says "declined" or "rejected" or nothing at all. You stop parsing prose and start querying data, and that shift is the whole point.

What A Good Line Actually Looks Like

Here is roughly the shape we settled on. It is not exotic. Every line is one JSON object, one object per line, so tools can read it a line at a time without holding the whole file in memory.

{
  "timestamp": "2026-07-29T14:22:03.184Z",
  "level": "warn",
  "service": "authorization-api",
  "trace_id": "b1f3c9a2e7d04f1a",
  "merchant_id": "4471",
  "event": "authorization_declined",
  "decline_reason": "insufficient_funds",
  "amount_minor": 1299,
  "currency": "EUR",
  "latency_ms": 34
}

Notice what is not in there. There is no free-text sentence trying to describe all of the above in English. The event key is a stable machine token, not a sentence: authorization_declined, never "The authorization was declined because...". Amounts are in minor units as integers, because I have been burned by a float turning 12.99 into 12.9899999998 after two currency conversions, in a log that later got quoted back at us in a dispute. Currency is separate. And the trace_id ties this line to every other line from the same request across every service it touched.

The Trace ID Is Where It Pays For Itself

If you take one thing from this, take this. Put a correlation identifier on every log line and thread it through every service call. In a payments flow a single customer tap might cross an API gateway, a risk engine, a tokenization service, an acquirer connector, and a ledger writer. When something goes wrong, the question is never "what did one service do," it is "what happened to this request everywhere."

With a trace_id on every JSON line, that becomes a single filter. You pull every line matching the id, sort by timestamp, and you are reading the story of one payment across six systems in one view. Before we did this, reconstructing a cross-service failure meant a war room, three engineers, and a shared screen. After, it is usually one person and about ninety seconds.

The best log line is one you never have to read individually, because you can select thousands of them at once and ask a question of the aggregate.

The Fields You Will Thank Yourself For

Getting the format right matters less than getting the field discipline right. A JSON object with sloppy, inconsistent keys is barely better than plaintext, because now you have the overhead of JSON and none of the queryability. We keep a short, boring list of fields that every service must emit, and we enforce it in a shared logging library rather than hoping people remember.

• timestamp in UTC, ISO 8601, always. Local time in logs is a special kind of self-harm.
• level as a lowercase enum, so you can filter to warn-and-above without a case-insensitive match.
• service and version, so you can tell which deployment produced a line during a rollout.
• trace_id, non-negotiable, generated at the edge if the caller did not supply one.
• event as a stable token from a known vocabulary, never an ad-hoc sentence.

The stable event vocabulary is the part teams resist and later love. When authorization_declined always means exactly the same thing across every service, you can build an alert on it that survives refactors. When it is a sentence someone rewrote for clarity, your alert is one pull request away from silence.

Making It Painless In .NET

The single biggest reason JSON logging fails in practice is that it is more effort to write a good line than a lazy one, so people write lazy ones. The fix is to make the structured path the easy path. In .NET we lean on the built-in structured logging in the message template, so the developer writes something that reads naturally and the framework produces the JSON.

_logger.LogWarning(
    "Authorization declined for {MerchantId} reason {DeclineReason} amount {AmountMinor} {Currency}",
    merchantId, declineReason, amountMinor, currency);

The trick is that the values in braces are not string interpolation. They become named fields in the emitted JSON: MerchantId, DeclineReason, and so on. The developer gets a readable call site, and the sink gets structured data. If someone uses C# string interpolation with a dollar sign instead, that structure is lost and the whole thing collapses back into a sentence, so we have a Roslyn analyzer that fails the build on interpolated strings inside a log call. It is a small rule that has saved us more grief than most of our fancier tooling.

Yes, It Costs More, And That Is A Real Trade-off

I am not going to pretend JSON is free. A structured line is bigger than a terse plaintext one, often two to three times the bytes once you have keys on everything. If your log platform bills by ingested volume, and most do, that is a line item you will feel. Our ingestion bill jumped roughly 40 percent the month we cut over, and the finance team asked some pointed questions.

My answer then and now is that the cost is real and it is worth it, but only if you pair it with sampling discipline. We do not log every successful, boring, sub-50-millisecond authorization at full fidelity. We sample the happy path at a low rate and keep everything for warnings, errors, and anything touching a dispute or a regulatory obligation. That kept the bill sane without giving up the lines that actually matter during an incident. The mistake is treating "log everything in JSON" and "log everything" as the same decision. They are not.

The Part Everyone Underestimates: PII

Structured logging makes it trivially easy to log a full card number into a field called card_number, and then it is sitting in your log platform forever, searchable, and you have a compliance incident. This is the genuine downside nobody mentions in the blog posts. When logging was a hand-written sentence, a developer had to consciously type out the sensitive value. When it is a structured object built from a domain model, the sensitive fields come along for the ride unless you stop them.

We handle this with a redaction layer in the logging pipeline that runs before anything is written to disk or shipped anywhere. Certain field names are hard-blocked, certain patterns like anything resembling a PAN get masked to the first six and last four, and the redaction rules are tested like production code because they are production code. The rule I give every new engineer is simple.

If you would not put it on a postcard, it does not go in a log field. And if you are not sure, assume the postcard.

What Actually Changed On The Team

The technical wins are easy to list. The cultural change is the one I did not expect. Once logs became queryable data, engineers started treating them as a product surface rather than debugging exhaust. People began asking "what question will future-me want to ask of this line" before writing it, which is a completely different mindset from "print enough that I can eyeball it tonight."

Our mean time to diagnose a production incident dropped by more than half over the following year. I cannot cleanly attribute all of that to logging, because we improved other things too, but the on-call engineers will tell you the logging change was the biggest single factor, and I believe them. The 11pm settlement mystery from the start of this piece would today be a saved query that anyone on the team could run in under a minute, PII already redacted, every service in one timeline.

Conclusion

If you are running a single script on a single box, keep printing sentences and go enjoy your weekend. This is not a religion. But the moment you have more than a couple of services, or you handle money, or an auditor can subpoena your logs, the calculus flips hard. Structured logging is not about looking sophisticated. It is about the difference between answering a question and hunting for one. Do the boring migration before the incident that makes you wish you had, not during it. The Saturday night you save will probably be your own.