Structured Logging in .NET

Every incident review I have ever sat through eventually arrives at the same uncomfortable question: what was the system actually doing at the moment it failed? In a regulated payments environment, that question is not academic. We need to reconstruct the sequence of events with enough precision to satisfy ourselves, our customers, and occasionally an auditor. The honest answer, more often than it should be, is that we cannot — because the logs we captured were a wall of unstructured strings that read like a diary entry rather than a queryable record.

Structured logging is the practice that fixed this for my teams. It is not glamorous, it does not appear on a roadmap, and it will never demo well. But it is one of the highest-leverage investments a .NET engineering organisation can make in its own ability to operate. In this post I want to lay out how I think about it, how we implement it on .NET, and the specific traps that turn a well-intentioned logging strategy into expensive noise.

What Structured Logging Actually Means

The distinction is simple but consequential. Traditional logging produces a formatted sentence: a string that has already been assembled, with the variables interpolated directly into the text. Structured logging instead preserves the message template and its parameters as discrete, named fields. The log event becomes a small object with properties you can index, filter, and aggregate, rather than a line of prose you have to parse with a regular expression at three in the morning.

Concretely, the difference is whether your log line is a sentence or a record. A sentence tells you that payment 8842 failed for merchant 19. A record tells you the same thing while also carrying PaymentId=8842 and MerchantId=19 as first-class fields that your log store understands. The first is readable by a human; the second is readable by a machine, which means it is readable by a human at scale.

Once you internalise that a log event is data, not text, almost every downstream decision becomes clearer. You stop debating log formatting and start designing a schema. You stop grepping and start querying. And you begin to treat your logs with the same discipline you apply to any other data your platform emits.

The Message Template Pattern

.NET has converged on a single idiomatic mechanism for this: the message template. The ILogger abstraction in Microsoft.Extensions.Logging accepts a template string with named placeholders, followed by the values that fill them. The crucial detail is that the named placeholders are not just string formatting — they become the property names on the resulting structured event.

// Correct: named placeholders become structured properties
_logger.LogInformation(
    "Settlement {SettlementId} for merchant {MerchantId} completed in {ElapsedMs}ms",
    settlement.Id, settlement.MerchantId, stopwatch.ElapsedMilliseconds);

// Wrong: string interpolation destroys the structure
_logger.LogInformation(
    $"Settlement {settlement.Id} for merchant {settlement.MerchantId} completed");

The second form looks almost identical and compiles cleanly, which is exactly why it is dangerous. By the time the logging framework receives that interpolated string, the values have already been flattened into text. There are no SettlementId or MerchantId properties to query later, and every settlement produces a message with a unique text that cannot be grouped. I treat the interpolated-string-into-a-logger pattern as a hard review failure, and I recommend enabling the analyser that flags it so the rule is enforced by the build rather than by a tired reviewer.

The placeholder names matter too. They are a public contract with your observability tooling. Once dashboards and alerts reference MerchantId, renaming it to MerchantIdentifier in a refactor will silently break them. We treat field names as part of our API surface and version them accordingly.

Choosing a Logging Stack on .NET

The built-in logging abstractions are excellent and provider-agnostic, which is precisely the point. Your application code should depend only on ILogger<T>; the decision about where logs go and in what format belongs to configuration, not to the call site. That separation has let us swap sinks several times without touching business logic.

For the actual structured pipeline we have standardised on Serilog, though the principles apply equally to NLog or the native OpenTelemetry logging exporter. Serilog earned its place because its enrichers, its first-class JSON formatting, and its sink ecosystem map cleanly onto the message-template model. The configuration that matters most is to emit compact JSON to standard output and let the platform — whether that is a container log collector or a hosted log service — handle shipping and indexing.

• Application code depends on ILogger<T> only, never on a concrete logging library.
• Composition root wires up the concrete provider, sinks, enrichers, and minimum levels.
• Output format is structured JSON in every environment except a developer's local console, where a human-readable formatter is fine.
• Sinks are configured, not coded, so a change of log destination is an operational decision, not a deployment.

Correlation and Scopes

A single log line is rarely useful in isolation. What you actually want during an investigation is every event associated with one request, one payment, or one customer journey, ordered in time and joinable across service boundaries. This is where logging scopes and correlation identifiers stop being a nice-to-have and become the entire reason the system is operable.

In .NET, ILogger.BeginScope attaches a set of properties to every log event emitted within a block, including by code further down the call stack. We open a scope at the edge of each request carrying a correlation identifier, and where appropriate the merchant and tenant context, so that nothing deeper in the stack has to remember to pass those values along.

using (_logger.BeginScope(new Dictionary<string, object>
{
    ["CorrelationId"] = context.CorrelationId,
    ["MerchantId"] = context.MerchantId
}))
{
    // Every log event inside this block, at any depth,
    // now carries CorrelationId and MerchantId automatically.
    await _settlementService.ProcessAsync(request);
}

In a distributed system this pairs naturally with the W3C trace context that System.Diagnostics.Activity propagates. When your correlation identifier and your trace identifier are both present on every event, a single query reconstructs the full path of a request across every service it touched. That capability is the difference between a fifteen-minute investigation and a three-hour one.

What Belongs in a Log Event

Discipline about content is as important as discipline about format. A log event should answer who, what, and in what context, with enough identifiers to join it to related events and enough detail to be actionable on its own. It should never contain a full object dump, a stack trace embedded in a message string, or anything that belongs in a metric rather than a log.

I encourage teams to think in terms of stable identifiers and outcomes. Log the merchant identifier, the operation name, the result, the elapsed time, and any business-meaningful status code. Do not log the entire request body because it was convenient. Verbose logging feels thorough in the moment and becomes a liability the day you need to search through a terabyte of it.

A log line you cannot query at scale is a log line you do not have. If the only way to find it is to read every line by eye, it will not be found during the incident that needs it.

There is also a correctness dimension. Because structured events are typed, a value logged as a number stays a number in your store, which means you can compute percentiles over elapsed times or sum amounts directly. Flattening everything to strings throws that away and forces brittle reparsing downstream.

Sensitive Data and Redaction

In fintech this section is non-negotiable, and it is the one I scrutinise most closely in review. Structured logging makes data extraction trivial, which is wonderful for engineers and equally wonderful for anyone who later breaches your log store. A primary account number that lands in a log is a compliance incident regardless of how well-indexed it is, and a log store is rarely held to the same access controls as the primary database.

We handle this in two layers. First, we never pass sensitive fields to the logger in the first place; the safest data is the data you never captured. Second, because human discipline is not a control, we add a redaction layer in the pipeline that masks known-sensitive property names and applies pattern detection for card-like and account-like values before anything is written to a sink. .NET's Microsoft.Extensions.Compliance.Redaction packages formalise this with data-classification attributes, so a property tagged as sensitive is redacted consistently wherever it appears.

The principle I repeat to every new engineer is that logs leave the trust boundary. They are shipped to collectors, replicated, retained for months, and read by people who never see production data directly. Treat every log event as if it will be read by someone outside your team, because eventually it will be.

Log Levels and Cost Discipline

Log levels are a budget, not a decoration. Every event you emit at information level in a high-throughput service has a real cost in ingestion, storage, and the cognitive load of the people who later have to filter past it. I have watched teams set everything to information because it was easier than thinking, then spend six figures a year ingesting heartbeat messages nobody ever reads.

My working rules are unglamorous but they hold up. Errors and warnings describe something a human may need to act on. Information describes business-significant events — a payment settled, a webhook delivered — at a rate proportional to those events, not to internal loop iterations. Debug and trace are for development and targeted investigation, switched on dynamically for a specific category rather than left running globally.

The dynamic part matters. .NET's configuration providers let you change the minimum level for a specific logging category at runtime, which means you can raise verbosity for one suspect component during an incident and lower it again afterwards, without a redeploy. That capability removes the temptation to over-log defensively, because the detail is available on demand rather than always-on by default.

Testing and Enforcing the Contract

If logs are data and field names are a contract, then both deserve to be tested and enforced rather than left to good intentions. We assert on log output in a handful of critical paths — verifying that a failed settlement emits an event at the expected level carrying the expected identifiers — using a fake ILogger implementation that captures structured state. These tests catch the regression where someone quietly converts a structured call into an interpolated string.

Beyond unit tests, the Roslyn analysers shipped with the logging libraries enforce template correctness at compile time, and the high-performance LoggerMessage source generator gives us strongly-typed, allocation-free log methods for hot paths. Defining log events as generated methods turns the message template into something the compiler checks, which is exactly where I want that contract enforced.

public static partial class Log
{
    [LoggerMessage(
        EventId = 4102,
        Level = LogLevel.Warning,
        Message = "Settlement {SettlementId} retried {Attempt} of {MaxAttempts}")]
    public static partial void SettlementRetried(
        this ILogger logger, long settlementId, int attempt, int maxAttempts);
}

This pattern pays off twice. The event identifier becomes a stable handle that alerts and dashboards can reference even if the message wording changes, and the generated method makes it impossible to call the log incorrectly. For a platform team, that stability is worth far more than the marginal performance gain.

Conclusion

Structured logging is one of those investments where the cost is paid upfront, by disciplined engineers, and the return arrives later, during the incidents and audits you cannot schedule. The .NET ecosystem gives us everything we need to do it well: a clean logging abstraction, message templates that preserve structure, scopes for correlation, source generators for performance, and redaction tooling for the compliance obligations that come with handling money. None of it is exotic; the difficulty is entirely in the discipline.

If you take one thing from this, let it be that a log event is data and should be designed like data — with named fields, a stable schema, deliberate levels, and an honest accounting of what must never appear in it. Do that consistently, and the next time someone in an incident review asks what the system was actually doing, you will be able to answer with a query instead of a shrug.