ASP.NET Core WebAPI Payload Logging with Serilog

Problem Statement

Imagine debugging a tricky API integration where you’re getting intermittent errors. You suspect the data being sent is the culprit, but you’re flying blind. Wouldn’t it be great if you could see exactly what’s going over the wire?.
For some API-based applications, logging the payload for a given transaction is helpful from both audit and monitoring perspectives. While it’s generally not recommended to log the request or response body due to performance/compliance/security considerations, if there is a need for such a requirement and you understand all the risks, then hopefully this guide provides one of the many ways to achieve it.
In this article, we will look at how to create JSON structured log events for HTTP API request/response transactions in an ASP.NET Core Web API using the Serilog logger and the HTTP Logging Middleware provided by Microsoft.

Goal

By the end of this article, you’ll be able to:

Have well-structured log events containing the request/response payloads as distinct JSON objects (no complex regex required!).
Avoid buffering payloads for non-JSON requests/responses to optimize performance.(specially for static content serving)
Achieve all of this without writing custom middleware, leveraging Serilog flexibility & Microsoft built-in interceptor!!.

Note: Following example entire code is located here https://github.com/imran9m/serilog-payload-demo.

Tools Used

Serilog: We will configure Serilog & Serilog.Formatting.Compact to send logs in JSON format.
Microsoft HTTP Logging Middleware: We will enable this middleware to it can capture the Request & Response payload without us writing custom middleware code to do it.
Serilog.Expressions: We will use this to conditionally route log events between different logger contexts.

Steps

Create New WebAPI Project

Lets create new Dotnet API Project with following CLI command
```
mkdir serilog-payload-demo
cd serilog-payload-demo
dotnet new webapi
```

Add Dependencies

Lets add dependencies we will be using.

dotnet add package Serilog
dotnet add package Serilog.AspNetCore
dotnet add package Serilog.Settings.Configuration
dotnet add package Serilog.Sinks.Console
dotnet add package Serilog.Enrichers.Process
dotnet add package Serilog.Enrichers.Thread
dotnet add package Serilog.Expressions
dotnet add package Serilog.Formatting.Compact

Load Serilog Configuration from appsettings

Lets create SerilogConfiguration class which loads the Serilog configuration from appsettings.json files.

Content for SerilogConfiguration.cs

using Serilog;

namespace serilog_payload_demo.Configuration;

public static class SerilogConfiguration
{
    public static ILoggingBuilder AddCustomSerilogConfiguration(this ILoggingBuilder builder, IConfiguration configuration)
    {
        var loggerConfiguration = new LoggerConfiguration()
            .Enrich.FromLogContext()
            .ReadFrom.Configuration(configuration);

        Log.Logger = loggerConfiguration.CreateLogger();

        return builder.AddSerilog(Log.Logger);
    }
}

Custom HTTP Interceptor

We will create custom Interceptor to enable request & response payload log buffer when content type of the payload includes json so this way we are not buffering payloads for all the requests which doesn’t need payload logging(static content etc.,).

You can also further customize this interceptor for following.

Filter to specific requests matching based on URL patterns or based specific HTTP Response return code range etc.,
Also create custom log fields containing masked sensitive data etc.,
Find more things at IHttpLoggingInterceptor documentation.

Content for CustomHttpLoggingInterceptor.cs

using Microsoft.AspNetCore.HttpLogging;

namespace serilog_payload_demo.Configuration;

internal sealed class CustomHttpLoggingInterceptor: IHttpLoggingInterceptor
{
    public ValueTask OnRequestAsync(HttpLoggingInterceptorContext logContext)
    {
        if (logContext.HttpContext.Request.ContentType != null && logContext.HttpContext.Request.ContentType.Contains("json"))
        {
            logContext.LoggingFields = HttpLoggingFields.RequestBody
                | HttpLoggingFields.RequestHeaders
                | HttpLoggingFields.RequestProtocol
                | HttpLoggingFields.RequestQuery;
        }
        else
        {
            logContext.LoggingFields = HttpLoggingFields.None;
        }
        return default;
    }

    public ValueTask OnResponseAsync(HttpLoggingInterceptorContext logContext)
    {
        if (logContext.HttpContext.Response.ContentType != null && logContext.HttpContext.Response.ContentType.Contains("json"))
        {
            logContext.LoggingFields = HttpLoggingFields.RequestBody
                                    | HttpLoggingFields.RequestHeaders
                                    | HttpLoggingFields.RequestProtocol
                                    | HttpLoggingFields.RequestQuery
                                    | HttpLoggingFields.ResponseBody
                                    | HttpLoggingFields.ResponseHeaders
                                    | HttpLoggingFields.Duration
                                    | HttpLoggingFields.ResponseStatusCode;
        }
        return default;
    }
}

Capturing Payload Events

Now for the tricky part, since we let HttpLoggingMiddleware capture payloads with above interceptor, let’s get these log events routed through custom Serilog Formatter to properly parse JSON payloads and create a useful JSON log event.

Note: This formatter is heavily inspired from CompactJsonFormatter
Content for PayloadLogFormatter.cs.

using System.Globalization;
using Newtonsoft.Json;
using Serilog.Events;
using Serilog.Formatting;
using Serilog.Formatting.Json;
using Serilog.Parsing;

namespace serilog_payload_demo.Configuration;

public class PayloadLogFormatter : ITextFormatter
{
    readonly JsonValueFormatter _valueFormatter;

    public PayloadLogFormatter(JsonValueFormatter? valueFormatter = null)
    {
        _valueFormatter = valueFormatter ?? new JsonValueFormatter(typeTagName: "$type");
    }

    public void Format(LogEvent logEvent, TextWriter output)
    {
        FormatEvent(logEvent, output, _valueFormatter);
        output.WriteLine();
    }

    public static void FormatEvent(LogEvent logEvent, TextWriter output, JsonValueFormatter valueFormatter)
    {
        if (logEvent == null) throw new ArgumentNullException(nameof(logEvent));
        if (output == null) throw new ArgumentNullException(nameof(output));
        if (valueFormatter == null) throw new ArgumentNullException(nameof(valueFormatter));

        output.Write("{\"@t\":\"");
        output.Write(logEvent.Timestamp.UtcDateTime.ToString("O"));
        output.Write("\",\"@m\":\"PayloadEvent\"");

        output.Write(",\"@l\":\"");
        output.Write(logEvent.Level);
        output.Write('\"');

        if (logEvent.TraceId != null)
        {
            output.Write(",\"@tr\":\"");
            output.Write(logEvent.TraceId.Value.ToHexString());
            output.Write('\"');
        }

        if (logEvent.SpanId != null)
        {
            output.Write(",\"@sp\":\"");
            output.Write(logEvent.SpanId.Value.ToHexString());
            output.Write('\"');
        }


        foreach (var property in logEvent.Properties)
        {
            var name = property.Key;
            if (name.Length > 0 && name[0] == '@')
            {
                // Escape first '@' by doubling
                name = '@' + name;
            }
            //TO-DO: We can write the code in programmatic way to allow which properties to be logged part of this event.
            switch (name)
            {
                case "HttpLog":
                    break;
                case "SourceContext":
                    output.Write(',');
                    JsonValueFormatter.WriteQuotedJsonString("@sc", output);
                    output.Write(':');
                    valueFormatter.Format(property.Value, output);
                    break;
                case "Duration":
                    output.Write(',');
                    JsonValueFormatter.WriteQuotedJsonString("duration", output);
                    output.Write(':');
                    valueFormatter.Format(property.Value, output);
                    break;
                case "StatusCode":
                    output.Write(',');
                    JsonValueFormatter.WriteQuotedJsonString("statusCode", output);
                    output.Write(':');
                    valueFormatter.Format(property.Value, output);
                    break;
                case "ResponseBody":
                    output.Write(',');
                    JsonValueFormatter.WriteQuotedJsonString("responseBody", output);
                    output.Write(':');
                    output.Write(JsonConvert.DeserializeObject(property.Value.ToString()).ToString().Replace("\n", ""));
                    break;
                case "RequestBody":
                    output.Write(',');
                    JsonValueFormatter.WriteQuotedJsonString("requestBody", output);
                    output.Write(':');
                    output.Write(JsonConvert.DeserializeObject(property.Value.ToString()).ToString().Replace("\n", ""));
                    break;
                case "RequestPath":
                    output.Write(',');
                    JsonValueFormatter.WriteQuotedJsonString("requestPath", output);
                    output.Write(':');
                    valueFormatter.Format(property.Value, output);
                    break;
                case "Method":
                    output.Write(',');
                    JsonValueFormatter.WriteQuotedJsonString("requestMethod", output);
                    output.Write(':');
                    valueFormatter.Format(property.Value, output);
                    break;
                default:
                    output.Write(',');
                    JsonValueFormatter.WriteQuotedJsonString(name, output);
                    output.Write(':');
                    valueFormatter.Format(property.Value, output);
                    break;
            }
        }

        output.Write('}');
    }
}

AppSettings Serilog Configuration

Lets add following Serilog section to appsettings-*.json so, we can conditionally route the log events to our formatter for Microsoft.AspNetCore.HttpLogging.HttpLoggingMiddleware events.
Note: This conditional routing is achieved using Serilog.Expressions. Again, for the demo purpose, I have used Serilog Console Sink but you can change it to your preferred Sinks.
Below is the appsettings-Development.json content.

{
    "Serilog": {
        "MinimumLevel": "Information",
        "WriteTo": [
            {
                "Name": "Conditional",
                "Args": {
                    "expression": "SourceContext <> 'Microsoft.AspNetCore.HttpLogging.HttpLoggingMiddleware'",
                    "configureSink": {
                        "Console": {
                            "Name": "Console",
                            "Args": {
                                "formatter": "Serilog.Formatting.Compact.CompactJsonFormatter, Serilog.Formatting.Compact"
                            }
                        }
                    }
                }
            },
            {
                "Name": "Conditional",
                "Args": {
                    "expression": "SourceContext = 'Microsoft.AspNetCore.HttpLogging.HttpLoggingMiddleware'",
                    "configureSink": {
                        "Console": {
                            "Name": "Console",
                            "Args": {
                                "formatter": {
                                    "type": "serilog_payload_demo.Configuration.PayloadLogFormatter, serilog-payload-demo"
                                }
                            }
                        }
                    }
                }
            }
        ]
    },
    "Logging": {
        "LogLevel": {
            "Default": "Information",
            "Microsoft.AspNetCore": "Warning"
        }
    }
}

Lets bring this together

Lets enable Serilog and HTTP Middleware with custom interceptor in Program.cs or wherever applicable for your app before builder.Build(); line.

using Microsoft.AspNetCore.HttpLogging;
using Serilog;
using serilog_payload_demo.Configuration;
using Microsoft.Extensions.Options;

builder.Services.AddSerilog();
builder.Logging.AddCustomSerilogConfiguration(builder.Configuration);
builder.Services.AddHttpLogging(o =>
{
    o.CombineLogs = true;
}).AddHttpLoggingInterceptor<CustomHttpLoggingInterceptor>();

Enable HTTPLogging for the app.

app.UseHttpLogging();

Let’s look at the Results

I have performed same changes in demo app which you can checkout and run.

Once you run dotnet run and execute an API call at http://localhost:5077/weatherforecast/ then you should be able to see following logs produced in Console.

{"@t":"2025-02-23T04:31:38.3818101Z","@mt":"Now listening on: {address}","address":"http://localhost:5077","EventId":{"Id":14,"Name":"ListeningOnAddress"},"SourceContext":"Microsoft.Hosting.Lifetime"}
{"@t":"2025-02-23T04:31:38.3989156Z","@mt":"Application started. Press Ctrl+C to shut down.","SourceContext":"Microsoft.Hosting.Lifetime"}
{"@t":"2025-02-23T04:31:38.3992277Z","@mt":"Hosting environment: {EnvName}","EnvName":"Development","SourceContext":"Microsoft.Hosting.Lifetime"}

For the API execution, you should also see a log event for PayloadEvent message in this format.

{
    "@t": "2025-02-23T04:31:47.2198850Z",
    "@m": "PayloadEvent",
    "@l": "Information",
    "@tr": "b5fa803d71687958c047a05acffd013c",
    "@sp": "e8b0c58db6c9835e",
    "statusCode": 200,
    "Content-Type": "application/json; charset=utf-8",
    "responseBody": [
        {
            "date": "2025-02-23",
            "temperatureC": 4,
            "summary": "Chilly",
            "temperatureF": 39
        },
        {
            "date": "2025-02-24",
            "temperatureC": -9,
            "summary": "Bracing",
            "temperatureF": 16
        },
        {
            "date": "2025-02-25",
            "temperatureC": 45,
            "summary": "Hot",
            "temperatureF": 112
        },
        {
            "date": "2025-02-26",
            "temperatureC": -3,
            "summary": "Balmy",
            "temperatureF": 27
        },
        {
            "date": "2025-02-27",
            "temperatureC": 38,
            "summary": "Hot",
            "temperatureF": 100
        }
    ],
    "duration": 11.5259,
    "EventId": {
        "Id": 9,
        "Name": "RequestResponseLog"
    },
    "@sc": "Microsoft.AspNetCore.HttpLogging.HttpLoggingMiddleware",
    "RequestId": "0HNAJOBTUGLRV:00000001",
    "requestPath": "/weatherforecast/",
    "ConnectionId": "0HNAJOBTUGLRV"
}

Final Words

This is one of the ways to achieve payload logging in ASP.NETCore Web APIs. Keep in mind, this code was not tested for performance benchmarks so use it with caution.
IMO, the complex part of this solution is PayloadFormatter, just keep an eye for any exceptions when deserializing the payloads which might result in no log event being generated. Hint: Add better exception handling!!

If you have any feedback/issues, you can submit an issue in site repo.

Problem Statement#

Goal#

Tools Used#

Steps#

Create New WebAPI Project#

Add Dependencies#

Load Serilog Configuration from appsettings#

Custom HTTP Interceptor#

Capturing Payload Events#

AppSettings Serilog Configuration#

Lets bring this together#

Let’s look at the Results#

Final Words#