Practical OpenTelemetry — Part 8: Logging: Enrich, Don't Replace

Here’s a sentence that surprises people coming to OpenTelemetry fresh: the Logs signal is the newest, least stable part of the specification — and the project’s design intent is explicitly not to replace your logging framework.

Chapter 8 of Practical OpenTelemetry covers why: logs predate every other signal, the ecosystem is enormous (Log4j, Logback, Zap, Winston, plus every platform’s stdout), and the only realistic path is integration, not replacement. Most of that stdout convention traces back to one influential piece of guidance — the Twelve-Factor App’s instruction to treat logs as an unbuffered event stream written to stdout and let the execution environment handle routing, rather than have the application manage log files itself. OTel’s logging design leans into that convention instead of fighting it: this post covers the Logs and Events APIs, the SDK, and the two ways OTel plugs into your existing logging stack.

What Logs Are Good For

Part 6 argued tracing beats logs for operations inside distributed transactions. But logs still own four territories:

1. Operations outside user transactions — startup/shutdown routines, background tasks, internal replica state. No span exists to hang these on.
2. Legacy and third-party code — libraries instrumented with standard logging frameworks that you can’t (or won’t) modify. OTel can enrich their output with trace and span IDs, making them correlatable.
3. Real User Monitoring — browser and mobile events (page load timers, JS errors, Core Web Vitals). Millions of devices can’t be aggregated at the producer; raw events must be exported and aggregated centrally.
4. OS and infrastructure events — log files, Kubernetes events, Syslog, Windows Event Logs. Operators can’t change the content; Collectors can enrich it with standardized attributes.

The book’s line about the pre-OTel problem is worth quoting: without trace context, “log sampling is difficult and rarely done” — log events can only be considered individually, so organizations rarely sample logs at all, incurring high cost for little debugging value. Meanwhile, tracing samples intelligently per transaction. The result: teams pay to store a mountain of logs, most of which never get read.

The Logs API vs. the Events API

The API splits in an unusual way:

• Logs API — treated as a backend API. Not intended for direct application use; it’s what logging appenders and handlers call under the hood.
• Events API — the application-facing interface for emitting events with a defined set of attributes.

The reasoning is deliberate: structured logs blur the line with events (same underlying data structures, different meaning). Rather than force a new logging API on a world that already has good ones, OTel provides both interfaces and encourages standard frameworks for everyday logging.

Each log record carries: timestamp, observed timestamp (when a Collector saw it), context (for trace correlation), severity number and text (17–20 = ERROR), body, and attributes. The Events API adds two mandatory conventions — event.domain (browser, device, k8s, or custom — which system this event belongs to) and event.name (what kind of event it is within that domain).

The book is careful to caveat this next snippet — using the Logs API directly is explicitly not recommended once an appender is configured, since that’s the whole point of the two-integration-paths design below. But seeing the raw shape clarifies everything else in this chapter. In Java, a Logger obtained from a LoggerProvider emits a LogRecord like this:

logger.logRecordBuilder()
    .setBody("something bad happened")
    .setSeverity(Severity.ERROR)
    .setAttribute(AttributeKey.stringKey("myKey"), "myvalue")
    .emit();

That’s the entire surface area: body, severity, attributes, emit(). No spans, no separate metric-shaped API — a LogRecord is a flat structured record, which is exactly why the book calls out that logs and events share a data type and only differ in who’s meant to read them.

The Two Integration Paths

The book presents two concrete approaches for wiring OTel into existing logging:

Appenders/handlers — Logback, Log4j, and JBoss Logging have OTel appenders that route log events through the configured LoggerProvider. This is the “full” path: logs become first-class OTLP citizens, exported alongside traces and metrics, with full correlation.

Context injectors — the lighter path: inject the SpanContext into the framework’s existing context (MDC in Java, Winston hooks in Node, log record factories in Python). Your log format gains trace_id, span_id, and trace_flags fields with zero logging config changes. The logs stay in your existing pipeline, but now they’re joinable with traces.

The Java agent makes the choice easy: otel.logs.exporter defaults to none (no OTLP log export — deliberate conservatism), with logging and otlp available. The appender-and-MDC combination gives you the best of both: enriched local logs for operators, exported logs for correlation.

Closing the Loop on the dropwizard-example

Part 4 and Part 6 both ran the book’s dropwizard-example — and its logback-appender and logback-mdc instrumentation were running the whole time, since every instrumentation package ships enabled by default (Part 4). But its logs never showed a trace_id or span_id, for a mundane reason: the example’s log formatter simply didn’t output MDC values. That’s a one-line fix in example.yml, Dropwizard’s own bootstrap config, not an OTel setting:

logging:
  appenders:
    - type: console
      logFormat: |-
        %-6level [%d{HH:mm:ss.SSS}] [%t] %logger{5} - %X{code} %msg trace_id=%X{trace_id} span_id=%X{span_id} trace_flags=%X{trace_flags}%n

With the Collector stack from Part 4 already running — extended with a logs pipeline batching to console, since Part 4’s own walkthrough only wired up traces and metrics — restarting the app with OTLP log export turned on (still none by default, so it needs an explicit flag) is one extra system property on the same command from Part 4:

# Change directory to Dropwizard-example
cd dropwizard-example

# Start application
java -javaagent:opentelemetry-javaagent.jar \
  -Dotel.service.name=dropwizard-example \
  -Dotel.logs.exporter=otlp \
  -jar target/dropwizard-example-2.1.1.jar \
  server example.yml

Startup logs still show no trace_id — there’s no active span context before a request arrives, which is exactly the “operations outside user transactions” case from earlier in this chapter. Hitting http://localhost:8080/hello-world/date?date=2023-01-15 produces a log line with both IDs now populated, correlatable to the exact trace that produced it:

INFO [18:09:00.888] [dw-65 - GET /hello-world/date?date=2023-01-15] c.e.h.r.HelloWorldResource - Received a date: 2023-01-15 trace_id=6e62f4d5927df8bb790ad7990d9af516 span_id=2409065755502df3 trace_flags=01

And on the Collector side, the same record arrives as a structured LogRecord, trace and span IDs intact:

2023-01-16T16:09:01.031Z info ResourceLog #0
Resource SchemaURL: https://opentelemetry.io/schemas/1.16.0
Resource attributes:
     -> host.arch: STRING(x86_64)
     ...
     -> telemetry.sdk.version: STRING(1.21.0)
ScopeLogs #0
ScopeLogs SchemaURL:
InstrumentationScope com.example.helloworld.resources.HelloWorldResource
LogRecord #0
ObservedTimestamp: 1970-01-01 00:00:00 +0000 UTC
Timestamp: 2023-01-16 16:09:00.888 +0000 UTC
Severity: INFO
Body: Received a date: 2023-01-15
Trace ID: 6e62f4d5927df8bb790ad7990d9af516
Span ID: 2409065755502df3
Flags: 1

That’s the whole loop closed: one config line in example.yml, one system property on startup, and a log line that used to be an isolated string is now a LogRecord joinable to its trace by ID — no changes to the Dropwizard resource code that logged it in the first place.

The SDK: Processors, Batching, and Limits

The Logging SDK mirrors the tracing SDK’s shape: SdkLoggerProvider with a Resource and a set of LogRecordProcessors. Registering an empty one — equivalent to calling .build() with no processors — looks like this:

// Empty provider, equivalent to calling build() directly
SdkLoggerProvider loggerProvider = SdkLoggerProvider.builder()
    .setResource(Resource.getDefault())
    .setClock(Clock.getDefault())
    .setLogLimits(LogLimits::getDefault)
    .build();

OpenTelemetry openTelemetry = OpenTelemetrySdk.builder()
    .setLoggerProvider(loggerProvider)
    .buildAndRegisterGlobal();

The same two processors from tracing appear here, unchanged in shape:

• SimpleLogRecordProcessor — exports immediately; not for production (logging throughput will overwhelm it).
• BatchLogRecordProcessor — queues and batches; drops logs when the queue fills; default export delay of just 200ms (tighter than span batching, since log latency tolerance is lower). Pass it a MeterProvider and it emits queue-size and processed-event-count metrics — useful for tuning the queue before it starts dropping.

A realistic setup wires both at once — a batch processor exporting to a real backend over OTLP, and a simple processor writing to stdout for local visibility, using the same standalone-Java pattern from Part 4:

SdkLoggerProvider loggerProvider = SdkLoggerProvider
    .builder()
    .setResource(resource)
    .addLogRecordProcessor(BatchLogRecordProcessor
        .builder(OtlpGrpcLogRecordExporter
            .builder()
            .setEndpoint("http://otel-collector:4317")
            .build())
        .setMeterProvider(meterProvider)
        .setMaxQueueSize(10240)
        .build())
    .addLogRecordProcessor(SimpleLogRecordProcessor
        .create(SystemOutLogRecordExporter.create()))
    .build();

Note the stdout exporter is SystemOutLogRecordExporter, not java.util.logging — a deliberate choice to avoid a logging loop if a java.util.logging appender happens to be configured too.

Attribute limits apply globally across spans, span events, span links, and logs — otel.attribute.value.length.limit and otel.attribute.count.limit — the same guard against runaway cardinality and payload size.

What Not to Do With Logs

The chapter’s “don’ts” are as valuable as its “dos”:

• Don’t use logs as a metrics substitute. Counting ERROR lines to compute an error rate is strictly worse than emitting a real counter: metrics aggregate at the source, cutting transfer, storage, and query cost while maximizing signal stability. Anything computable into an aggregate at the producer level should be a metric.
• Don’t store unsampled audit logs in observability platforms. Audit data needs completeness, tolerates delay, and needs no cross-signal correlation — the opposite of observability data. Keep them in cheaper storage or reliable-event pipelines, and let the OTel Collector route each class of data to the right destination.
• Don’t wait for the Logs signal to stabilize before adopting. The data model is already stable; the appender/MDC integration paths are production-ready; and the correlation value (logs linked to traces) doesn’t depend on the API’s stability status.

The chapter’s quiet theme: logs are the signal OTel inherits, not the one it invents. The design treats that as a feature — every existing line of logging code becomes an asset you can correlate, rather than a liability you’d need to migrate.

Next: Practical OpenTelemetry — Part 9: The Collector and OTLP — the component that ties everything together.

References

• Practical OpenTelemetry (Apress, 2023) — Daniel Gomez Blanco
• OpenTelemetry Logs Specification
• Twelve-Factor App: Logs