Chapter 08

Sidecar / OTel integration

Reflects Holonograph v0.23.0 · ~5 min read

The lens sees every call that goes through it. Real systems have work that doesn't go through it — a job on a queue, a background worker, a webhook fanning out to three services. This chapter is about carrying the two things the lens needs (run mode, correlation id) across those boundaries via OpenTelemetry, so the work on the other side rejoins the same evaluation.

What it is

Holonograph composes on top of OpenTelemetry rather than replacing it. Two pieces of the OTel spec do the work:

• Baggage carries key-value context alongside a trace. Holonograph reads and writes two Baggage keys (holonograph.runmode, holonograph.correlation-id) so the run mode and correlation id survive async boundaries automatically.
• Auto-instrumentation covers your framework's normal request/response cycle, so HTTP calls, queue publishes, and pub-sub messages carry the OTel context without you writing propagation code by hand.

Where auto-instrumentation doesn't reach, there's a manual fallback: encode the same values as message attributes on the payload and reconstruct them on the receiving side.

Why it matters

The four-source attribution downstream only works if the events on both sides of an async hop can be tied back to the same call. If the queue drops your run mode, the worker's call defaults to whatever the worker's client was constructed with — often the wrong thing. Baggage keeps that from happening quietly.

How to use it

Enable OTel in your service

Use any OTel SDK for your language. Enable auto-instrumentation for the transports you use (HTTP, gRPC, your queue library, your pub-sub client). Nothing Holonograph-specific here — the standard OTel setup is enough.

Set Baggage on the originating side

At the top of the request that started the work, put the run mode and correlation id into Baggage:

import { propagation, context } from "@opentelemetry/api";

const baggage = propagation.getBaggage(context.active())
  ?? propagation.createBaggage();

const withHolonograph = baggage
  .setEntry("holonograph.runmode",       { value: "production" })
  .setEntry("holonograph.correlation-id",{ value: correlationId });

const ctx = propagation.setBaggage(context.active(), withHolonograph);

// run the rest of the request under `ctx`
context.with(ctx, async () => {
  await enqueueBillingReview({ userId });
});

Auto-instrumentation on the queue publish serializes Baggage into the message; no message-body changes required.

Read Baggage on the receiving side

The worker or subscriber picks up the Baggage after auto-instrumentation extracts it. Use it to construct the HolonographClient:

import { propagation, context } from "@opentelemetry/api";

async function handleJob(job) {
  const baggage = propagation.getBaggage(context.active());
  const runMode = baggage?.getEntry("holonograph.runmode")?.value ?? "local_dev";
  const correlationId = baggage?.getEntry("holonograph.correlation-id")?.value;

  const client = new HolonographClient({
    endpoint: "http://127.0.0.1:8080",
    runMode,
    lensVersion: "billing-lens/v14",
    substrate: { /* ... */ }
  });

  const handle = await client.messages.create({
    surfaceId: "review-flagged-charge",
    messages: [ /* ... */ ],
    correlationId
  });
  // ...
}

The call in the worker now shares a correlationId with the request that produced the job, and it inherits the originator's run mode — so a test request that enqueued a job doesn't turn into a production call on the worker.

Message-attribute fallback

For transports where Baggage propagation isn't automatic — some managed queues, some pub-sub platforms — encode the two values as explicit message attributes on the payload, and reconstruct them on the receiving side before setting Baggage manually.

await queue.publish({
  body: { userId },
  attributes: {
    "holonograph.runmode": "production",
    "holonograph.correlation-id": correlationId
  }
});

On the receiver, read the attributes and set them into Baggage the same way you would in a fully-instrumented setup. The rest of your code doesn't need to know which path they took.

Reading the output

Nothing changes at read time. Events written from the worker come back through GET /lens/events with the same correlationId as the originating request, and their run_mode column matches the originator's — which is what makes them queryable as one evaluation across the async boundary.

Reference

Baggage keys

Message-attribute keys

Same keys as above, used verbatim as message attributes when Baggage propagation isn't available on the transport.