npm.io
4.2.2 • Published 6d ago

autotel

Licence
MIT
Version
4.2.2
Deps
13
Size
4.2 MB
Vulns
0
Weekly
0
SummaryDependencyVersions

autotel

npm version npm subscribers License: MIT

Write once, observe everywhere. Instrument your Node.js code a single time, keep the DX you love, and stream traces, metrics, logs, and product events to any observability stack without vendor lock-in.

  • Drop-in DX : one init() and ergonomic helpers like trace(), span(), withTracing(), decorators, and batch instrumentation.
  • Platform freedom : OTLP-first design plus subscribers for PostHog, Mixpanel, Amplitude, and anything else via custom exporters/readers.
  • Production hardening : adaptive sampling (10% baseline, 100% errors/slow paths), rate limiting, circuit breakers, payload validation, and automatic sensitive-field redaction.
  • Auto enrichment : service metadata, deployment info, and AsyncLocalStorage-powered correlation IDs automatically flow into spans, metrics, logs, and events.

Raw OpenTelemetry is verbose, and vendor SDKs create lock-in. Autotel gives you the best parts of both: clean ergonomics and total ownership of your telemetry.

Migrating from OpenTelemetry?

Migration Guide - Pattern-by-pattern migration walkthrough with side-by-side comparisons and deployment checklist.

Replace NODE_OPTIONS and 30+ lines of SDK boilerplate with init(), wrap functions with trace() instead of manual span.start()/span.end().

Table of Contents

Why Autotel

Challenge With autotel
Writing raw OpenTelemetry spans/metrics takes dozens of lines and manual lifecycle management. Wrap any function in trace() or span() and get automatic span lifecycle, error capture, attributes, and adaptive sampling.
Vendor SDKs simplify setup but trap your data in a single platform. Autotel is OTLP-native and works with Grafana Cloud, Datadog, New Relic, Tempo, Honeycomb, Elasticsearch, or your own collector.
Teams need both observability and product events. Ship technical telemetry and funnel/behavior events through the same API with contextual enrichment.
Production readiness requires redaction, rate limiting, and circuit breakers. Those guardrails are on by default so you can safely enable telemetry everywhere.

Entry point map

autotel has ~35 subpath exports. For most scenarios you only need one or two. Cloudflare Workers and other edge runtimes should use autotel-cloudflare or autotel-edge, not autotel (which expects Node).

Scenario Package Import
Node service — init + spans autotel import { init, trace, span } from 'autotel'
Node service — pino-style logger with trace context autotel import { createLogger } from 'autotel/logger'
Node service — class decorators autotel import { Trace, Span } from 'autotel/decorators'
Node service — Drizzle ORM spans autotel-drizzle import { instrumentDrizzleClient } from 'autotel-drizzle'
Node service — testing assertions autotel import { createTraceCollector } from 'autotel/testing'
Cloudflare Worker — fetch with spans autotel-cloudflare import { wrapModule, trace } from 'autotel-cloudflare'
Cloudflare Worker — logs only, no nodejs_compat autotel-cloudflare import { createEdgeLogger } from 'autotel-cloudflare/logger'
Cloudflare Worker — queue consumer autotel-cloudflare import { wrapModule, getQueueLogger } from 'autotel-cloudflare'
Cloudflare Worker — Durable Object autotel-cloudflare import { wrapDurableObject } from 'autotel-cloudflare'
Hono on Workers autotel-cloudflare + autotel-hono wrapModule from autotel-cloudflare, middleware from autotel-hono
Vercel Edge / Netlify Edge / Deno Deploy autotel-edge import { init, trace } from 'autotel-edge'
Edge runtime — logs only autotel-edge import { createEdgeLogger } from 'autotel-edge/logger'

Quick Start

Want to follow along in code? This repo ships with apps/example-basic (mirrors the steps below) and apps/example-http for an Express server, you can run either with pnpm start after pnpm install && pnpm build at the root.

1. Install
npm install autotel
npm install -D autotel-devtools # optional but recommended for local DX
# or
pnpm add autotel
pnpm add -D autotel-devtools
2. Initialize once at startup
import { init } from 'autotel';

init({
  service: 'checkout-api',
  devtools: true,
});

Defaults:

  • OTLP endpoint: process.env.OTLP_ENDPOINT || http://localhost:4318
  • Metrics: on in every environment
  • Sampler: adaptive (10% baseline, 100% for errors/slow spans)
  • Version: auto-detected from package.json
  • Events auto-flush when the root span finishes

Recommended local workflow:

init({
  service: 'checkout-api',
  devtools: true,
});
  • devtools: true points traces, metrics, and logs at local autotel-devtools
  • devtools: { embedded: true } tries to start autotel-devtools for you
  • when you switch to a hosted backend, replace devtools with endpoint and optional headers

Example remote backend config:

init({
  service: 'checkout-api',
  endpoint: process.env.OTEL_EXPORTER_OTLP_ENDPOINT,
  headers: process.env.OTEL_EXPORTER_OTLP_HEADERS,
});

Sampling presets:

  • Simple path: sampling: 'development' | 'errors-only' | 'production' | 'off'
  • Advanced path: samplingPresets.development(), samplingPresets.errorsOnly(), samplingPresets.production({...}), samplingPresets.off()
  • Precedence is always sampler > sampling > default
  • If you use YAML sampling.preset, extra tuning fields in that same block are ignored. Use the programmatic API with sampler or samplingPresets.production({...}) when you need overrides.
  • Tail-sampling hint attributes use the autotel.* namespace, for example autotel.sampling.tail.keep. This is intentional: OpenTelemetry does not define an official semantic-convention key for these internal hints, so autotel uses library-prefixed custom attributes rather than inventing fake otel.* semconv keys.
3. Instrument code with trace()
import { trace } from 'autotel';

export const createUser = trace(async function createUser(
  data: CreateUserData,
) {
  const user = await db.users.insert(data);
  return user;
});
  • Named function expressions automatically become span names (code.function).
  • Errors are recorded, spans are ended, and status is set automatically.
4. See the value everywhere
import { init, track } from 'autotel';

init({
  service: 'checkout-api',
  endpoint: 'https://otlp-gateway-prod.grafana.net/otlp',
  subscribers: [new PostHogSubscriber({ apiKey: process.env.POSTHOG_KEY! })],
});

export const processOrder = trace(async function processOrder(order) {
  track('order.completed', { amount: order.total });
  return charge(order);
});

Every span, metric, log line, and event includes traceId, spanId, operation.name, service.version, and deployment.environment automatically.

Choose Any Destination

import { init } from 'autotel';

init({
  service: 'my-app',
  // Grafana / Tempo / OTLP collector
  endpoint: 'https://otlp-gateway-prod.grafana.net/otlp',
});

init({
  service: 'my-app',
  // Datadog (traces + metrics + logs via OTLP)
  endpoint: 'https://otlp.datadoghq.com',
  headers: 'dd-api-key=...',
});

init({
  service: 'my-app',
  // Honeycomb (gRPC protocol)
  protocol: 'grpc',
  endpoint: 'api.honeycomb.io:443',
  headers: {
    'x-honeycomb-team': process.env.HONEYCOMB_API_KEY!,
  },
});

init({
  service: 'my-app',
  // Multi-backend OTLP fan-out without manual exporter wiring
  logs: true,
  destinations: [
    {
      endpoint: 'https://otlp-gateway-prod.grafana.net/otlp',
      headers: 'Authorization=Basic ...',
    },
    {
      endpoint: 'https://api.honeycomb.io',
      headers: {
        'x-honeycomb-team': process.env.HONEYCOMB_API_KEY!,
      },
      signals: ['traces'],
    },
  ],
});

init({
  service: 'my-app',
  // Custom pipeline with your own exporters/readers
  spanProcessor: new BatchSpanProcessor(
    new JaegerExporter({ endpoint: 'http://otel:14268/api/traces' }),
  ),
  metricReader: new PeriodicExportingMetricReader({
    exporter: new OTLPMetricExporter({
      url: 'https://metrics.example.com/v1/metrics',
    }),
  }),
  logRecordProcessors: [
    new BatchLogRecordProcessor(
      new OTLPLogExporter({ url: 'https://logs.example.com/v1/logs' }),
    ),
  ],
  instrumentations: [new HttpInstrumentation()],
});

init({
  service: 'my-app',
  // Product events subscribers (ship alongside OTLP)
  subscribers: [
    new PostHogSubscriber({ apiKey: process.env.POSTHOG_KEY! }),
    new MixpanelSubscriber({ projectToken: process.env.MIXPANEL_TOKEN! }),
  ],
});

init({
  service: 'my-app',
  // OpenLLMetry integration for LLM observability
  openllmetry: {
    enabled: true,
    options: {
      disableBatch: process.env.NODE_ENV !== 'production',
      apiKey: process.env.TRACELOOP_API_KEY,
    },
  },
});

Autotel never owns your data, it's a thin layer over OpenTelemetry with optional adapters.

LLM Observability with OpenLLMetry

Autotel integrates seamlessly with OpenLLMetry to provide comprehensive observability for LLM applications. OpenLLMetry automatically instruments LLM providers (OpenAI, Anthropic, etc.), vector databases, and frameworks (LangChain, LlamaIndex, etc.).

Installation

Install the OpenLLMetry SDK as an optional peer dependency:

pnpm add @traceloop/node-server-sdk
# or
npm install @traceloop/node-server-sdk
Usage

Enable OpenLLMetry in your autotel configuration:

import { init } from 'autotel';

init({
  service: 'my-llm-app',
  endpoint: process.env.OTLP_ENDPOINT,
  openllmetry: {
    enabled: true,
    options: {
      // Disable batching in development for immediate traces
      disableBatch: process.env.NODE_ENV !== 'production',
      // Optional: Traceloop API key if using Traceloop backend
      apiKey: process.env.TRACELOOP_API_KEY,
    },
  },
});

OpenLLMetry will automatically:

  • Instrument LLM calls (OpenAI, Anthropic, Cohere, etc.)
  • Track vector database operations (Pinecone, Chroma, Qdrant, etc.)
  • Monitor LLM frameworks (LangChain, LlamaIndex, LangGraph, etc.)
  • Reuse autotel's OpenTelemetry tracer provider for unified traces

All LLM spans will appear alongside your application traces in your observability backend.

AI Workflow Patterns: See AI/LLM Workflow Documentation for comprehensive patterns including:

  • Multi-agent workflows (orchestration and handoffs)
  • RAG pipelines (embeddings, search, generation)
  • Streaming responses
  • Evaluation loops
  • Working examples in apps/example-ai-agent

Sampling

Autotel defaults to production-ready adaptive sampling: a 10% baseline, with errors and slow requests kept automatically.

Preset Shorthand

Use the sampling field on init() when you want the shortest path:

import { init } from 'autotel';

init({
  service: 'checkout-api',
  sampling: 'production',
});

Available string presets:

  • 'development': keep everything
  • 'errors-only': drop healthy baseline traffic, keep errors
  • 'production': 10% baseline plus errors and slow traces
  • 'off': disable sampling entirely

String presets intentionally use kebab-case. For example, the string form is sampling: 'errors-only'.

Tuned Presets

Use samplingPresets when you want preset behavior with tuned thresholds or rates:

import { init, samplingPresets } from 'autotel';

init({
  service: 'checkout-api',
  sampler: samplingPresets.production({
    baselineSampleRate: 0.05,
    slowThresholdMs: 500,
  }),
});

Factory names intentionally use JavaScript-style camelCase. For example, the factory form is samplingPresets.errorsOnly().

YAML Configuration

Use sampling.preset for the simple YAML path:

sampling:
  preset: production

If you need tuned sampling in YAML today, prefer the explicit sampler config block:

sampling:
  type: adaptive
  baseline_rate: 0.05
  always_sample_errors: true
  always_sample_slow: true
  slow_threshold_ms: 500

When sampling.preset is set, other keys in the same YAML sampling block are ignored and autotel will warn. Use the programmatic API with sampler or samplingPresets.production({...}) for tuned presets.

Precedence Rules

Sampling always resolves in this order:

sampler > sampling > default

That means:

  • sampler always wins if you provide both
  • sampling is the simple preset shorthand
  • OpenTelemetry env vars such as OTEL_TRACES_SAMPLER are used after explicit config and YAML
  • default behavior is samplingPresets.production()

Example:

import { init, NeverSampler } from 'autotel';

init({
  service: 'checkout-api',
  sampler: new NeverSampler(),
  sampling: 'development', // ignored because sampler wins
});

For OpenTelemetry SDK compatibility, autotel also reads OTEL_TRACES_SAMPLER and OTEL_TRACES_SAMPLER_ARG.

Supported values:

  • always_on
  • always_off
  • traceidratio
  • parentbased_always_on
  • parentbased_always_off
  • parentbased_traceidratio

Currently unsupported and ignored with an error log:

  • jaeger_remote
  • parentbased_jaeger_remote
  • xray
Tail-Sampling Attributes

Autotel uses internal span attributes such as autotel.sampling.tail.keep and autotel.sampling.tail.evaluated to communicate tail-sampling decisions.

These use the autotel.* namespace intentionally. OpenTelemetry does not define an official semantic-convention key for these internal hints, so autotel uses library-prefixed custom attributes rather than inventing fake otel.* semantic convention keys.

Core Building Blocks

trace()

Wrap any sync/async function to create spans automatically.

import { trace } from 'autotel';

export const updateUser = trace(async function updateUser(
  id: string,
  data: UserInput,
) {
  return db.users.update(id, data);
});

// Explicit name (useful for anonymous/arrow functions)
export const deleteUser = trace('user.delete', async (id: string) => {
  return db.users.delete(id);
});

// Factory form exposes the `ctx` helper (see below)
export const createOrder = trace((ctx) => async (order: Order) => {
  ctx.setAttribute('order.id', order.id);
  return submit(order);
});

// Immediate execution - wraps and executes instantly (for middleware/wrappers)
function timed<T>(operation: string, fn: () => Promise<T>): Promise<T> {
  return trace(operation, async (ctx) => {
    ctx.setAttribute('operation', operation);
    return await fn();
  });
}
// Executes immediately, returns Promise<T> directly

Two patterns supported:

  1. Factory pattern trace(ctx => (...args) => result) : Returns a wrapped function for reuse
  2. Immediate execution trace(ctx => result) : Executes once immediately, returns the result directly
  • Automatic span lifecycle (start, end, status, and error recording).
  • Function names feed operation.name, code.function, and events enrichment.
  • Works with promises, async/await, or sync functions.
span()

Create nested spans for individual code blocks without wrapping entire functions.

import { span, trace } from 'autotel';

export const rollDice = trace(async function rollDice(rolls: number) {
  const results: number[] = [];

  for (let i = 0; i < rolls; i++) {
    await span(
      { name: 'roll.once', attributes: { roll: i + 1 } },
      async (span) => {
        span.setAttribute('range', '1-6');
        const value = rollOnce();
        span.setAttribute('dice.value', value);
        results.push(value);
      },
    );
  }

  return results;
});

Nested spans automatically inherit context and correlation IDs.

Trace Context (ctx)

Every trace((ctx) => ...) factory receives a type-safe helper backed by AsyncLocalStorage.

import { trace, recordStructuredError } from 'autotel';
import { SpanStatusCode } from '@opentelemetry/api';

export const createUser = trace((ctx) => async (input: CreateUserData) => {
  logger.info({ traceId: ctx.traceId }, 'Handling request');
  ctx.setAttributes({ 'user.id': input.id, 'user.plan': input.plan });

  try {
    const user = await db.users.create(input);
    ctx.setStatus({ code: SpanStatusCode.OK });
    return user;
  } catch (error) {
    // Records the error on the span and sets ERROR status. Pair with
    // `getRequestLogger(ctx).error(error)` if you also want a correlated log.
    recordStructuredError(ctx, error as Error);
    throw error;
  }
});

Available helpers: traceId, spanId, correlationId, setAttribute, setAttributes, setStatus, getBaggage, setBaggage, deleteBaggage, getAllBaggage.

Errors and events: prefer recordStructuredError(ctx, error) and the request logger over the raw Span.recordException / Span.addEvent APIs. See MIGRATION.md for the OTel Span Event deprecation direction.

Baggage (Context Propagation)

Baggage allows you to propagate custom key-value pairs across distributed traces. Baggage is automatically included in HTTP headers when using injectTraceContext() from autotel/http.

import { trace, withBaggage } from 'autotel';
import { injectTraceContext } from 'autotel/http';

// Set baggage for downstream services
export const createOrder = trace((ctx) => async (order: Order) => {
  return await withBaggage({
    baggage: {
      'tenant.id': order.tenantId,
      'user.id': order.userId,
    },
    fn: async () => {
      // Baggage is available to all child spans and HTTP calls
      const tenantId = ctx.getBaggage('tenant.id');
      ctx.setAttribute('tenant.id', tenantId || 'unknown');

      // HTTP headers automatically include baggage
      const headers = injectTraceContext();
      await fetch('/api/charge', { headers, body: JSON.stringify(order) });
    },
  });
});

Typed Baggage (Optional):

For type-safe baggage operations, use defineBaggageSchema():

import { trace, defineBaggageSchema } from 'autotel';

type TenantBaggage = { tenantId: string; region?: string };
const tenantBaggage = defineBaggageSchema<TenantBaggage>('tenant');

export const handler = trace<TenantBaggage>((ctx) => async () => {
  // Type-safe get
  const tenant = tenantBaggage.get(ctx);
  if (tenant?.tenantId) {
    console.log('Tenant:', tenant.tenantId);
  }

  // Type-safe set with proper scoping
  return await tenantBaggage.with(ctx, { tenantId: 't1' }, async () => {
    // Baggage is available here and in child spans
  });
});

Automatic Baggage → Span Attributes:

Enable baggage: true in init() to automatically copy all baggage entries to span attributes, making them visible in trace UIs without manual ctx.setAttribute() calls:

import { init, trace, withBaggage } from 'autotel';

init({
  service: 'my-app',
  baggage: true, // Auto-copy baggage to span attributes
});

export const processOrder = trace((ctx) => async (order: Order) => {
  return await withBaggage({
    baggage: {
      'tenant.id': order.tenantId,
      'user.id': order.userId,
    },
    fn: async () => {
      // Span automatically has baggage.tenant.id and baggage.user.id attributes!
      // No need for: ctx.setAttribute('tenant.id', ctx.getBaggage('tenant.id'))
      await chargeCustomer(order);
    },
  });
});

Custom prefix:

init({
  service: 'my-app',
  baggage: 'ctx', // Creates ctx.tenant.id, ctx.user.id
  // Or use '' for no prefix: tenant.id, user.id
});

Extracting Baggage from Incoming Requests:

import { extractTraceContext, trace, context } from 'autotel';

// In Express middleware
app.use((req, res, next) => {
  const extractedContext = extractTraceContext(req.headers);
  context.with(extractedContext, () => {
    next();
  });
});

Key Points:

  • Typed baggage is completely optional - existing untyped baggage code continues to work without changes
  • baggage: true in init() eliminates manual attribute setting for baggage
  • Baggage values are strings (convert numbers/objects before setting)
  • Never put PII in baggage - it propagates in HTTP headers across services!
Reusable Middleware Helpers
  • withTracing(options) : create a preconfigured wrapper (service name, default attributes, skip rules).
  • instrument(object, options) : batch-wrap entire modules while skipping helpers or private functions.
import { withTracing, instrument } from 'autotel';

const traceFn = withTracing({ serviceName: 'user' });

export const create = traceFn((ctx) => async (payload) => {
  /* ... */
});
export const update = traceFn((ctx) => async (id, payload) => {
  /* ... */
});

export const repository = instrument(
  {
    createUser: async () => {
      /* ... */
    },
    updateUser: async () => {
      /* ... */
    },
    _internal: async () => {
      /* skipped */
    },
  },
  { serviceName: 'repository', skip: ['_internal'] },
);
Decorators (TypeScript 5+)

Prefer classes or NestJS-style services? Use the @Trace decorator.

import { Trace } from 'autotel/decorators';

class OrderService {
  @Trace('order.create', { withMetrics: true })
  async createOrder(data: OrderInput) {
    return db.orders.create(data);
  }

  // No arguments → method name becomes the span name
  @Trace()
  async processPayment(orderId: string) {
    return charge(orderId);
  }

  @Trace()
  async refund(orderId: string) {
    const ctx = (this as any).ctx;
    ctx.setAttribute('order.id', orderId);
    return refund(orderId);
  }
}

Decorators are optional, everything also works in plain functions.

Database Instrumentation

Turn on query tracing in one line.

import { instrumentDatabase } from 'autotel/db';

const db = drizzle(pool);

instrumentDatabase(db, {
  dbSystem: 'postgresql',
  database: 'myapp',
});

await db.select().from(users); // queries emit spans automatically

Type-Safe Attributes

Autotel provides type-safe attribute builders following OpenTelemetry semantic conventions. These helpers give you autocomplete, compile-time validation, and automatic PII redaction.

Pattern A: Key Builders

Build individual attributes with full autocomplete:

import { attrs, mergeAttrs } from 'autotel/attributes';

// Single attribute
ctx.setAttributes(attrs.user.id('user-123'));
// → { 'user.id': 'user-123' }

ctx.setAttributes(attrs.http.request.method('GET'));
// → { 'http.request.method': 'GET' }

ctx.setAttributes(attrs.db.client.system('postgresql'));
// → { 'db.system.name': 'postgresql' }

// Combine multiple attributes
ctx.setAttributes(
  mergeAttrs(
    attrs.user.id('user-123'),
    attrs.session.id('sess-456'),
    attrs.http.response.statusCode(200),
  ),
);
Pattern B: Object Builders

Pass an object to set multiple related attributes at once:

import { attrs } from 'autotel/attributes';

// User attributes
ctx.setAttributes(
  attrs.user.data({
    id: 'user-123',
    email: 'user@example.com',
    roles: ['admin', 'editor'],
  }),
);
// → { 'user.id': 'user-123', 'user.email': 'user@example.com', 'user.roles': ['admin', 'editor'] }

// HTTP server attributes
ctx.setAttributes(
  attrs.http.server({
    method: 'POST',
    route: '/api/users/:id',
    statusCode: 201,
  }),
);
// → { 'http.request.method': 'POST', 'http.route': '/api/users/:id', 'http.response.status_code': 201 }

// Database attributes
ctx.setAttributes(
  attrs.db.client.data({
    system: 'postgresql',
    name: 'myapp_db', // Maps to db.namespace
    operation: 'SELECT',
    collectionName: 'users',
  }),
);
Attachers (Signal Helpers)

Attachers know WHERE to attach attributes - they handle spans, resources, and apply guardrails automatically:

import { setUser, httpServer, identify, dbClient } from 'autotel/attributes';

// Set user attributes with automatic PII redaction
export const handleRequest = trace((ctx) => async (req) => {
  setUser(ctx, {
    id: req.userId,
    email: req.userEmail, // Automatically redacted by default
  });

  // HTTP attributes + automatic span name update
  httpServer(ctx, {
    method: req.method,
    route: req.route,
    statusCode: 200,
  });
  // Span name becomes: "HTTP GET /api/users"
});

// Bundle user, session, and device attributes together
export const identifyUser = trace((ctx) => async (data) => {
  identify(ctx, {
    user: { id: data.userId, name: data.userName },
    session: { id: data.sessionId },
    device: { id: data.deviceId, manufacturer: 'Apple' },
  });
});

// Database client attributes
export const queryUsers = trace((ctx) => async () => {
  dbClient(ctx, {
    system: 'postgresql',
    operation: 'SELECT',
    collectionName: 'users',
  });
  return await db.query('SELECT * FROM users');
});
PII Guardrails

safeSetAttributes() applies automatic PII detection and configurable guardrails:

import { safeSetAttributes, attrs } from 'autotel/attributes';

export const processUser = trace((ctx) => async (user) => {
  // Default: PII is redacted automatically
  safeSetAttributes(ctx, attrs.user.data({ email: 'user@example.com' }));
  // → { 'user.email': '[REDACTED]' }

  // Allow PII (use with caution)
  safeSetAttributes(ctx, attrs.user.data({ email: 'user@example.com' }), {
    guardrails: { pii: 'allow' },
  });
  // → { 'user.email': 'user@example.com' }

  // Hash PII for correlation without exposing raw values
  safeSetAttributes(ctx, attrs.user.data({ email: 'user@example.com' }), {
    guardrails: { pii: 'hash' },
  });
  // → { 'user.email': 'hash_a1b2c3d4...' }

  // Truncate long values
  safeSetAttributes(ctx, attrs.user.data({ id: 'a'.repeat(500) }), {
    guardrails: { maxLength: 255 },
  });
  // → { 'user.id': 'aaaa...aaa...' } (truncated with ellipsis)

  // Warn on deprecated attributes
  safeSetAttributes(
    ctx,
    { 'http.method': 'GET' }, // Deprecated!
    { guardrails: { warnDeprecated: true } },
  );
  // Console: [autotel/attributes] Attribute "http.method" is deprecated. Use "http.request.method" instead.
});

Guardrail Options:

Option Values Default Description
pii 'allow', 'redact', 'hash', 'block' 'redact' How to handle PII in attribute values
maxLength number 255 Maximum string length before truncation
validateEnum boolean true Normalize enum values (e.g., HTTP methods)
warnDeprecated boolean true Log warnings for deprecated attributes
Domain Helpers

Domain helpers bundle multiple attribute groups for common scenarios:

import { transaction } from 'autotel/attributes';

// Bundle HTTP request with user context
export const handleRequest = trace((ctx) => async (req) => {
  transaction(ctx, {
    user: { id: req.userId },
    session: { id: req.sessionId },
    method: req.method,
    route: req.route,
    statusCode: 200,
    clientIp: req.ip,
  });
  // Sets: user.id, session.id, http.request.method, http.route,
  //       http.response.status_code, network.peer.address
  // Also updates span name to "HTTP GET /api/users"
});
Available Attribute Domains
Domain Key Builders Object Builder
user id, email, name, fullName, hash, roles attrs.user.data()
session id, previousId attrs.session.data()
device id, manufacturer, modelIdentifier, modelName attrs.device.data()
http request.*, response.*, route attrs.http.server(), attrs.http.client()
db client.system, client.operation, etc. attrs.db.client.data()
service name, instance, version attrs.service.data()
network peerAddress, peerPort, transport, etc. attrs.network.data()
error type, message, stackTrace, code attrs.error.data()
exception escaped, message, stackTrace, type attrs.exception.data()
cloud provider, accountId, region, etc. attrs.cloud.data()
messaging system, destination, operation, etc. attrs.messaging.data()
genAI system, requestModel, responseModel, etc. -
rpc system, service, method -
graphql document, operationName, operationType -
Resource Merging

For enriching OpenTelemetry Resources with service attributes (Resource.attributes is readonly), use mergeServiceResource:

import { mergeServiceResource } from 'autotel/attributes';
import { Resource } from '@opentelemetry/resources';

// Create enriched resource for custom SDK configurations
const baseResource = Resource.default();
const enrichedResource = mergeServiceResource(baseResource, {
  name: 'my-service',
  version: '1.0.0',
  instance: 'instance-1',
});

// Use with custom TracerProvider
const provider = new NodeTracerProvider({ resource: enrichedResource });

Event-Driven Architectures

Autotel provides first-class support for tracing message-based systems like Kafka, SQS, and RabbitMQ. The traceProducer and traceConsumer helpers automatically set semantic attributes, handle context propagation, and create proper span links.

Message Producers (Kafka, SQS, RabbitMQ)

Use traceProducer to wrap message publishing functions with automatic tracing:

import { traceProducer, type ProducerContext } from 'autotel';

// Kafka producer
export const publishUserEvent = traceProducer({
  system: 'kafka',
  destination: 'user-events',
  messageIdFrom: (args) => args[0].eventId, // Extract message ID from args
})((ctx) => async (event: UserEvent) => {
  // Get W3C trace headers to inject into message
  const headers = ctx.getTraceHeaders();

  await producer.send({
    topic: 'user-events',
    messages: [
      {
        key: event.userId,
        value: JSON.stringify(event),
        headers, // Trace context propagates to consumers
      },
    ],
  });
});

// SQS producer with custom attributes
export const publishOrder = traceProducer({
  system: 'sqs',
  destination: 'orders-queue',
  attributes: { 'custom.priority': 'high' },
})((ctx) => async (order: Order) => {
  ctx.setAttribute('order.total', order.total);

  await sqs.sendMessage({
    QueueUrl: QUEUE_URL,
    MessageBody: JSON.stringify(order),
    MessageAttributes: {
      traceparent: {
        DataType: 'String',
        StringValue: ctx.getTraceHeaders().traceparent,
      },
    },
  });
});

Automatic Span Attributes (OTel Semantic Conventions):

  • messaging.system - The messaging system (kafka, sqs, rabbitmq, etc.)
  • messaging.operation - Always "publish" for producers
  • messaging.destination.name - Topic/queue name
  • messaging.message.id - Extracted message ID (if configured)
  • messaging.kafka.destination.partition - Partition number (Kafka-specific)
Message Consumers

Use traceConsumer to wrap message handlers with automatic link extraction and DLQ support:

import { traceConsumer, extractLinksFromBatch } from 'autotel';

// Single message consumer
export const processUserEvent = traceConsumer({
  system: 'kafka',
  destination: 'user-events',
  consumerGroup: 'event-processor',
  headersFrom: (msg) => msg.headers, // Extract trace headers
})((ctx) => async (message: KafkaMessage) => {
  // Links to producer span are automatically created
  const event = JSON.parse(message.value);
  await processEvent(event);
});

// Batch consumer with automatic link extraction
export const processBatch = traceConsumer({
  system: 'kafka',
  destination: 'user-events',
  consumerGroup: 'batch-processor',
  batchMode: true, // Extract links from all messages
  headersFrom: (msg) => msg.headers,
})((ctx) => async (messages: KafkaMessage[]) => {
  // ctx.links contains SpanContext from each message's traceparent
  for (const msg of messages) {
    await processMessage(msg);
  }
});

// Consumer with DLQ handling
export const processWithDLQ = traceConsumer({
  system: 'sqs',
  destination: 'orders-queue',
  headersFrom: (msg) => msg.MessageAttributes,
})((ctx) => async (message: SQSMessage) => {
  try {
    await processOrder(JSON.parse(message.Body));
  } catch (error) {
    if (message.ApproximateReceiveCount > 3) {
      // Record DLQ routing
      ctx.recordDLQ('orders-dlq', error.message);
      throw error; // Let SQS move to DLQ
    }
    throw error; // Retry
  }
});

Consumer-Specific Attributes:

  • messaging.consumer.group - Consumer group name
  • messaging.batch.message_count - Batch size (if batch mode)
  • messaging.operation - "receive" or "process"
Consumer Lag Metrics

Track consumer lag for performance monitoring:

import { traceConsumer } from 'autotel';

export const processWithLag = traceConsumer({
  system: 'kafka',
  destination: 'events',
  consumerGroup: 'processor',
  lagMetrics: {
    getCurrentOffset: (msg) => Number(msg.offset),
    getEndOffset: async () => {
      const offsets = await admin.fetchTopicOffsets('events');
      return Number(offsets[0].high);
    },
    partition: 0,
  },
})((ctx) => async (message) => {
  // Lag attributes automatically added:
  // - messaging.kafka.consumer_lag
  // - messaging.kafka.message_offset
  await processMessage(message);
});
Custom Messaging System Adapters

For messaging systems not directly supported (NATS, Temporal, Cloudflare Queues, etc.), use pre-built adapters or create your own:

import { traceConsumer, traceProducer } from 'autotel/messaging';
import {
  natsAdapter,
  temporalAdapter,
  cloudflareQueuesAdapter,
  datadogContextExtractor,
  b3ContextExtractor,
} from 'autotel/messaging/adapters';

// NATS JetStream consumer with automatic attribute extraction
const processNatsMessage = traceConsumer({
  system: 'nats',
  destination: 'orders.created',
  consumerGroup: 'order-processor',
  ...natsAdapter.consumer, // Adds nats.subject, nats.stream, nats.consumer
})((ctx) => async (msg) => {
  await handleOrder(msg.data);
  msg.ack();
});

// Temporal activity with workflow context
const processActivity = traceConsumer({
  system: 'temporal',
  destination: 'order-activities',
  ...temporalAdapter.consumer, // Adds temporal.workflow_id, temporal.run_id, temporal.attempt
})((ctx) => async (info, input) => {
  return processOrder(input);
});

// Consume messages with Datadog trace context (non-W3C format)
const processFromDatadog = traceConsumer({
  system: 'kafka',
  destination: 'events',
  customContextExtractor: datadogContextExtractor, // Converts Datadog decimal IDs to OTel hex
})((ctx) => async (msg) => {
  // Links to parent Datadog span automatically
});

Available Adapters:

Adapter Captures
natsAdapter subject, stream, consumer, pending, redelivery_count
temporalAdapter workflow_id, run_id, activity_id, task_queue, attempt
cloudflareQueuesAdapter message_id, timestamp, attempts
datadogContextExtractor Converts Datadog decimal trace IDs to OTel hex
b3ContextExtractor Parses B3/Zipkin single or multi-header format
xrayContextExtractor Parses AWS X-Ray trace header

Building Custom Adapters:

See Bring Your Own System Guide for step-by-step instructions on creating adapters for any messaging system.

Safe Baggage Propagation

Baggage allows key-value pairs to propagate across service boundaries. Autotel provides safe baggage schemas with built-in guardrails for PII detection, size limits, and high-cardinality value hashing.

BusinessBaggage (Pre-built Schema)

Use the pre-built BusinessBaggage schema for common business context:

import { BusinessBaggage, trace } from 'autotel';

export const processOrder = trace((ctx) => async (order: Order) => {
  // Set business context (propagates to downstream services)
  BusinessBaggage.set(ctx, {
    tenantId: order.tenantId,
    userId: order.userId, // Auto-hashed for privacy
    priority: 'high', // Validated against enum
    correlationId: order.id,
  });

  // Make downstream call - baggage propagates automatically
  await fetch('/api/charge', {
    headers: ctx.getTraceHeaders(), // Includes baggage header
  });
});

// In downstream service
export const chargeOrder = trace((ctx) => async () => {
  // Read business context
  const { tenantId, userId, priority } = BusinessBaggage.get(ctx);

  // Use for routing, logging, access control, etc.
  logger.info({ tenantId, priority }, 'Processing charge');
});

Pre-defined Fields:

  • tenantId - String, max 64 chars
  • userId - String, auto-hashed for privacy
  • correlationId - String, for request correlation
  • workflowId - String, for saga/workflow tracking
  • priority - Enum: 'low', 'normal', 'high', 'critical'
  • region - String, deployment region
  • channel - String (web, mobile, api, etc.)
Custom Baggage Schemas

Create type-safe baggage schemas with validation and guardrails:

import { createSafeBaggageSchema } from 'autotel';

// Define custom schema
const OrderBaggage = createSafeBaggageSchema(
  {
    orderId: { type: 'string', maxLength: 36 },
    customerId: { type: 'string', hash: true }, // Auto-hash for privacy
    tier: { type: 'enum', values: ['free', 'pro', 'enterprise'] as const },
    amount: { type: 'number' },
    isVip: { type: 'boolean' },
  },
  {
    prefix: 'order', // Baggage keys: order.orderId, order.tier, etc.
    maxKeyLength: 64, // Validate key length
    maxValueLength: 256, // Validate value length
    redactPII: true, // Auto-detect and redact PII patterns
    hashHighCardinality: true, // Hash values that look high-cardinality
  },
);

// Use in traced functions
export const processOrder = trace((ctx) => async (order: Order) => {
  // Type-safe set (TypeScript validates fields)
  OrderBaggage.set(ctx, {
    orderId: order.id,
    customerId: order.customerId, // Will be hashed
    tier: order.tier, // Must be 'free' | 'pro' | 'enterprise'
    amount: order.total,
    isVip: order.customer.isVip,
  });

  // Type-safe get
  const { orderId, tier, isVip } = OrderBaggage.get(ctx);

  // Check if specific field is set
  if (OrderBaggage.has(ctx, 'customerId')) {
    // ...
  }

  // Delete specific field
  OrderBaggage.delete(ctx, 'amount');

  // Clear all fields
  OrderBaggage.clear(ctx);
});

Guardrails:

  • Size Limits - Prevents baggage from growing unbounded
  • PII Detection - Auto-redacts email, phone, SSN patterns
  • High-Cardinality Hashing - Hashes UUIDs, timestamps to reduce cardinality
  • Enum Validation - Rejects invalid enum values
  • Type Coercion - Numbers/booleans serialized correctly

Workflow & Saga Tracing

Track distributed workflows and sagas with compensation support. Each step creates a linked span, and failed steps can trigger automatic compensation.

Basic Workflows

Use traceWorkflow and traceStep for multi-step processes:

import { traceWorkflow, traceStep } from 'autotel';

// Define workflow with unique ID
export const processOrder = traceWorkflow({
  name: 'OrderFulfillment',
  workflowId: (order) => order.id, // Generate from first arg
})((ctx) => async (order: Order) => {
  // Step 1: Validate order
  await traceStep({ name: 'ValidateOrder' })((ctx) => async () => {
    await validateOrder(order);
  })();

  // Step 2: Reserve inventory (links to previous step)
  await traceStep({
    name: 'ReserveInventory',
    linkToPrevious: true,
  })((ctx) => async () => {
    await inventoryService.reserve(order.items);
  })();

  // Step 3: Process payment
  await traceStep({
    name: 'ProcessPayment',
    linkToPrevious: true,
  })((ctx) => async () => {
    await paymentService.charge(order);
  })();

  return { success: true };
});

Workflow Attributes:

  • workflow.name - Workflow type name
  • workflow.id - Unique instance ID
  • workflow.version - Optional version
  • workflow.step.name - Current step name
  • workflow.step.index - Step sequence number
  • workflow.step.status - completed, failed, compensated
Saga Pattern with Compensation

Define compensating actions for rollback on failure:

import { traceWorkflow, traceStep } from 'autotel';

export const orderSaga = traceWorkflow({
  name: 'OrderSaga',
  workflowId: (order) => order.id,
})((ctx) => async (order: Order) => {
  // Step 1: Reserve inventory (with compensation)
  await traceStep({
    name: 'ReserveInventory',
    compensate: async (stepCtx, error) => {
      // Called if later step fails
      await inventoryService.release(order.items);
      stepCtx.setAttribute('compensation.reason', error.message);
    },
  })((ctx) => async () => {
    await inventoryService.reserve(order.items);
  })();

  // Step 2: Charge payment (with compensation)
  await traceStep({
    name: 'ChargePayment',
    linkToPrevious: true,
    compensate: async (stepCtx, error) => {
      await paymentService.refund(order.id);
    },
  })((ctx) => async () => {
    await paymentService.charge(order);
  })();

  // Step 3: Ship order (no compensation - point of no return)
  await traceStep({
    name: 'ShipOrder',
    linkToPrevious: true,
  })((ctx) => async () => {
    await shippingService.ship(order);
  })();
});

// If ShipOrder fails, compensations run in reverse:
// 1. ChargePayment.compensate (refund)
// 2. ReserveInventory.compensate (release)

WorkflowContext Methods:

  • ctx.getWorkflowId() - Get current workflow instance ID
  • ctx.getWorkflowName() - Get workflow type name
  • ctx.getStepIndex() - Current step number
  • ctx.getPreviousStepContext() - SpanContext for linking

Compensation Attributes:

  • workflow.step.compensated - Boolean, true if compensation ran
  • workflow.compensation.executed - Number of compensations executed
  • compensation.reason - Why compensation was triggered

Business Metrics & Product Events

Autotel treats metrics and events as first-class citizens so engineers and product teams share the same context.

OpenTelemetry Metrics (Metric class + helpers)
import { Metric, createHistogram } from 'autotel';

const metrics = new Metric('checkout');
const revenue = createHistogram('checkout.revenue');

export const processOrder = trace((ctx) => async (order) => {
  metrics.trackEvent('order.completed', {
    orderId: order.id,
    amount: order.total,
  });
  metrics.trackValue('revenue', order.total, { currency: order.currency });
  revenue.record(order.total, { currency: order.currency });
});
  • Emits OpenTelemetry counters/histograms via the OTLP endpoint configured in init().
  • Infrastructure metrics are enabled by default in every environment.
Product Events (PostHog, Mixpanel, Amplitude, …)

Track user behavior, conversion funnels, and business outcomes alongside your OpenTelemetry traces.

Recommended: Configure subscribers in init(), use global track() function:

import { init, track, trace } from 'autotel';
import { PostHogSubscriber } from 'autotel-subscribers/posthog';

init({
  service: 'checkout',
  subscribers: [new PostHogSubscriber({ apiKey: process.env.POSTHOG_KEY! })],
});

export const signup = trace('user.signup', async (user) => {
  // All events use subscribers from init() automatically
  track('user.signup', { userId: user.id, plan: user.plan });
  track.funnelStep('checkout', 'completed', { cartValue: user.cartTotal });
  track.value('lifetimeValue', user.cartTotal, { currency: 'USD' });
  track.outcome('user.signup', 'success', { cohort: user.cohort });
});

Event instance (inherits subscribers from init()):

import { Event } from 'autotel/event';

// Uses subscribers configured in init() - no need to pass them again
const events = new Event('checkout');

events.trackEvent('order.completed', { amount: 99.99 });
events.trackFunnelStep('checkout', 'started', { cartValue: 99.99 });

Override subscribers for specific Event instance:

import { Event } from 'autotel/event';
import { MixpanelSubscriber } from 'autotel-subscribers/mixpanel';

// Override: use different subscribers for this instance (multi-tenant, A/B testing, etc.)
const marketingEvents = new Event('marketing', {
  subscribers: [new MixpanelSubscriber({ token: process.env.MIXPANEL_TOKEN! })],
});

marketingEvents.trackEvent('campaign.viewed', { campaignId: '123' });

Subscriber Resolution:

  • If subscribers passed to Event constructor → uses those (instance override)
  • If no subscribers passed → falls back to init() subscribers (global config)
  • If neither configured → events logged only (graceful degradation)

Auto-enrichment adds traceId, spanId, correlationId, operation.name, service.version, and deployment.environment to every event payload without manual wiring.

Logging with Trace Context

Bring your own logger (Pino, Winston, Bunyan, etc.) and autotel automatically instruments it to:

  • Inject trace context (traceId, spanId, correlationId) into every log record
  • Record errors in the active OpenTelemetry span
  • Bridge logs to the OpenTelemetry Logs API for OTLP export to Grafana, Datadog, etc.
Using Pino (recommended)

Note: While @opentelemetry/auto-instrumentations-node includes Pino instrumentation, you may need to install @opentelemetry/instrumentation-pino separately for trace context injection to work reliably.

npm install pino
# Optional but recommended:
npm install @opentelemetry/instrumentation-pino
import pino from 'pino';
import { init, trace } from 'autotel';

const logger = pino({
  level: process.env.LOG_LEVEL || 'info',
});

init({
  service: 'user-service',
  logger,
  autoInstrumentations: ['pino'], // Enable Pino instrumentation for trace context
});

export const createUser = trace(async (data: UserData) => {
  logger.info({ userId: data.id }, 'Creating user');
  try {
    const user = await db.users.create(data);
    logger.info({ userId: user.id }, 'User created');
    return user;
  } catch (error) {
    logger.error({ err: error, userId: data.id }, 'Create failed');
    throw error;
  }
});
Using Winston

Note: While @opentelemetry/auto-instrumentations-node includes Winston instrumentation, you must install @opentelemetry/instrumentation-winston separately for trace context injection to work.

npm install winston @opentelemetry/instrumentation-winston
import winston from 'winston';
import { init } from 'autotel';

const logger = winston.createLogger({
  level: 'info',
  format: winston.format.json(),
  transports: [new winston.transports.Console()],
});

init({
  service: 'user-service',
  logger,
  autoInstrumentations: ['winston'], // Enable Winston instrumentation for trace context
});
Using Bunyan (or other loggers)

Note: While @opentelemetry/auto-instrumentations-node includes Bunyan instrumentation, you must install @opentelemetry/instrumentation-bunyan separately for trace context injection to work.

npm install bunyan @opentelemetry/instrumentation-bunyan
import bunyan from 'bunyan';
import { init } from 'autotel';

const logger = bunyan.createLogger({ name: 'user-service' });

init({
  service: 'user-service',
  logger,
  autoInstrumentations: ['bunyan'], // Enable Bunyan instrumentation for trace context
});

Note: For manual instrumentation configuration, you can also use:

import { BunyanInstrumentation } from '@opentelemetry/instrumentation-bunyan';

init({
  service: 'user-service',
  logger,
  instrumentations: [new BunyanInstrumentation()], // Manual instrumentation with custom config
});

Can't find your logger? Check OpenTelemetry JS Contrib for available instrumentations, or open an issue to request official support!

What you get automatically
  • Logs include traceId, spanId, correlationId for correlation with traces
  • Errors are automatically recorded in the active span
  • Logs export via OTLP to your observability backend (Grafana, Datadog, etc.)
  • Simple setup - install the instrumentation package and enable it in autoInstrumentations

Canonical Log Lines (Wide Events)

Canonical log lines implement the "wide events" pattern: one comprehensive log line per request with ALL context. This makes logs queryable as structured data instead of requiring string search.

Key Benefits:

  • One log line per request with all context (user, cart, payment, errors, etc.)
  • High-cardinality, high-dimensionality data for powerful queries
  • Automatic - no manual logging needed, just use trace() and ctx.setAttribute()
  • Queryable - WHERE user.id = 'user-123' AND error.code IS NOT NULL
Basic Usage
import { init, trace, setUser, httpServer } from 'autotel';
import pino from 'pino';

const logger = pino();
init({
  service: 'checkout-api',
  logger,
  canonicalLogLines: {
    enabled: true,
    rootSpansOnly: true, // One canonical log line per request
    logger, // Use Pino for canonical log lines
  },
});

export const processCheckout = trace((ctx) => async (order: Order) => {
  setUser(ctx, {
    id: order.userId,
    subscription: order.plan,
    accountAgeDays: daysSince(order.userCreatedAt),
  });

  httpServer(ctx, {
    method: 'POST',
    route: '/api/checkout',
    statusCode: 200,
  });

  ctx.setAttributes({
    'cart.total_cents': order.total,
    'payment.method': order.paymentMethod,
    'payment.provider': 'stripe',
  });

  // When this span ends, a canonical log line is automatically emitted
  // with ALL attributes: user.id, user.subscription, cart.total_cents, etc.
});
What You Get

When a span ends, a canonical log line is automatically emitted with:

  • Core fields: operation, traceId, spanId, correlationId, duration_ms, duration, status_code
  • ALL span attributes: Every attribute you set with ctx.setAttribute()
  • Resource attributes: service.name, service.version, deployment.environment
  • Timestamp: ISO 8601 format

Example canonical log line:

{
  "level": "info",
  "msg": "[processCheckout] Request completed",
  "operation": "processCheckout",
  "traceId": "4bf92f3577b34da6a3ce929d0e0e4736",
  "spanId": "00f067aa0ba902b7",
  "correlationId": "4bf92f3577b34da",
  "duration_ms": 124.7,
  "duration": "125ms",
  "status_code": 1,
  "user.id": "user-123",
  "user.subscription": "premium",
  "user.account_age_days": 847,
  "cart.total_cents": 15999,
  "payment.method": "card",
  "payment.provider": "stripe",
  "service.name": "checkout-api",
  "timestamp": "2024-01-15T10:23:45.612Z"
}
Query Examples

With canonical log lines, you can run powerful queries:

-- Find all checkout failures for premium users
SELECT * FROM logs
WHERE user.subscription = 'premium'
  AND error.code IS NOT NULL;

-- Group errors by code
SELECT error.code, COUNT(*)
FROM logs
WHERE error.code IS NOT NULL
GROUP BY error.code;

-- Find slow checkouts with coupons
SELECT * FROM logs
WHERE duration_ms > 200
  AND cart.coupon_applied IS NOT NULL;
Configuration Options
init({
  service: 'my-app',
  canonicalLogLines: {
    enabled: true,
    rootSpansOnly: true,
    minLevel: 'info',
    logger: pino(),
    pretty: true, // tree-formatted console output (defaults to NODE_ENV=development)
    keep: [{ status: 500 }, { durationMs: 1000 }], // declarative tail sampling
    shouldEmit: ({ event }) => {
      // or use a custom predicate (overrides keep)
      const isError = Number(event.status_code) === 2;
      const isSlow = Number(event.duration_ms ?? 0) >= 1000;
      return isError || isSlow || Math.random() < 0.1;
    },
    drain: async ({ event }) => {
      await fetch('https://logs.example.com/ingest', {
        method: 'POST',
        body: JSON.stringify(event),
      });
    },
    messageFormat: (span) => {
      const status = span.status.code === 2 ? 'ERROR' : 'SUCCESS';
      return `${span.name} [${status}]`;
    },
    includeResourceAttributes: true,
  },
});
Request Logger DX

For teams that prefer log.set({...}) ergonomics, you can use getRequestLogger(). It writes correlated request context and log records, so canonical log lines still emit one wide event per request.

import { trace, getRequestLogger, createStructuredError } from 'autotel';

export const checkout = trace((ctx) => async (order: Order) => {
  const log = getRequestLogger(ctx);

  log.set({ user: { id: order.userId, plan: order.plan } });
  log.set({
    cart: { total_cents: order.totalCents, item_count: order.items.length },
  });

  try {
    await processPayment(order);
  } catch (cause) {
    log.error(
      createStructuredError({
        message: 'Payment failed',
        why: 'Card declined by issuer',
        fix: 'Try another payment method',
        link: 'https://docs.example.com/errors/payment-declined',
        cause,
      }),
      { step: 'payment' },
    );
    throw cause;
  }
});

You also get:

  • log.getContext() to inspect the accumulated request context.
  • log.emitNow(overrides?) to capture an immediate snapshot (emits a correlated log-style snapshot and returns { timestamp, traceId, spanId, correlationId, context }).
Drain Pipeline (Batch + Retry + Flush)

If you want batching/retry behavior for canonical drains, wrap drain with createDrainPipeline() style logging:

import { createDrainPipeline, init } from 'autotel';

const pipeline = createDrainPipeline({
  batch: { size: 50, intervalMs: 2000 },
  retry: { maxAttempts: 3, backoff: 'exponential' },
  maxBufferSize: 1000,
});

const drain = pipeline(async (batch) => {
  await fetch('https://logs.example.com/ingest', {
    method: 'POST',
    body: JSON.stringify(batch.map((ctx) => ctx.event)),
  });
});

init({
  service: 'my-app',
  canonicalLogLines: {
    enabled: true,
    rootSpansOnly: true,
    drain,
  },
});

await drain.flush();
parseError (Frontend/API Consumers)

Use parseError() to normalize unknown caught errors into a stable shape:

import { parseError } from 'autotel';

try {
  await api.checkout(order);
} catch (error) {
  const parsed = parseError(error);
  showError(parsed.message, parsed.why, parsed.fix);
}

Diagnostics Channels

autotel/diagnostics bridges Node's built-in diagnostics_channel into autotel spans and events — no monkey-patching, no import-in-the-middle. Every entry point is opt-in and degrades to a no-op on runtimes (edge, old Node) that lack the underlying channels.

Capture console.* as correlated wide events (the patch-free way to get console.log into your traces):

import { captureConsole } from 'autotel/diagnostics';

const stop = captureConsole(); // every console.* → an OTel log record,
// correlated to the active span by trace context

Each call becomes a log record (severity mapped from the method, printf-formatted body, log.source/log.method attributes). Pass { target: 'span-event' } to add events to the active span instead, or { target: 'both' }. Nothing patches the global console, so there's no load-order fragility.

Lightweight HTTP spans + W3C propagation without import-in-the-middle:

import { instrumentHttp } from 'autotel/diagnostics';

const stop = instrumentHttp(); // SERVER span per inbound request (parented to the
// incoming traceparent) + CLIENT span per outbound
// request (injects traceparent downstream)

This is an opt-in alternative to @opentelemetry/instrumentation-http for span coverage and propagation. Limitation: a plain channel can't wrap the request handler, so it does not set an ambient context for the handler's duration — application spans created inside a handler won't auto-nest under the SERVER span. Use @opentelemetry/instrumentation-http if you need that nesting.

Bridge any channel with the shared primitive (also used by autotel-genai's ai:telemetry subscriber):

import { subscribeChannel, subscribeTracingChannel } from 'autotel/diagnostics';

const off = subscribeChannel('my-lib:event', (message) => {
  /* turn message into a span/event */
});

Auto Instrumentation & Advanced Configuration

  • autoInstrumentations : Enable OpenTelemetry auto-instrumentations (HTTP, Express, Fastify, Prisma, Pino…). Requires @opentelemetry/auto-instrumentations-node.
  • instrumentations : Provide manual instrumentation instances, e.g., new HttpInstrumentation().
  • resource / resourceAttributes : Declare cluster/region/tenant metadata once and it flows everywhere.
  • spanProcessor, metricReader, logRecordProcessors : Plug in any OpenTelemetry exporter or your in-house pipeline.
  • headers : Attach vendor auth headers when using the built-in OTLP HTTP exporters.
  • sdkFactory : Receive the Autotel defaults and return a fully customized NodeSDK for the rare cases you need complete control.
import { init } from 'autotel';
import { HttpInstrumentation } from '@opentelemetry/instrumentation-http';

init({
  service: 'checkout',
  environment: 'production',
  resourceAttributes: {
    'cloud.region': 'us-east-1',
    'deployment.environment': 'production',
  },
  autoInstrumentations: ['http', 'express', 'pino'],
  instrumentations: [new HttpInstrumentation()],
  headers: 'Authorization=Basic ...',
  subscribers: [new PostHogSubscriber({ apiKey: 'phc_xxx' })],
});
autoInstrumentations vs. Manual Instrumentations

When using both autoInstrumentations and instrumentations, manual instrumentations always take precedence. If you need custom configs (like requireParentSpan: false for standalone scripts), use one or the other:

Option A: Auto-instrumentations only (all defaults)
init({
  service: 'my-app',
  autoInstrumentations: true, // All libraries with default configs
});
Option B: Manual instrumentations with custom configs
import { MongoDBInstrumentation } from '@opentelemetry/instrumentation-mongodb';
import { MongooseInstrumentation } from '@opentelemetry/instrumentation-mongoose';

init({
  service: 'my-app',
  autoInstrumentations: false, // Must be false to avoid conflicts
  instrumentations: [
    new MongoDBInstrumentation({
      requireParentSpan: false, // Custom config for scripts/cron jobs
    }),
    new MongooseInstrumentation({
      requireParentSpan: false,
    }),
  ],
});
Option C: Mix auto + manual (best of both)
import { MongoDBInstrumentation } from '@opentelemetry/instrumentation-mongodb';

init({
  service: 'my-app',
  autoInstrumentations: ['http', 'express'], // Auto for most libraries
  instrumentations: [
    // Manual config only for libraries that need custom settings
    new MongoDBInstrumentation({
      requireParentSpan: false,
    }),
  ],
});

Why requireParentSpan matters: Many instrumentations default to requireParentSpan: true, which prevents spans from being created in standalone scripts, cron jobs, or background workers without an active parent span. Set it to false for these use cases.

Auto-Instrumentation Setup Requirements

OpenTelemetry's auto-instrumentation packages require special setup depending on your module system:

ESM Setup (Recommended for Node 18.19+)

Use autotel/register for clean ESM instrumentation without complex NODE_OPTIONS:

// instrumentation.mjs (or .ts)
import 'autotel/register'; // MUST be first import!
import { init } from 'autotel';
import { getNodeAutoInstrumentations } from '@opentelemetry/auto-instrumentations-node';

init({
  service: 'my-app',
  instrumentations: getNodeAutoInstrumentations({
    '@opentelemetry/instrumentation-pino': { enabled: true },
  }),
});
# Run with --import flag
tsx --import ./instrumentation.mjs src/server.ts
# or with Node
node --import ./instrumentation.mjs src/server.js

Requirements for ESM instrumentation:

  • Install @opentelemetry/auto-instrumentations-node as a direct dependency in your app
  • Import autotel/register before any other imports
  • Use --import flag (not --require)

CommonJS Setup

No special flags required. Just use --require:

// package.json
{
  "type": "commonjs" // or remove "type" field
}
node --require ./instrumentation.js src/server.js

Zero-Config ESM (reads from env vars):

OTEL_SERVICE_NAME=my-app tsx --import autotel/auto src/index.ts

Legacy ESM (Node 18.0-18.18)

If you can't use autotel/register, use the --experimental-loader flag:

NODE_OPTIONS="--experimental-loader=@opentelemetry/instrumentation/hook.mjs --import ./instrumentation.ts" tsx src/server.ts

Note: The loader hook is an OpenTelemetry upstream requirement for ESM, not an autotel limitation. See OpenTelemetry ESM docs for details.

Operational Safety & Runtime Controls

  • Adaptive sampling : 10% baseline, 100% for errors/slow spans by default (override via sampler).
  • Rate limiting & circuit breakers : Prevent telemetry storms when backends misbehave.
  • Validation : Configurable attribute/event name lengths, maximum counts, and nesting depth.
  • Sensitive data redaction : Passwords, tokens, API keys, and any custom regex you provide are automatically masked before export.
  • Auto-flush : Events buffers drain when root spans end (disable with flushOnRootSpanEnd: false).
  • Runtime flags : Toggle metrics or swap endpoints via env vars without code edits.
# Disable metrics without touching code (metrics are ON by default)
AUTOTEL_METRICS=off node server.js

# Point at a different collector
OTLP_ENDPOINT=https://otel.mycompany.com node server.js

Configuration Reference

init({
  service: string; // required
  subscribers?: EventSubscriber[];
  devtools?: boolean | {
    enabled?: boolean;
    endpoint?: string;
    embedded?: boolean;
    host?: string;
    port?: number;
    verbose?: boolean;
  };
  endpoint?: string; // single OTLP destination shorthand
  destinations?: Array<{
    endpoint: string;
    protocol?: 'http' | 'grpc';
    headers?: Record<string, string> | string;
    signals?: Array<'traces' | 'metrics' | 'logs'>;
  }>;
  protocol?: 'http' | 'grpc'; // OTLP protocol (default: 'http')
  metrics?: boolean | 'auto';
  sampler?: Sampler; // explicit sampler, highest precedence
  sampling?: 'development' | 'errors-only' | 'production' | 'off'; // preset shorthand
  version?: string;
  environment?: string;
  baggage?: boolean | string; // Auto-copy baggage to span attributes
  flushOnRootSpanEnd?: boolean;  // Auto-flush events (default: true)
  forceFlushOnShutdown?: boolean;  // Force-flush spans on shutdown (default: false)
  autoInstrumentations?: string[] | boolean | Record<string, { enabled?: boolean }>;
  instrumentations?: NodeSDKConfiguration['instrumentations'];
  spanProcessor?: SpanProcessor;
  metricReader?: MetricReader;
  logRecordProcessors?: LogRecordProcessor[];
  resource?: Resource;
  resourceAttributes?: Record<string, string>;
  headers?: Record<string, string> | string;
  sdkFactory?: (defaults: NodeSDK) => NodeSDK;
  validation?: Partial<ValidationConfig>;
  logger?: Logger; // created via createLogger() or bring your own
  openllmetry?: {
    enabled: boolean;
    options?: Record<string, unknown>; // Passed to @traceloop/node-server-sdk
  };
});

Local-first recommendation:

init({
  service: 'my-app',
  devtools: true,
});
  • devtools: true routes traces, metrics, and logs to local autotel-devtools
  • devtools: { embedded: true } attempts to start autotel-devtools automatically
  • use endpoint and headers when you are ready to ship telemetry to a hosted backend

Sampling Configuration:

import { init, samplingPresets } from 'autotel';

init({
  service: 'my-app',
  sampling: 'production',
});

init({
  service: 'my-app',
  sampler: samplingPresets.production({ baselineSampleRate: 0.05 }),
});
  • Use sampling for the simple preset path.
  • Use sampler for advanced customization.
  • Precedence is always sampler > sampling > default.

Event Subscribers:

Configure event subscribers globally to send product events to PostHog, Mixpanel, Amplitude, etc.:

import { init } from 'autotel';
import { PostHogSubscriber } from 'autotel-subscribers/posthog';

init({
  service: 'my-app',
  subscribers: [new PostHogSubscriber({ apiKey: process.env.POSTHOG_KEY! })],
});

Event instances automatically inherit these subscribers unless you explicitly override them. See Product Events for details.

Baggage Configuration:

Enable automatic copying of baggage entries to span attributes:

init({
  service: 'my-app',
  baggage: true, // Copies baggage to span attributes with 'baggage.' prefix
});

// With custom prefix
init({
  service: 'my-app',
  baggage: 'ctx', // Copies with 'ctx.' prefix → ctx.tenant.id
});

// No prefix
init({
  service: 'my-app',
  baggage: '', // Copies directly → tenant.id
});

This eliminates the need to manually call ctx.setAttribute() for baggage values. See Baggage (Context Propagation) for usage examples.

Protocol Configuration:

Use the protocol parameter to switch between HTTP/protobuf (default) and gRPC:

// HTTP (default) - uses port 4318
init({
  service: 'my-app',
  protocol: 'http', // or omit (defaults to http)
  endpoint: 'http://localhost:4318',
});

// gRPC - uses port 4317, better performance
init({
  service: 'my-app',
  protocol: 'grpc',
  endpoint: 'localhost:4317',
});

Vendor Backend Configurations:

For simplified setup with popular observability platforms, see autotel-backends:

npm install autotel-backends
import { init } from 'autotel';
import { createDatadogConfig } from 'autotel-backends/datadog';
import { createHoneycombConfig } from 'autotel-backends/honeycomb';

// Datadog
init(
  createDatadogConfig({
    apiKey: process.env.DATADOG_API_KEY!,
    service: 'my-app',
    environment: 'production',
  }),
);

// Honeycomb (automatically uses gRPC)
init(
  createHoneycombConfig({
    apiKey: process.env.HONEYCOMB_API_KEY!,
    service: 'my-app',
    environment: 'production',
    dataset: 'production', // optional, for classic accounts
  }),
);

Environment Variables:

Autotel supports standard OpenTelemetry environment variables for zero-code configuration across environments:

# Service configuration
export OTEL_SERVICE_NAME=my-app

# OTLP collector endpoint
export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318

# Protocol: 'http' or 'grpc' (default: 'http')
export OTEL_EXPORTER_OTLP_PROTOCOL=http

# Authentication headers (comma-separated key=value pairs)
export OTEL_EXPORTER_OTLP_HEADERS=x-honeycomb-team=YOUR_API_KEY

# Resource attributes (comma-separated key=value pairs)
export OTEL_RESOURCE_ATTRIBUTES=service.version=1.2.3,deployment.environment=production,team=backend

Configuration Precedence: Explicit init() config > env vars > defaults

Example: Honeycomb with env vars

export OTEL_SERVICE_NAME=my-app
export OTEL_EXPORTER_OTLP_ENDPOINT=https://api.honeycomb.io
export OTEL_EXPORTER_OTLP_PROTOCOL=grpc
export OTEL_EXPORTER_OTLP_HEADERS=x-honeycomb-team=YOUR_API_KEY
export OTEL_RESOURCE_ATTRIBUTES=deployment.environment=production

Example: Datadog with env vars

export OTEL_SERVICE_NAME=my-app
export OTEL_EXPORTER_OTLP_ENDPOINT=https://http-intake.logs.datadoghq.com
export OTEL_EXPORTER_OTLP_HEADERS=DD-API-KEY=YOUR_API_KEY
export OTEL_RESOURCE_ATTRIBUTES=deployment.environment=production

See packages/autotel/.env.example for a complete template.

Validation tuning example:

init({
  service: 'checkout',
  validation: {
    sensitivePatterns: [/password/i, /secret/i, /creditCard/i],
    maxAttributeValueLength: 5_000,
    maxAttributeCount: 100,
    maxNestingDepth: 5,
  },
});

Building Custom Instrumentation

Autotel is designed as an enabler - it provides composable primitives that let you instrument anything in your codebase. Here's how to use the building blocks to create custom instrumentation for queues, cron jobs, and other patterns.

Instrumenting Queue Consumers
import { trace, span, track } from 'autotel';

// Wrap your consumer handler with trace()
export const processMessage = trace(async function processMessage(
  message: Message,
) {
  // Use span() to break down processing stages
  await span({ name: 'parse.message' }, async (ctx) => {
    ctx.setAttribute('message.id', message.id);
    ctx.setAttribute('message.type', message.type);
    return parseMessage(message);
  });

  await span({ name: 'validate.message' }, async () => {
    return validateMessage(message);
  });

  await span({ name: 'process.business.logic' }, async () => {
    return handleMessage(message);
  });

  // Track events
  track('message.processed', {
    messageType: message.type,
    processingTime: Date.now() - message.timestamp,
  });
});

// Use in your queue consumer
consumer.on('message', async (msg) => {
  await processMessage(msg);
});
Instrumenting Scheduled Jobs / Cron
import { trace, getMetrics } from 'autotel';

export const dailyReportJob = trace(async function dailyReportJob() {
  const metrics = getMetrics();
  const startTime = Date.now();

  try {
    const report = await generateReport();

    // Record success metrics
    metrics.recordHistogram('job.duration', Date.now() - startTime, {
      job_name: 'daily_report',
      status: 'success',
    });

    return report;
  } catch (error) {
    // Record failure metrics
    metrics.recordHistogram('job.duration', Date.now() - startTime, {
      job_name: 'daily_report',
      status: 'error',
    });
    throw error;
  }
});

// Schedule with your preferred library
cron.schedule('0 0 * * *', () => dailyReportJob());
Creating Custom Event Subscribers

Implement the EventSubscriber interface to send events to any events platform:

import { type EventSubscriber, type EventAttributes } from 'autotel';

export class CustomEventSubscriber implements EventSubscriber {
  constructor(private config: { apiKey: string; endpoint: string }) {}

  async track(
    eventName: string,
    attributes: EventAttributes,
    timestamp: Date,
  ): Promise<void> {
    await fetch(this.config.endpoint, {
      method: 'POST',
      headers: { 'X-API-Key': this.config.apiKey },
      body: JSON.stringify({
        event: eventName,
        properties: attributes,
        timestamp,
      }),
    });
  }

  async identify(userId: string, traits: EventAttributes): Promise<void> {
    // Implement user identification
  }

  async flush(): Promise<void> {
    // Implement flush if buffering
  }
}

// Use it in init()
init({
  service: 'my-app',
  subscribers: [new CustomEventSubscriber({ apiKey: '...', endpoint: '...' })],
});
Low-Level Span Manipulation

For maximum control, use the ctx proxy or the ergonomic tracer helpers:

import { ctx, getTracer, getActiveSpan, runWithSpan } from 'autotel';

export async function customWorkflow() {
  // Access current trace context anywhere (via AsyncLocalStorage)
  console.log('Current trace:', ctx.traceId);
  ctx.setAttribute('workflow.step', 'start');

  // Or create custom spans with the tracer helpers
  const tracer = getTracer('my-custom-tracer');
  const span = tracer.startSpan('custom.operation');

  try {
    // Your logic here
    span.setAttribute('custom.attribute', 'value');
    span.setStatus({ code: SpanStatusCode.OK });
  } finally {
    span.end();
  }
}

// Add attributes to the currently active span
export function enrichCurrentSpan(userId: string) {
  const span = getActiveSpan();
  if (span) {
    span.setAttribute('user.id', userId);
    span.setAttribute('user.identified_at', Date.now());
  }
}

// Add attributes with queue context
export async function processQueue() {
  const span = getActiveSpan();
  if (span) {
    span.setAttributes({
      queue_size: 42,
      queue_name: 'order-processing',
    });
    // Process queue...
  }
}

// Run code with a specific span as active
export async function backgroundJob() {
  const tracer = getTracer('background-processor');
  const span = tracer.startSpan('process.batch');

  try {
    await runWithSpan(span, async () => {
      // Any spans created here will be children of 'process.batch'
      await processRecords();
    });
    span.setStatus({ code: SpanStatusCode.OK });
  } catch (error) {
    span.setStatus({ code: SpanStatusCode.ERROR });
    throw error;
  } finally {
    span.end();
  }
}

Available tracer helpers:

  • getTracer(name, version?) - Get a tracer for creating custom spans
  • getActiveSpan() - Get the currently active span
  • getActiveContext() - Get the current OpenTelemetry context
  • runWithSpan(span, fn) - Execute a function with a span set as active

Note: For most use cases, prefer trace(), span(), or instrument() which handle span lifecycle automatically.

Custom Metrics

Create custom business metrics using the meter helpers:

import { getMeter, createCounter, createHistogram } from 'autotel';

// Create custom metrics
const requestCounter = createCounter('http.requests.total', {
  description: 'Total HTTP requests',
});

const responseTimeHistogram = createHistogram('http.response.time', {
  description: 'HTTP response time in milliseconds',
  unit: 'ms',
});

export async function handleRequest(req: Request) {
  const startTime = Date.now();

  requestCounter.add(1, { method: req.method, path: req.path });

  const response = await processRequest(req);

  responseTimeHistogram.record(Date.now() - startTime, {
    method: req.method,
    status: response.status,
  });

  return response;
}

Key Principle: All these primitives work together - spans automatically capture context, metrics and events inherit trace IDs, and everything flows through the same configured exporters and adapters. Build what you need, when you need it.

Serverless & Short-lived Processes

For serverless environments (AWS Lambda, Vercel, Cloud Functions) and other short-lived processes, telemetry may not export before the process ends. Autotel provides two approaches:

Manual Flush (Recommended for Serverless)

Use the flush() function to force-export all telemetry before the function returns:

import { init, flush } from 'autotel';

init({
  service: 'my-lambda',
  endpoint: process.env.OTEL_EXPORTER_OTLP_ENDPOINT,
});

export const handler = async (event) => {
  // Your business logic here
  const result = await processEvent(event);

  // Force-flush telemetry before returning
  await flush();

  return result;
};

The flush() function:

  • Flushes events from the queue
  • Force-flushes OpenTelemetry spans to exporters
  • Includes timeout protection (default: 2000ms)
  • Safe to call multiple times

Custom timeout:

await flush({ timeout: 5000 }); // 5 second timeout
Auto-Flush Spans (Opt-in)

Enable automatic span flushing on root span completion:

init({
  service: 'my-lambda',
  flushOnRootSpanEnd: true, // enabled by default (events only)
  forceFlushOnShutdown: true, // flush spans on root completion
});

export const handler = trace(async (event) => {
  // Auto-flushes when trace completes
  return await processEvent(event);
});

Trade-offs:

  • Zero boilerplate - no manual flush() needed
  • Guaranteed export before process ends (async functions only)
  • Adds ~50-200ms latency per request (network I/O)
  • Only needed for short-lived processes
  • Only applies to async traced functions (synchronous functions cannot await flush)

When to use:

  • Use forceFlushOnShutdown: true for serverless functions where latency is acceptable
  • Use manual flush() for more control over when flushing occurs
  • Use neither for long-running services (batch export is more efficient)
Edge Runtimes (Cloudflare Workers, Vercel Edge)

For edge runtimes with different constraints:

  • Cloudflare Workers: use autotel-cloudflare.
  • Other edge runtimes (Vercel/Netlify/Deno): use autotel-edge.

Cloudflare Workers example:

import { wrapModule, trace, init } from 'autotel-cloudflare';

const processOrder = trace(async (orderId: string, kv: KVNamespace) => {
  return kv.get(orderId);
});

export default wrapModule(
  { service: { name: 'my-worker' } },
  {
    async fetch(_req, env) {
      return Response.json(await processOrder('123', env.ORDERS_KV));
    },
  },
);

autotel-edge remains optimized for vendor-agnostic edge runtimes with automatic flush behavior.

API Reference

  • init(config) : Bootstraps the SDK (call once).
  • trace(fn | name, fn) : Wraps functions with spans and optional context access.
  • span(options, fn) : Creates nested spans for ad-hoc blocks.
  • withTracing(options) : Produces reusable wrappers with shared configuration.
  • instrument(target, options) : Batch-wraps an object of functions.
  • Trace decorator : Adds tracing to class methods (TypeScript 5+).
  • instrumentDatabase(db, options) : Adds automatic DB spans (Drizzle, etc.).
  • Metric class & helpers (createHistogram, etc.) : Emit OpenTelemetry metrics.
  • Event class & track() helper : Send product events/funnels/outcomes/values via subscribers.
  • Logger interface : Bring your own Pino/Winston logger; autotel auto-instruments it for trace context and OTLP export.
  • PostHogSubscriber, MixpanelSubscriber, … : Provided in autotel-subscribers; create your own by implementing the EventSubscriber interface.

Each API is type-safe, works in both ESM and CJS, and is designed to minimize boilerplate while staying close to OpenTelemetry primitives.

FAQ & Next Steps

  • Do I need to abandon my current tooling? No. Autotel layers on top of OpenTelemetry and forwards to whatever you already use (Datadog, Grafana, Tempo, Honeycomb, etc.).
  • Is this just for traces? No. Spans, metrics, logs, and events all share the same context and exporters.
  • Can I customize everything? Yes. Override exporters, readers, resources, validation, or even the full NodeSDK via sdkFactory.
  • Does it work in production? Yes. Adaptive sampling, redaction, validation, rate limiting, and circuit breakers are enabled out of the box.
  • What about frameworks? Use decorators, withTracing(), or instrument() for NestJS, Fastify, Express, Next.js actions, queues, workers, anything in Node.js.

Next steps:

  1. npm install autotel and call init() at startup.
  2. Wrap your critical paths with trace() (or Trace decorators if you prefer classes).
  3. Point the OTLP endpoint at your favorite observability backend and optionally add events adapters.
  4. Expand coverage with instrumentDatabase(), withTracing(), metrics, logging, and auto-instrumentations.

Troubleshooting & Debugging

Quick Debug Mode (Recommended)

The simplest way to see spans locally during development - perfect for progressive development:

import { init } from 'autotel';

// Start with console-only (no backend needed)
init({
  service: 'my-app',
  debug: true, // Outputs spans to console
});

// Later: add endpoint to send to backend while keeping console output
init({
  service: 'my-app',
  debug: true,
  endpoint: 'https://otlp.datadoghq.com', // Now sends to both console AND Datadog
});

// Production: remove debug to send to backend only
init({
  service: 'my-app',
  endpoint: 'https://otlp.datadoghq.com', // Backend only (clean production config)
});

How it Works:

  • debug: true: Print spans to console AND send to backend (if endpoint configured)
    • No endpoint = console-only (perfect for local development)
    • With endpoint = console + backend (verify before choosing provider)
  • No debug flag: Export to backend only (default production behavior)

Environment Variable:

# Enable debug mode
AUTOTEL_DEBUG=true node server.js
# or
AUTOTEL_DEBUG=1 node server.js

# Disable debug mode
AUTOTEL_DEBUG=false node server.js
Manual Configuration (Advanced)

When developing or debugging your instrumentation, you may want more control over span export. Autotel supports manual exporter configuration:

ConsoleSpanExporter (Visual Debugging)

Use ConsoleSpanExporter to print all spans to the console in real-time. This is great for:

  • Quick visual inspection during development
  • Seeing spans as they're created
  • Debugging span structure and attributes
  • Examples and demos
import { init } from 'autotel';
import { ConsoleSpanExporter } from '@opentelemetry/sdk-trace-base';

init({
  service: 'my-app',
  spanExporter: new ConsoleSpanExporter(), // Prints spans to console
});
InMemorySpanExporter (Testing & Assertions)

Use InMemorySpanExporter for programmatic access to spans in tests. This is ideal for:

  • Writing test assertions on spans
  • Querying spans by name or attributes
  • Verifying instrumentation behavior
  • Automated testing
import { init } from 'autotel';
import {
  InMemorySpanExporter,
  SimpleSpanProcessor,
} from '@opentelemetry/sdk-trace-base';

const exporter = new InMemorySpanExporter();

init({
  service: 'test',
  spanProcessor: new SimpleSpanProcessor(exporter),
});

// After running your code...
const spans = exporter.getFinishedSpans();
expect(spans).toHaveLength(1);
expect(spans[0]?.name).toBe('my.operation');
Using Both (Advanced)

For comprehensive debugging, use the debug: true option to combine console output with backend export. See the "Quick Debug Mode" section above for the recommended approach.

Quick Reference:

  • ConsoleSpanExporter: See spans in console output (development/debugging)
  • InMemorySpanExporter: Query spans programmatically (testing/assertions)

Creating Custom Instrumentation

Autotel provides utilities that make it easy to instrument any library with OpenTelemetry tracing. Whether you need to instrument an internal tool, a database driver without official support, or any other library, autotel's helper functions handle the complexity for you.

Quick Start Template

Here's the minimal code to instrument any library:

import { trace, SpanKind } from '@opentelemetry/api';
import { runWithSpan, finalizeSpan } from 'autotel/trace-helpers';

const INSTRUMENTED_FLAG = Symbol('instrumented');

export function instrumentMyLibrary(client) {
  if (client[INSTRUMENTED_FLAG]) return client;

  const tracer = trace.getTracer('my-library');
  const originalMethod = client.someMethod.bind(client);

  client.someMethod = async function (...args) {
    const span = tracer.startSpan('operation.name', {
      kind: SpanKind.CLIENT,
    });

    span.setAttribute('operation.param', args[0]);

    try {
      const result = await runWithSpan(span, () => originalMethod(...args));
      finalizeSpan(span);
      return result;
    } catch (error) {
      finalizeSpan(span, error);
      throw error;
    }
  };

  client[INSTRUMENTED_FLAG] = true;
  return client;
}
Step-by-Step Tutorial: Instrumenting Axios

Let's walk through instrumenting the popular axios HTTP client:

import { trace, SpanKind } from '@opentelemetry/api';
import { runWithSpan, finalizeSpan } from 'autotel/trace-helpers';
import type { AxiosInstance, AxiosRequestConfig, AxiosResponse } from 'axios';

// Step 1: Create instrumentation flag to prevent double-instrumentation
const INSTRUMENTED_FLAG = Symbol('axiosInstrumented');

interface InstrumentedAxios {
  [INSTRUMENTED_FLAG]?: boolean;
}

// Step 2: Define configuration for your instrumentation
export interface InstrumentAxiosConfig {
  tracerName?: string;
  captureHeaders?: boolean;
  captureRequestBody?: boolean;
  captureResponseBody?: boolean;
}

// Step 3: Create the instrumentation function
export function instrumentAxios(
  axios: AxiosInstance,
  config?: InstrumentAxiosConfig,
): AxiosInstance {
  const instrumented = axios as AxiosInstance & InstrumentedAxios;

  // Idempotent check
  if (instrumented[INSTRUMENTED_FLAG]) {
    return axios;
  }

  const {
    tracerName = 'axios-http-client',
    captureHeaders = false,
    captureRequestBody = false,
    captureResponseBody = false,
  } = config ?? {};

  // Step 4: Get tracer instance
  const tracer = trace.getTracer(tracerName);

  // Step 5: Add request interceptor to start spans
  axios.interceptors.request.use((requestConfig: AxiosRequestConfig) => {
    const url = requestConfig.url || '';
    const method = requestConfig.method?.toUpperCase() || 'GET';

    // Step 6: Start span with appropriate attributes
    const span = tracer.startSpan(`HTTP ${method}`, {
      kind: SpanKind.CLIENT,
    });

    // Follow OpenTelemetry semantic conventions
    span.setAttribute('http.method', method);
    span.setAttribute('http.url', url);

    if (captureHeaders && requestConfig.headers) {
      span.setAttribute(
        'http.request.headers',
        JSON.stringify(requestConfig.headers),
      );
    }

    if (captureRequestBody && requestConfig.data) {
      span.setAttribute(
        'http.request.body',
        JSON.stringify(requestConfig.data),
      );
    }

    // Store span in request config for response interceptor
    (requestConfig as any).__span = span;

    return requestConfig;
  });

  // Step 7: Add response interceptor to finalize spans
  axios.interceptors.response.use(
    (response: AxiosResponse) => {
      const span = (response.config as any).__span;
      if (span) {
        span.setAttribute('http.status_code', response.status);

        if (captureResponseBody && response.data) {
          span.setAttribute(
            'http.response.body',
            JSON.stringify(response.data),
          );
        }

        // Step 8: Finalize span on success
        finalizeSpan(span);
      }
      return response;
    },
    (error) => {
      const span = error.config?.__span;
      if (span) {
        if (error.response) {
          span.setAttribute('http.status_code', error.response.status);
        }
        // Step 9: Finalize span on error (records exception)
        finalizeSpan(span, error);
      }
      return Promise.reject(error);
    },
  );

  // Step 10: Mark as instrumented
  instrumented[INSTRUMENTED_FLAG] = true;
  return axios;
}

// Usage:
import axios from 'axios';
import { init } from 'autotel';

init({ service: 'my-api' });

const client = axios.create({ baseURL: 'https://api.example.com' });
instrumentAxios(client, { captureHeaders: true });

// All requests are now traced
await client.get('/users');
Best Practices
1. Idempotent Instrumentation

Always use symbols or flags to prevent double-instrumentation:

const INSTRUMENTED_FLAG = Symbol('instrumented');

export function instrument(client) {
  if (client[INSTRUMENTED_FLAG]) {
    return client; // Already instrumented
  }

  // ... instrumentation code ...

  client[INSTRUMENTED_FLAG] = true;
  return client;
}
2. Error Handling

Always use try/catch with finalizeSpan to ensure spans are properly closed:

try {
  const result = await runWithSpan(span, () => operation());
  finalizeSpan(span); // Sets OK status and ends span
  return result;
} catch (error) {
  finalizeSpan(span, error); // Records exception, sets ERROR status, ends span
  throw error;
}
3. Security - Don't Capture Sensitive Data

Be extremely careful about what you capture in spans:

export interface Config {
  captureQueryText?: boolean; // Default: false for security
  captureFilters?: boolean; // Default: false for security
  captureHeaders?: boolean; // Default: false for security
}

function instrument(client, config) {
  // Only capture if explicitly enabled
  if (config.captureQueryText) {
    span.setAttribute('db.statement', sanitize(query));
  }
}
4. Follow OpenTelemetry Semantic Conventions

Use standard attribute names from OpenTelemetry Semantic Conventions:

// ✅ Good - Standard conventions
span.setAttribute('http.method', 'GET');
span.setAttribute('http.status_code', 200);
span.setAttribute('db.system', 'postgresql');
span.setAttribute('db.operation', 'SELECT');
span.setAttribute('messaging.system', 'kafka');

// ❌ Bad - Custom names
span.setAttribute('method', 'GET');
span.setAttribute('status', 200);
span.setAttribute('database', 'postgres');
5. Choose the Right SpanKind
import { SpanKind } from '@opentelemetry/api';

// CLIENT: Outgoing requests, database calls, API calls
tracer.startSpan('http.request', { kind: SpanKind.CLIENT });

// SERVER: Incoming requests (usually auto-instrumented)
tracer.startSpan('http.server', { kind: SpanKind.SERVER });

// INTERNAL: Internal operations, business logic
tracer.startSpan('process.data', { kind: SpanKind.INTERNAL });

// PRODUCER: Publishing messages to queues
tracer.startSpan('kafka.produce', { kind: SpanKind.PRODUCER });

// CONSUMER: Consuming messages from queues
tracer.startSpan('kafka.consume', { kind: SpanKind.CONSUMER });
6. TypeScript Type Safety

Make your instrumentation type-safe:

import type { MyLibrary } from 'my-library';

interface InstrumentedClient {
  __instrumented?: boolean;
}

export function instrument<T extends MyLibrary>(client: T, config?: Config): T {
  const instrumented = client as T & InstrumentedClient;
  // ... instrumentation ...
  return client;
}
Available Utilities

Autotel provides these utilities for custom instrumentation:

From autotel/trace-helpers
import {
  getTracer, // Get tracer instance
  runWithSpan, // Execute function with span as active context
  finalizeSpan, // Set status and end span with error handling
  getActiveSpan, // Get currently active span
  getTraceContext, // Get trace IDs for correlation
  enrichWithTraceContext, // Add trace context to objects
  getActiveContext, // Get current OpenTelemetry context
} from 'autotel/trace-helpers';

// Get a tracer
const tracer = getTracer('my-service', '1.0.0');

// Start a span
const span = tracer.startSpan('operation.name');

// Run code with span as active context
const result = await runWithSpan(span, async () => {
  // Any spans created here will be children of 'span'
  return await doWork();
});

// Finalize span (OK status if no error, ERROR status if error provided)
finalizeSpan(span); // Success
finalizeSpan(span, error); // Error

// Get current active span (to add attributes)
const currentSpan = getActiveSpan();
if (currentSpan) {
  currentSpan.setAttribute('user.id', userId);
}

// Get trace context for logging correlation
const context = getTraceContext();
// { traceId: '...', spanId: '...', correlationId: '...' }

// Enrich log objects with trace context
logger.info(
  enrichWithTraceContext({
    message: 'User logged in',
    userId: '123',
  }),
);
// Logs: { message: '...', userId: '123', traceId: '...', spanId: '...' }
From @opentelemetry/api
import {
  trace, // Access to tracer provider
  context, // Context management (advanced)
  SpanKind, // CLIENT, SERVER, INTERNAL, PRODUCER, CONSUMER
  SpanStatusCode, // OK, ERROR, UNSET
  type Span, // Span interface
  type Tracer, // Tracer interface
} from '@opentelemetry/api';

// Span methods
span.setAttribute(key, value); // Add single attribute
span.setAttributes({ key: value }); // Add multiple attributes
span.setStatus({ code: SpanStatusCode.OK }); // Mark span successful
span.setStatus({ code: SpanStatusCode.ERROR, message }); // Mark span failed
span.end(); // End span

// For events and exceptions, prefer the autotel APIs over raw Span methods:
//   - recordStructuredError(ctx, error)         // span error + status
//   - getRequestLogger(ctx).info(name, fields)  // correlated log event
// See MIGRATION.md for the OTel Span Event deprecation direction.
Semantic Conventions (Optional)

For database instrumentation, you can reuse constants from autotel-plugins:

import {
  SEMATTRS_DB_SYSTEM,
  SEMATTRS_DB_OPERATION,
  SEMATTRS_DB_NAME,
  SEMATTRS_DB_STATEMENT,
  SEMATTRS_NET_PEER_NAME,
  SEMATTRS_NET_PEER_PORT,
} from 'autotel-plugins/common/constants';

span.setAttribute(SEMATTRS_DB_SYSTEM, 'postgresql');
span.setAttribute(SEMATTRS_DB_OPERATION, 'SELECT');
Real-World Examples

Complete instrumentation template: See INSTRUMENTATION_TEMPLATE.ts for a comprehensive, commented template you can copy and customize.

Production example: Check autotel-plugins/drizzle for a real-world instrumentation of Drizzle ORM showing:

  • Idempotent instrumentation
  • Multiple instrumentation levels (client, database, session)
  • Configuration options
  • Security considerations (query text capture)
  • Full TypeScript support
When to Create Custom Instrumentation

Create custom instrumentation when:

  • No official @opentelemetry/instrumentation-* package exists
  • You're instrumenting internal tools or proprietary libraries
  • You need more control over captured data
  • You want simpler configuration than official packages

Use official packages when available:

  • MongoDB: @opentelemetry/instrumentation-mongodb
  • Mongoose: @opentelemetry/instrumentation-mongoose
  • PostgreSQL: @opentelemetry/instrumentation-pg
  • MySQL: @opentelemetry/instrumentation-mysql2
  • Redis: @opentelemetry/instrumentation-redis
  • See all: opentelemetry-js-contrib
Using Official Instrumentation

To use official OpenTelemetry instrumentation with autotel:

import { init } from 'autotel';
import { MongoDBInstrumentation } from '@opentelemetry/instrumentation-mongodb';
import { RedisInstrumentation } from '@opentelemetry/instrumentation-redis';

init({
  service: 'my-service',
  instrumentations: [
    new MongoDBInstrumentation({
      enhancedDatabaseReporting: true,
    }),
    new RedisInstrumentation(),
  ],
});

// MongoDB and Redis operations are now automatically traced

See also

Happy observing!

Keywords

opentelemetryoteltypescriptobservabilitytracinglogginginstrumentationapmfunctionalanalyticsmetricsbusiness-metricsproduct-analyticstelemetrymonitoring