Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Configuration reference

The engine reads exactly one family of configuration keys — canonical sutra.* properties, each with a canonical SUTRA_* environment-variable mirror. There is no separate framework prefix and no legacy alias layer to reason about.

Sources and precedence

canonical env  >  config file  >  built-in default

The config file path comes from SUTRA_CONFIG (default sutra.properties, read from the engine’s working directory, and only if present); file values may themselves use ${ENV} / env:NAME indirection. A key that has both a sutra.* file entry and its SUTRA_* environment variable set always resolves from the environment.

rust/crates/sutra-engine/src/config.rs is the single source of truth for the exact key list — this page is a map of what’s there and how the pieces fit together, not a duplicate of it; treat that file (and otel.rs for the telemetry keys) as authoritative if this page and the running engine ever disagree.

Deployment source

sutra.* keyenvmeaning
sutra.deployment.sourceSUTRA_DEPLOYMENT_SOURCEdir (default) — watch a folder of sealed archives — or db — the database-backed store, activated only via POST /admin/deployments. See Deployment model.
sutra.deployments.dirSUTRA_DEPLOYMENTS_DIRRequired for the dir source: the directory of .sutra archives the engine watches.
sutra.deployments.poll-intervalSUTRA_DEPLOYMENTS_POLL_INTERVALHow often the dir source rescans (default a few seconds).
sutra.http.portSUTRA_HTTP_PORTListen port; 0 binds an OS-assigned port (always use 0 locally — see Quickstart).

The engine’s own datasource

sutra.* keyenv
sutra.datasource.urlSUTRA_DATASOURCE_URL
sutra.datasource.usernameSUTRA_DATASOURCE_USERNAME
sutra.datasource.passwordSUTRA_DATASOURCE_PASSWORD
sutra.datasource.max-connectionsSUTRA_DATASOURCE_MAX_CONNECTIONS
sutra.datasource.dispatch-max-connectionsSUTRA_DATASOURCE_DISPATCH_MAX_CONNECTIONS

Since 0.6.5 the engine opens two pools over this database. The main pool (max-connections, default 8, the size it always had) serves the execution lanes: intake, steps and resumes. Size it at least to sutra.engine.shards, because each lane holds a connection while it commits a step. The dispatch pool (dispatch-max-connections, default 4 since 0.8.3, 2 before) serves the outbox dispatcher and the timer poller, so a burst of intake can never leave delivery waiting for a connection. The claimer and the delete batcher hold one connection each; the others serve the defers and requeues of failed sends in a burst.

This is the engine-internal database (instances, outbox, lease, audit, inbox) — never a package’s own business data store, which owns its connection independently in datastores.yaml (see Data stores). Note the CLI’s sutra migrate reads a different-named set (SUTRA_DB_URL / SUTRA_DB_USERNAME / SUTRA_DB_PASSWORD / SUTRA_DB_SCHEMA) even when pointed at the same database — see the CLI reference.

Outbox and acknowledgement

sutra.* keyenv
sutra.outbox.tick-intervalSUTRA_OUTBOX_TICK_INTERVAL
sutra.outbox.workersSUTRA_OUTBOX_WORKERS
sutra.outbox.destination-concurrencySUTRA_OUTBOX_DESTINATION_CONCURRENCY
sutra.outbox.max-bufferedSUTRA_OUTBOX_MAX_BUFFERED
sutra.outbox.lag-warnSUTRA_OUTBOX_LAG_WARN
sutra.outbox.batch-sizeSUTRA_OUTBOX_BATCH_SIZE
sutra.outbox.retry.base-delay / .max-delay / .jitterSUTRA_OUTBOX_RETRY_BASE_DELAY / _MAX_DELAY / _JITTER
sutra.outbox.retry.max-attemptsSUTRA_OUTBOX_RETRY_MAX_ATTEMPTS
sutra.ack.deferred.capacitySUTRA_ACK_DEFERRED_CAPACITY
sutra.ack.deferred.timeoutSUTRA_ACK_DEFERRED_TIMEOUT
sutra.ack.deferred.sweep-intervalSUTRA_ACK_DEFERRED_SWEEP_INTERVAL

The outbox delivers many rows at once, under three bounds (since 0.8.3):

  • sutra.outbox.workers (default 32, 1..1024) is the most sends in flight at once, across every destination. 1 restores one delivery at a time. Before 0.8.3 it was a count of workers rows were spread over by instance, defaulting to the lane count and at least 8.
  • sutra.outbox.destination-concurrency (default 32) is the most sends in flight to one destination authority: scheme, host and port, the path ignored. It is clamped to workers with a WARN. A row waiting on a busy destination holds no global slot, so a slow destination does not delay the others.
  • sutra.outbox.max-buffered (default 4096, from workers to 100 000) is the most rows held in memory, in flight or queued. Past it the claimer waits.

One instance’s deliveries still go out one after another, in the order they were enqueued; different instances never share a queue. sutra.outbox.lag-warn (default PT10S, PT0S = off) logs one WARN a minute while delivered rows exceed it, enqueue to delivery. A step that enqueues wakes the dispatcher at once, and a local:// delivery is handed to it in-process as soon as the step’s commit is durable, so the tick (tick-interval, default PT5S) only paces retries that fall due and rows another replica enqueued. sutra.outbox.batch-size (default 50) is how many due rows one claim takes per deployment. Delivered rows are deleted in batches. See Tuning for throughput.

Outbound deliveries retry with backoff forever by default. sutra.outbox.retry.base-delay defaults to PT1S: retry n waits about base-delay × 2ⁿ, so the first retry comes about 2 s after the failure. Since 0.6.20 each failed attempt that will be retried logs a WARN with the destination, the cause’s code and message, the attempt number and next_attempt_at. sutra.outbox.retry.max-attempts is the opt-in ceiling: a delivery that exhausts it is marked terminally poisoned — retained with its last error, never retried again, one incident recorded for a required delivery, and no longer counted by the draining-deployment retirement gate. “We gave up” is a durable, visible state, never a silent disappearance.

Full explanation of the deferred-ack registry the sutra.ack.deferred.* keys tune: Acknowledgement modes. Configuring the attempt ceiling is also what makes a poisoned delivery reachable as a <q:retry> failure — see Retries, history, and schedules.

Instance lifecycle — ownership and retention

sutra.* keyenv
sutra.instance.sweep-intervalSUTRA_INSTANCE_SWEEP_INTERVAL
sutra.instance.claim-timeoutSUTRA_INSTANCE_CLAIM_TIMEOUT
sutra.instance.retentionSUTRA_INSTANCE_RETENTION
sutra.instance.retention-sweep-intervalSUTRA_INSTANCE_RETENTION_SWEEP_INTERVAL

Every resume claims the instance first, so two replicas can never advance one instance concurrently; the stuck-instance sweep (sweep-interval, default PT1M) clears claims whose owner has been silent longer than claim-timeout (default PT5M).

A finished instance is history, not a 404: terminal (completed/terminated) snapshots are retained for sutra.instance.retention (ISO-8601 duration, default P7D) and served by GET /sutra/instances/{id} and GET /admin/instances/{id}/history; a lease-gated sweeper purges rows past the window on the retention-sweep-interval cadence (default PT1H). PT0S restores delete-at-completion. Failed instances are always retained regardless of the window — they need an operator before their deployment may retire; the operator action is instance migration. See Retries, history, and schedules for what the retention window makes queryable.

With <q:audit capture="media">, retention also deletes the recordings the purged instances’ audit events reference, from whichever declared object store holds them (below).

Capacity pools

sutra.* keyenv
sutra.capacity-poolsSUTRA_CAPACITY_POOLS
sutra.capacity-pools.<name>.slotsSUTRA_CAPACITY_POOLS_<NAME>_SLOTS
sutra.capacity-pools.<name>.reserve.inboundSUTRA_CAPACITY_POOLS_<NAME>_RESERVE_INBOUND
sutra.capacity-pools.<name>.reserve.outboundSUTRA_CAPACITY_POOLS_<NAME>_RESERVE_OUTBOUND
sutra.capacity-pools.<name>.tenantsSUTRA_CAPACITY_POOLS_<NAME>_TENANTS

Named, cluster-wide slot counts that channels of any deployment draw from with pool: <name>. Full explanation, defaults and a walkthrough: Capacity pools and sessions.

Object stores

sutra.* keyenv
sutra.object-storesSUTRA_OBJECT_STORES
sutra.object-stores.<name>.uriSUTRA_OBJECT_STORES_<NAME>_URI
sutra.object-stores.<name>.endpointSUTRA_OBJECT_STORES_<NAME>_ENDPOINT
sutra.object-stores.<name>.regionSUTRA_OBJECT_STORES_<NAME>_REGION
sutra.object-stores.<name>.access-key / .secret-keySUTRA_OBJECT_STORES_<NAME>_ACCESS_KEY / _SECRET_KEY
sutra.object-stores.<name>.encryptionSUTRA_OBJECT_STORES_<NAME>_ENCRYPTION
sutra.object-stores.<name>.kekSUTRA_OBJECT_STORES_<NAME>_KEK
sutra.object-stores.<name>.key-idSUTRA_OBJECT_STORES_<NAME>_KEY_ID

Named S3-compatible buckets the engine reads recorded media from (an llm channel’s media-store) and deletes it in (erasure and retention). Credentials and the key-encryption secret are references, resolved at boot; a store that cannot be built refuses startup. Full explanation and a walkthrough: Recordings and media references.

External tasks (the pull worker surface)

sutra.* keyenv
sutra.external-task.default-lock-durationSUTRA_EXTERNAL_TASK_DEFAULT_LOCK_DURATION
sutra.external-task.max-lock-durationSUTRA_EXTERNAL_TASK_MAX_LOCK_DURATION
sutra.external-task.max-async-response-timeoutSUTRA_EXTERNAL_TASK_MAX_ASYNC_RESPONSE_TIMEOUT
sutra.external-task.max-tasksSUTRA_EXTERNAL_TASK_MAX_TASKS
sutra.external-task.retriesSUTRA_EXTERNAL_TASK_RETRIES
sutra.external-task.retry-timeoutSUTRA_EXTERNAL_TASK_RETRY_TIMEOUT

A channel declaring transport: pull parks its deliveries as fetchable tasks instead of dialing an endpoint; workers drive them over POST /sutra/external-tasks/fetch-and-lock (bounded long poll) and .../{id}/complete / .../{id}/failure. The defaults: a PT30S lock when the fetch names none (ceiling PT1H — a longer request is rejected, never clamped), a PT30S long-poll ceiling, 100 tasks per fetch, a worker-failure budget of 3 with PT10S between attempts. A spent budget turns the task terminal (failed) — retained, never fetchable again. The engine boots fail-closed if default-lock-duration exceeds its own ceiling or is zero. Full treatment: External tasks.

Execution lanes

sutra.* keyenvdefault
sutra.engine.shardsSUTRA_ENGINE_SHARDS1
sutra.engine.shard-queue-capacitySUTRA_ENGINE_SHARD_QUEUE_CAPACITYunbounded

sutra.engine.shards is the number of identical actor lanes the engine executes on inside one replica. At N > 1 all work for one instance — routed by a stable hash of its id — still runs on one lane, in arrival order, one request at a time: per-instance serialization is the contract, and it holds at every N. Values above 1 are accepted; the default stays 1 because turning it up has one advertised consequence.

Say it out loud before you raise it. Incidental cross-instance serialization disappears at N > 1. Two concurrent deliveries to two different instances of the same flow never interleave under a single lane — as a side effect of there being one lane, never as a promise. At N > 1 they genuinely run in parallel. A deployment silently leaning on that side effect will observe new interleavings. Every supported concurrency mechanism is unaffected: per-channel singleton / serial consumption, per-channel and per-tenant admission caps, and optimistic expect="unchanged" / pessimistic forUpdate data-store writes.

shard-queue-capacity bounds each lane’s mailbox (unset = unbounded; zero is rejected — “unset” is how you say unbounded). A bounded send awaits on the caller’s task, so backpressure propagates outward to the transport that offered the work — an in-flight HTTP request, a broker prefetch window, a poller tick — and never sideways into another lane.

Meters ship with the feature, each carrying the lane index as a dimension: sutra.engine.shard.queue-depth (per-lane backlog and skew), …dispatches / …parks / …resumes, …handoffs (cross-lane relay hops — expected, and rising with lane count by construction), and …claim-bounces split relay / timer. That last one is the mis-route alarm: on a healthy rollout it reads near zero outside genuine cross-replica contention.

The live lane count is readable without reading config: GET /sutra/health/ready reports it under the loader check’s data.shards, read off the running router rather than echoed back from configuration.

Full picture: Execution lanes.

Broker connectivity events

KeyDefaultMeaning
sutra.engine.connectivity-events.channel (env SUTRA_CONNECTIVITY_EVENTS_CHANNEL)unset(0.8.0) A channel name. When set, every broker outage a transport sees on one of a deployment’s channels is delivered to that deployment as an inbound JSON message on the channel of this name, if the deployment declares one. Unset: nothing is collected.

Declare the channel in the package as an in-process intake and start a process on it:

channels:
  - name: connectivity-in
    transport: local
    codec: json

Each outage produces two messages, one when it starts and one when it ends:

{"code":"SUTRA.INBOUND.AMQP.OUTAGE_ENDED","channel":"orders-in","direction":"inbound","state":"up",
 "startedAt":"2026-11-19T10:00:00.120Z","endedAt":"2026-11-19T10:01:00.420Z","ms":60300,
 "detail":"amqp broker mq.example:5672"}
  • state is down when the outage starts (no endedAt or ms) and up when it ends.
  • code is the transport’s own log code for the change, so the message and the engine log line match.
  • direction: inbound is an intake losing its connection. direction: outbound is a send channel whose send could not reach its broker; each send channel that sent during the outage reports its own window, ending on its first send that reaches the broker again. A send channel that sent nothing during the outage reports nothing.
  • A deployment only receives events for its own channels. One that declares no channel of the configured name receives nothing.
  • Delivery is best effort: up to 1024 events wait per deployment; past that the oldest is dropped and SUTRA.CONNECTIVITY.EVENTS_DROPPED is logged. An event the deployment refuses is logged as SUTRA.CONNECTIVITY.EVENT_UNDELIVERED and never retried.
  • The AMQP 1.0 transport reports these events. Transports without a long-lived broker connection (HTTP, for example) report none.

Broker outages as messages runs it against a broker that is stopped and started.

Limits

sutra.* keyenv
sutra.codec.max-payload-bytesSUTRA_CODEC_MAX_PAYLOAD_BYTES

Full explanation: Limits and quotas.

Admin API auth

sutra.* keyenv
sutra.admin.auth.schemeSUTRA_ADMIN_AUTH_SCHEME (apikey | bearer)
sutra.admin.auth.key-refSUTRA_ADMIN_AUTH_KEY_REF
sutra.admin.auth.headerSUTRA_ADMIN_AUTH_HEADER (default X-API-Key)
sutra.admin.oidc.issuer / .audience / .jwks / .role-claim / .required-roleSUTRA_ADMIN_OIDC_*
sutra.admin.oidc.dev-disabledSUTRA_ADMIN_OIDC_DEV_DISABLED

The /admin/* surface (deployments, instance inspection, subject erasure) is gated fail-closed: unconfigured returns 503, a missing/invalid credential returns 401, and a valid token missing the required role/claim returns 403 — never silently open. The auth-key scheme is the same static-secret model channels use for inbound HTTP auth, and it takes precedence when both are set. Gating is disabled only via the explicit sutra.admin.oidc.dev-disabled=true escape hatch. From 0.6.18 the gate also covers GET /sutra/instances, its ?alias= search and GET /sutra/instances/{id} when admin auth is configured. An engine with no admin configuration at all still answers them without a key, so keep such an engine where only operators can reach it. See Upgrading to 0.6.18.

Audit sinks

sutra.* keyenv
sutra.audit.jsonl.pathSUTRA_AUDIT_JSONL
sutra.audit.otel.endpointSUTRA_AUDIT_OTEL_ENDPOINT
sutra.audit.otel.service-nameSUTRA_AUDIT_OTEL_SERVICE_NAME (default sutra-audit, 0.6.12)
sutra.audit.sqlSUTRA_AUDIT_SQL

The SQL sink writes the audit journal behind execution and best-effort: an event is queued and written by the audit writer, not inside the transaction of the step that produced it, so a crash can lose events not yet written. Since 0.8.1 the writer takes whatever queued while its previous write ran, up to 256 events, and writes it as one multi-row insert per deployment in one transaction, where it used to commit every event on its own. On a durable PostgreSQL that is what lets the journal keep up with a busy engine. The journal is the same: each instance’s events in order, deduplicated, and a batch that fails is written again event by event. Nothing to configure.

See Logging and audit.

Telemetry (OTel)

sutra.* keycanonical envstandard OTEL_* env also accepted
sutra.telemetry.otlp.endpointSUTRA_TELEMETRY_OTLP_ENDPOINTOTEL_EXPORTER_OTLP_ENDPOINT
sutra.telemetry.service-nameSUTRA_TELEMETRY_SERVICE_NAMEOTEL_SERVICE_NAME
sutra.telemetry.enabledSUTRA_TELEMETRY_ENABLEDOTEL_SDK_DISABLED (inverted)
sutra.telemetry.metrics.export-intervalSUTRA_TELEMETRY_METRICS_EXPORT_INTERVALOTEL_METRIC_EXPORT_INTERVAL
sutra.telemetry.metrics.temporality-preferenceSUTRA_TELEMETRY_METRICS_TEMPORALITY_PREFERENCEOTEL_EXPORTER_OTLP_METRICS_TEMPORALITY_PREFERENCE
sutra.telemetry.metric-labelsSUTRA_TELEMETRY_METRIC_LABELS— (default tenant,module,version)
sutra.telemetry.resource-attributesSUTRA_TELEMETRY_RESOURCE_ATTRIBUTESOTEL_RESOURCE_ATTRIBUTES (both apply; the sutra.* value wins per key; 0.6.12)

No OTLP endpoint configured means no exporters at all, and this is the default — telemetry export is opt-in, never opt-out, and Sutra itself collects nothing regardless: see No telemetry, no phone-home for the exact guarantee, and Observability for what each signal actually carries once you do turn export on.

Secrets — never literal values

Every credential-shaped field across channels.yaml, datastores.yaml, and the admin-auth keys above is a reference, resolved at startup or channel-activation time — env:NAME, secret:KEY (a file under the mounted secrets directory, default /etc/sutra/secrets / SUTRA_SECRETS_DIR), ${NAME} / ${NAME:default} placeholders, or a vendor scheme (vault:…, aws-secrets:…, azure-kv:…, gcp-secret:…) resolved by whichever sutra-envref-<vendor> crate the binary was built with. A literal secret value in a resource file is rejected at package-validation time, not just discouraged by convention. See Domain neutrality and the SPI model for how a vendor secret backend plugs in.

Next