Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Deployment packages

A Sutra project is a single deployment package. sutra create app produces exactly one; sutra package <dir> seals it into one immutable, content-addressed .sutra archive; sutra deploy activates that archive on a running engine. There is no build step in between — you author declarative resources, seal them, and hand the sealed archive to a generic engine binary that never needs your source. Secrets are the one thing that never travels inside it: a resource names them by scheme (env:NAME, secret:KEY, vault:…), resolved at runtime, and package-time validation rejects a literal outright (see Channels and transports).

flowchart LR
    DIR["the package directory<br/>bpmn · schemas · templates<br/>channels.yaml · datastores.yaml"]
    DIR -->|"sutra package<br/>validates first, fail-closed"| AR["one immutable .sutra<br/>manifest of per-file digests"]
    AR -->|"deploymentId =<br/>sha256(manifest)"| DEP["sutra deploy"]
    DEP --> SLOT["the slot:<br/>tenant--module--version"]
    AR -.->|"carries env:NAME or<br/>secret:KEY, never a value"| SEC["resolved at runtime,<br/>outside the archive"]

Everything the engine needs to run the module is sealed and hashed, so identical bytes re-deploy as a no-op and the same archive promotes across environments unchanged — while the values that must differ per environment stay references it resolves on the way up.

Package interior

<package>/
├── package.yaml             # manifest: labels {tenant, module, version}, engine.minContract
├── bpmn/**/*.bpmn            # BPMN 2.0 processes
├── rules/                    # DMN decision tables (.dmn) + the .srl rule DSL, by extension
├── scripts/**/*.{hbs,xsl,xslt}     # derive/compute (Handlebars + XSLT)
├── templates/                # render output (Handlebars + XSLT)
├── schemas/                  # each leaf folder under here = one codec
│   └── <codec>/*.xsd + codec-manifest.yaml
├── migrations/<store>/**     # per-datastore SQL migrations
├── channels.yaml             # transport channels (bind codec URNs; hot-reloaded on flip)
└── datastores.yaml           # datastore declarations

Every artifact folder supports nested subfolders — bpmn/orders/checkout/…, templates/eu/invoices/… — and discovery is recursive. sutra create deployment <name> --from packages/my-app scaffolds a sibling package as an explicit copy; packages never inherit from one another.

package.yaml — labels, not scope

labels:
  "module": "money-transfer"
  "tenant": "default"
  "version": "1.0.0"
engine:
  minContract: 1

tenant / module / version are opaque labels — selectors for observability, routing, and row-level-security partitioning (tenant_id) — never a resource-tree dimension. A package archive is fully self-contained: there is no tenants/<id>/ overlay tree, no shared-library modules/ folder, and no inheritance between packages at deploy time. What a tenant “sees” is simply every process in the package deployed under its labels. The authoring tree may still be layered — see Composing a package from layers.

properties: — deployment configuration

properties:
  defaultWindow: "${DEFAULT_WINDOW:PT10S}"   # an env reference with a default
  callTimeout: "env:CALL_TIMEOUT"            # a whole-value env reference
  startAfter: "PT1H"                         # a literal

properties: is a flat map of keys to text. A timer expression reads it as config.<key> (see Timer intervals from FEEL). The archive seals the text as written, so the same archive runs in every environment. The engine resolves env:NAME, ${NAME} and ${NAME:default} from its own environment when it activates the deployment. A reference with no value and no default refuses the deployment (SUTRA.DEPLOY.PROPERTY_UNRESOLVED). Keys are letters, digits and _. Values are strings, so use number(config.x) for arithmetic. Secret references (secret:KEY, ${secret:KEY}, a secret store) are refused, because every timer expression and the audit trail can read the configuration. Editing a property changes the deploymentId, the same as an audit: change. Changing the environment does not, and takes effect when the engine next activates the deployment.

Composing a package from layers

When one process is deployed several ways — the same flow bound to HTTP for one tenant and to a broker for another, or a second edition of the same contract — keep the common content once and let each variant hold only what differs:

order-intake/
├── shared/                          # common to every variant; no package.yaml here
│   ├── bpmn/order-intake.bpmn
│   ├── rules/approval.dmn
│   └── templates/receipt.hbs
└── variants/
    ├── acme--order-intake--1.0.0/
    │   ├── package.yaml             # this variant's labels
    │   └── channels.yaml            # HTTP binding
    └── globex--order-intake--1.0.0/
        ├── package.yaml
        └── channels.yaml            # broker binding

Name the shared folder as an overlay, and the two commands see the same composed tree:

sutra lint    variants/acme--order-intake--1.0.0 --overlay shared
sutra package variants/acme--order-intake--1.0.0 --overlay shared -o dist/

The rules, in the order they bite:

  • Overlays are base layers. --overlay may be repeated; each is applied in the order given, and the package directory is applied last — so a variant’s file always wins over a shared one.
  • An overlay carries no package.yaml. Labels belong to the variant, and a package.yaml found inside an overlay is refused rather than quietly merged.
  • Every overridden path is reported, so a variant shadowing a shared file does it visibly.
  • Nothing is deleted. A variant can add to or replace shared content, never remove it. Content a variant must not carry does not belong in shared/ — or use a deployment manifest, whose layers can drop files with .ignore.
  • The package directory’s name names the archive (acme--order-intake--1.0.0.sutra), which is why variant directories are named after their labels.
  • Composition happens at package time. The sealed archive holds the merged file set and knows nothing about layers, and its deployment id is identical to sealing a hand-merged directory with the same content. Layering is an authoring convenience; it changes nothing about what deploys, or about how the engine loads it.

Deployment manifests — customers, editions, environments

--overlay covers one shared folder plus one variant. When a package is assembled from more dimensions — shared content, a product edition, a customer, an environment — write the stack down once in a deployment manifest and hand that file to sutra lint, sutra package or sutra deploy in place of the package directory:

order-hub/
├── shared/schemas/                  # used by several products; mounted at schemas/
├── app/
│   ├── shared/                      # bpmn/ rules/ templates/ channels.yaml
│   └── editions/
│       └── 2026r1/                  # what this edition changes (bpmn/intake.bpmn)
├── customers/customer-a/
│   ├── common/                      # this customer's templates
│   └── envs/
│       ├── dev/                     # channels.yaml + .ignore
│       └── prod/                    # channels.yaml
└── deploy/
    ├── customer-a.2026r1.dev.yaml
    └── customer-a.2026r1.prod.yaml
# deploy/customer-a.2026r1.dev.yaml
deployment: customer-a-dev            # optional, informational
package:                              # becomes the package's package.yaml
  tenant: customer-a                  # tenant / module / version are label shorthands
  module: order-hub
  version: 1.1.0
  engine: { minContract: 1 }
layers:                               # lowest first; later wins; relative to this file
  - path: ../shared/schemas
    at: schemas                       # mount this folder under schemas/
  - ../app/shared
  - ../app/editions/2026r1
  - ../customers/customer-a/common
  - ../customers/customer-a/envs/dev
target:                               # optional: sutra deploy's flags
  engine-url: http://localhost:8088
  api: true

deploy/customer-a.2026r1.prod.yaml is the same file with envs/prod as its last layer and its own target:. The dev layer overrides channels.yaml and drops a process dev does not run:

# customers/customer-a/envs/dev/.ignore
bpmn/reconcile.bpmn
sutra lint    deploy/customer-a.2026r1.dev.yaml
sutra package deploy/customer-a.2026r1.prod.yaml -o dist/     # → dist/customer-a.2026r1.prod.sutra
sutra deploy  deploy/customer-a.2026r1.dev.yaml

sutra lint prints the resolved stack before its findings, so what will be sealed is never a guess:

manifest deploy/customer-a.2026r1.dev.yaml (deployment customer-a-dev)
  layer 1 ../shared/schemas (at schemas)
  layer 2 ../app/shared
  layer 3 ../app/editions/2026r1
  layer 4 ../customers/customer-a/common
  layer 5 ../customers/customer-a/envs/dev
  override bpmn/intake.bpmn: layer 3 ../app/editions/2026r1 over layer 2 ../app/shared
  override channels.yaml: layer 5 ../customers/customer-a/envs/dev over layer 2 ../app/shared
  removed bpmn/reconcile.bpmn: by layer 5 ../customers/customer-a/envs/dev .ignore 'bpmn/reconcile.bpmn' (was from layer 2 ../app/shared)

The rules:

  • It is only a shorthand. The CLI turns the manifest into a generated package.yaml plus the layer stack, then validates and seals exactly as for a directory. Sealing a manifest produces the same archive bytes and the same deployment id as sealing a hand-merged directory holding the same files and the same package.yaml. The archive is named after the manifest file (customer-a.2026r1.dev.sutra).
  • A closed schema. The top-level keys are deployment, package, layers and target; any other key is refused. package: takes the package.yaml keys (labels, engine, entryProcesses, audit, properties) plus the tenant, module and version shorthands; module and version are required.
  • Layers apply in order, a later layer’s copy of a file replacing an earlier one’s; every override is reported. A layer may not carry package.yaml — the refusal names the layer. A missing layer folder, or an at: that is absolute or climbs out with .., is refused.
  • --overlay still works and stacks above the manifest’s last layer.
  • target: takes sutra deploy’s flag names (context, namespace, configmap, secret-name, engine-url, api, async, wait, wait-timeout, api-key-header). A flag given on the command line wins. Secrets never go in a manifest — pass them with --secret or --secret-from.

.ignore — removing what a lower layer brought

A layer may carry a .ignore file at its root. It uses gitignore syntax — # comments, *, ? and […] within a path segment, ** across segments, a trailing / for a directory, a leading or inner / to anchor, ! to re-include — and:

  • patterns are relative to the package root, not to the layer folder (nor to its at:);
  • when the layer is applied, its patterns first remove matching files contributed by the layers below it, then the layer’s own files are added — a layer never ignores its own files;
  • the last matching pattern decides, a pattern naming a directory covers everything under it, and a ! pattern can bring back a single file under an ignored directory;
  • package.yaml can never be ignored;
  • a pattern that removes nothing is reported as a warning (SUTRA.DEPLOY.PACKAGE.IGNORE_UNMATCHED) — it is usually stale or mistyped;
  • .ignore files are never sealed into the archive.

Referencing resources — convention over configuration

Because the package is self-contained, resources are referenced by their local key, derived from the folder tree — never by tenant/module/version:

ResourceResolved by
BPMN processprocessId — globally unique within the package
Rule (.dmn / .srl)its relative path under rules/
Script / templateits full relative path (folder + filename + extension)
Codeca URN — see below

Codecs are the one exception. A globally-named codec — a format the engine links in (json, xml, yaml, csv, raw-text, raw-bytes) or a codec crate a distribution force-links — is urn:sutra:codec:<name>; a package-defined codec is named by its path under schemas/, / folded to : — schemas/transfer/ becomes urn:transfer, schemas/hr/employee/ becomes urn:hr:employee. channels.yaml binds that URN:

# examples/money-transfer/.../channels.yaml
channels:
  - name: transfer-request
    transport: http
    bind: "POST /channels/transfer-request"
    codec: urn:transfer     # schemas/transfer/*.xsd, declared by schemas/transfer/codec-manifest.yaml

A codec-manifest.yaml sits inside its own schemas/<codec>/ folder and declares the schema kind plus the formats the codec accepts:

# examples/money-transfer/.../schemas/transfer/codec-manifest.yaml
schemaKind: xsd
formats: [xml, json, yaml]

Tabular formats — csv and fixed-width

formats: is not limited to document syntaxes. A schema codec can bind csv or fixed-width, which is what turns a delimited or fixed-column upload from a bag of untyped strings into a typed message:

# examples/call-log-load/.../schemas/cdr/codec-manifest.yaml
schemaKind: xsd
formats: [csv, fixed-width]

csv:
  delimiter: ","          # both default; written here only to show where layout lives
  header: true

fixed-width:
  fields:                 # REQUIRED — a fixed-width line has no structure of its own
    - {name: recordId,    width: 12}
    - {name: msisdn,      width: 16}
    - {name: durationSec, width:  6}

A tabular body is a batch, so it is validated row-wise: each row is checked as one instance of the declared root, in a single decode, before the process starts. Every violation carries a row-indexed path (value[3].durationSec), so a bad cell names its record. An unparseable file is fatal; any row’s violation is a soft error, so the payload still projects and q:onValidation decides the posture. The batch projects under value, so a flow reads payload.value[0].field.

The XSD’s leaf types reach the cells: an xs:int column arrives as a number, not "182". An empty cell reads as absence for an element the type declares minOccurs="0" — without that, an optional column would have to be populated in every row. An empty cell for a required element is still a violation.

Both tabular formats may be declared together. Their content types are disjoint — text/csv / application/csv against text/plain / application/x-fixed-width — so an upload selects its parser unambiguously and one schema serves a CSV feed and a fixed-width feed over the same channel.

Layout is parser config, not schema: it cannot be expressed in an XSD or a JSON Schema (neither has byte offsets), which is why it lives in the manifest. A header-bearing CSV names its own columns and needs no block at all; a fixed-width codec cannot work without one and declaring the format without it is a manifest error.

Because a fixed-width layout is configuration, its columns are checked against the bound type at package time: a column the type does not declare, or a required element with no column, fails sutra lint (SUTRA.CONFIG.CODEC_MANIFEST.REJECTED) rather than every row at runtime. A csv codec gets no equivalent check — its column names arrive in the header, at runtime, so there is nothing to compare against beforehand.

The opaque formats (raw-text, raw-bytes) are not schema-bindable: there is no map under raw bytes for a schema to type, so binding one is refused with that reason.

The reserved token sutra may not be used as a first-level subfolder name under any artifact folder (schemas/sutra/, bpmn/sutra/, …) — that would collide with the engine’s own urn:sutra:* namespace. A deeper sutra (schemas/hr/sutra/) is fine.

Schema bundles — when a codec is a whole profile

schemaKind: xsd and schemaKind: json-schema are the generic case: a folder of schema files the engine validates a decoded document against. Some standards aren’t that shape at all — they’re a whole profile: an envelope grammar, a mapping from a wire-level message name to a schema file, and versioned editions the profile revs on its own release cadence, independent of Sutra releases. For those, schemaKind names a bundle kind a codec crate has registered (see Domain neutrality and the SPI model), and the folder’s job shifts from “here are the schemas” to “here is the configuration that decides which schema backs which message, for this archive version.”

A codec crate registers its own bundle kind this way whenever the standard it serves is a whole profile rather than a bare schema folder — say, a market venue that publishes a wrapper envelope around every message and revs its schema editions on its own release calendar. That codec crate is a proprietary extension, not part of this distribution — the mechanism below is generic; naming a concrete kind is only for illustration. Its manifest maps wrapper element names — not a bare schema namespace — onto schema files archived alongside it:

# schemas/<your-kind>/codec-manifest.yaml
schemaKind: <your-kind>
# schemas/<your-kind>/<your-kind>-manifest.yaml
appHdr: edition-2024/Header_v1.xsd   # optional
incoming:
  OrderConfirmation: edition-2026/OrderConfirmation_v3.xsd
outgoing:
  OrderConfirmation: edition-2026/OrderConfirmation_v3.xsd
  ShipmentNotice: edition-2024/ShipmentNotice_v2.xsd
schemas/<your-kind>/codec-manifest.yaml      # schemaKind: <your-kind>
schemas/<your-kind>/<your-kind>-manifest.yaml   # appHdr? / incoming{wrapper: relpath} / outgoing{wrapper: relpath}
schemas/<your-kind>/<edition-folder>/*.xsd   # free-form folder names — the manifest is the only truth

A few things fall out of that shape:

  • Edition folders sit side by side. Because the venue revs its schemas per release (the same wrapper can move from one schema version to the next between editions), archiving each release’s files under its own folder and repointing the manifest is how a module adopts a new edition — a new version of the archive, no engine or codec change.
  • An unlisted wrapper falls back to the codec crate’s own base schema for its pinned default version, so a bundle only needs to carry the wrappers it actually wants to validate more strictly than that default — the codec stays useful with zero configuration otherwise.
  • An enriched edition can be licensed material. A codec’s base schemas may be freely redistributable while a fuller, usage-guideline edition is a licensed product participants obtain themselves and supply in their own archive — the engine never ships that tier, and neither does any artifact built from this repository.
  • Registration is deployment-scoped, exactly like every other artifact under the registration model: the bundle’s registry key is urn:sutra:codec:<folder-path-with-'/'-folded-to-':'>:<deploymentId>. Naming the folder <your-kind> shadows the globally-registered urn:sutra:codec:<your-kind> for that deployment only — a second version of the same module with a different edition mapping registers under its own deploymentId and runs side by side, no collision.
  • Deploy-time errors are fail-closed: an unknown wrapper name, a wrapper listed under the wrong direction, a missing or uncompilable schema file, or a schema whose namespace doesn’t match the expected one all reject the archive rather than deploy something that silently validates less than the manifest claims.
flowchart LR
    W["a wrapper element name on the wire<br/>OrderConfirmation"] --> M["the bundle manifest<br/>incoming / outgoing maps"]
    CM["codec-manifest.yaml<br/>schemaKind names the bundle kind<br/>a codec crate registered"] --> M
    M -->|"listed"| E["that edition folder's schema<br/>edition-2026/OrderConfirmation_v3.xsd"]
    M -.->|"unlisted"| B["the codec crate's own base schema,<br/>at its pinned default version"]
    NEW["a new edition of the profile"] -.->|"archive its folder alongside,<br/>repoint the manifest,<br/>ship a new archive version"| M

The manifest is the only truth about which schema backs which message — the folder names carry no meaning — so adopting a new edition is a package change and a new deploymentId, never an engine or codec-crate change.

Store migrations, and evolving a projected structure

Every sql data store the package declares brings its own schema, under a folder named for the store:

migrations/<store-id>/V001__<description>.sql
migrations/<store-id>/V002__<description>.sql

<store-id> is the store’s name in datastores.yaml, and the migrations: key of that store points at the folder (migrations: migrations/accounts). The scripts are yours: your dialect, your table names, your indexes and seed rows. The engine generates no DDL for a module store.

One store is the exception: the reserved coverage store, where path-coverage marks are persisted. Its declaration still picks the database, but the engine owns its schema — it ships that DDL per dialect and applies it to the connection on the same first-use path — so the store block carries no migrations: key and a package carries no migrations/coverage/ folder. See Coverage: declared routes as the compliance signal.

Three properties of how the package’s own scripts run are worth designing around:

  • Applied in V<n>__ order, before the store serves its first operation — and serialized across replicas, so two engines booting at once don’t race.
  • Each script runs once per database. Applied scripts are recorded in a history table, sutra_store_migrations, in the store’s own database; a recorded script is skipped on every later boot, and editing one after it was applied refuses the store (SUTRA.RUNTIME.DATASTORE.MIGRATION_CHECKSUM). A change is always a new, higher-numbered script. See Migrations run once for the full rules, including the five ${…} placeholders a script can use.
  • A store’s data carries across a version bump. There is no deployment_id on a business store’s rows: deploying 1.0.1 beside 1.0.0 doesn’t fork the data, and that is the feature, not an oversight.

When a projected structure changes

A store that declares a structure: block adds one obligation: a changed structure ships with the migration that makes the table match, in the same package. sutra lint derives the effective table shape from these very scripts, so it tells you at package time whether the pair is consistent — its job is to detect the mismatch, not to repair it.

Change to the declared typeWhat it costs
Add an optional scalar fieldAdditive. A new nullable column; rows written before it read the field as absent
Add a required scalar fieldLint error until the column exists and is nullable or has a DEFAULT — existing rows cannot satisfy a bare NOT NULL
Remove a fieldThe column becomes unmapped (a warning); existing data is untouched
Widen a facet (maxLength 35 → 70)Lint error until the ALTER ships in a new V… script; clean once it does
Rename a fieldModelled as remove + add. A columns: mapping can keep the physical column name instead
Scalar → nested or repeatedHard stop: STRUCTURE_NOT_FLAT. Either keep the field flat, or drop the structure block and go back to the opaque store — an explicit decision at package time, never a silent shape change

What follows from the table is a packaging rule: a type change and the ALTER that supports it belong in the same package. Lint replays every script in the folder in version order and compares the result against the declared type as it stands, so a package carrying both lands clean, while one carrying only the type change fails the gate rather than the deployment.

Building one by hand vs. scaffolding

sutra create app <name> (see Anatomy of an app) generates a package in this exact shape, verified through the engine’s own loaders before anything is written. Growing it from there:

sutra create bpmn my-process --package packages/my-app --validation fatal
sutra create deployment my-app-eu --from packages/my-app   # explicit variant copy

sutra lint <package-dir> runs the full package-time validation suite (the same checks sutra package runs before sealing) with no output on success — the fast pre-flight to run before every package.

Next