Run your instance

This page is for the person who runs a dynamo-core instance: the scheduler, keys at rest, backups, the control console and alerts. It describes the core at owner-api 1.11.0. The boot below runs in Control Mode (no money moves); to add the stablecoin mode on a local chain, see A stablecoin core on a local chain. For the hosted preview, see Preview.

There are two ways to start a core. node deploy/image/entry.mjs (in the image, node /app/server.cjs) is the hosted entry: it runs the production checks below, serves /healthz, rate-limits each client, writes a payload-free audit log, and has the rekey command and the multi-tenant mode. The plain core-host entry, node packages/core-host/dist/main.js after a build, is a development host without those checks. Use the hosted entry for anything another person depends on. The service levels and the alerts are in service levels and limits, a status page draft is in status page, and personal-data export, deletion and the admin audit trail are in privacy.

Boot a local instance

DYNAMO_REPO is the path of your clone of the repository; set it once with export DYNAMO_REPO=/path/to/your/clone. The command below stays in the foreground; a successful boot ends with a line naming owner-api/1.11.0 + plugin wire on http://127.0.0.1:8790.

cd "$DYNAMO_REPO"
pwd
pnpm install --frozen-lockfile
pnpm --filter @dynamo/core-host... -r build
export DYNAMO_OWNER_TOKEN="dp-local-owner-$(openssl rand -hex 16)"
[ -s .dynamo-local.key ] || (umask 077 && openssl rand -hex 32 > .dynamo-local.key)
export CORE_KEY_ENCRYPTION_KEY="$(cat .dynamo-local.key)"
OWNER_API_TOKEN="$DYNAMO_OWNER_TOKEN" PLUGIN_WIRE_TOKEN="dp-local-wire-$(openssl rand -hex 16)" \
  OWNER_API_PORT=8790 CORE_DB="$PWD/.dynamo-local/core" \
  node deploy/image/entry.mjs

The first boot writes a random key-encryption key to .dynamo-local.key (gitignored); later boots, and the plugin READMEs' quickstarts, reuse it for the same .dynamo-local/ database. A boot with another key refuses with KeyEncryptionError: key check value mismatch (wrong key-encryption key), followed by the database path and the fix: use the key the database was made with, or start over with rm -rf .dynamo-local .dynamo-local.key (that deletes the local budgets). On a real deployment, keep CORE_KEY_ENCRYPTION_KEY somewhere other than the data volume. Without it the instance cannot read its own keys, and its backups cannot be restored.

A stablecoin core on a local chain

make dev-chain starts a local Anvil with a one-second block time, deploys contract version 2 with a test stable, mints it to the budget owner (Anvil dev account 1) and the adopt user (account 4), and prints the environment block to add to the boot command. It stays in the foreground; Ctrl-C stops Anvil. It needs Foundry (anvil, forge) and a built engine.

cd "$DYNAMO_REPO"
pwd
pnpm --filter @dynamo/engine... -r build
make dev-chain

In a second terminal, boot the core with the printed block added (CORE_ALLOWED_MODES=none,usdc, CORE_ESCROW_SETTLEMENT=payee-kit, CORE_ESCROW_RPC_URL, CORE_ESCROW_OWNER_KEY and the CORE_ESCROW_V2_* settings: the stablecoin rail on contract version 2 alone). Every key it prints is a public Anvil dev key; never use them anywhere else. Use a fresh CORE_DB each time you restart make dev-chain, because the new chain does not have the old budgets.

Anvil must run with a block time. On Anvil's default (a block only when a transaction arrives) the latest block can be older than a voucher's signing time, and a settle reverts (SignedInFuture, which tools without the vault ABI show as custom error 0xd663ef37). make dev-chain passes --block-time 1; on your own Anvil, pass it too, or mine a block (cast rpc evm_mine) before settling.

Create a budget

With the instance above running, in a second terminal with the same DYNAMO_OWNER_TOKEN exported. Amounts are micro-USD as decimal strings; 5000000 is 5 USD, in Control Mode (no money moves).

cd "$DYNAMO_REPO"
pwd
curl -s -X POST http://127.0.0.1:8790/owner/v1/budgets \
  -H "authorization: Bearer $DYNAMO_OWNER_TOKEN" -H "content-type: application/json" \
  -d '{"funding": "none", "capTotal": "5000000", "mandate": {"categories": ["research"], "maxDepth": 3}}'

The same from the Python SDK, after the install step in packages/sdk-python/README.md:

cd "$DYNAMO_REPO"
pwd
DYNAMO_CORE_URL=http://127.0.0.1:8790 .venv-dynamo/bin/python - <<'PY'
import asyncio, os
from dynamoprotocol import Dynamo, usd

async def main():
    async with await Dynamo.open(os.environ["DYNAMO_CORE_URL"], os.environ["DYNAMO_OWNER_TOKEN"]) as dyn:
        budget = await dyn.open_budget("none", usd("5"), {"categories": ["research"], "maxDepth": 3})
        print(budget.budget_id)

asyncio.run(main())
PY

Streams, delegation, freeze, approvals and revocation are in the Python SDK README, packages/sdk-python/README.md in the repository.

Keys at rest

Session keys and the engine signer key are encrypted with AES-256-GCM under CORE_KEY_ENCRYPTION_KEY: 32 bytes, as hex or base64.

accepts CORE_ALLOW_PLAINTEXT_KEYS=1 for development only; the hosted entry refuses that variable outright.

transaction, and freed pages are zeroed so no plaintext remains in the database file. Running it again changes nothing.

match). A database that holds encrypted keys refuses to open without a key.

backup, restore and rekey commands never print a key.

node deploy/image/entry.mjs rekey <new-key-file> [--backup <file>]... (hosted entry only), then boot with the new key. rekey refuses while the instance's scheduler holds a live lease. It cannot see a running multi-tenant deployment, so stop that one yourself. If it stops part-way, boot refuses until you run the same command again.

Before upgrading an existing instance to 1.4.0, set CORE_KEY_ENCRYPTION_KEY on it.

Boot checks, health and metrics

The hosted entry refuses to boot, naming the reason, when:

CORE_ALLOWED_MODES (default none, Control Mode only);

shorter than 32 characters or equal to the owner token;

GET /healthz answers without a credential with the service name and the owner-api version only. Only the hosted entry and the multi-tenant router serve it. GET /metrics serves Prometheus metrics to CORE_METRICS_TOKEN only, never to the owner token, and only when it is set, on a single core and on a multi-tenant deployment alike (metrics are recorded either way). Its series are deployment totals with no tenant, budget or stream label. The hosted entry limits each client to CORE_RATE_LIMIT_RPS requests per second (default 20) with a burst of CORE_RATE_LIMIT_BURST (default 40), and answers 429 past it.

The scheduler

The scheduler runs the duties the money layer depends on, so none of them waits for a client call. The hosted entry always runs it: it refuses to boot with CORE_SCHEDULER=off. The plain core-host entry accepts CORE_SCHEDULER=off and only prints a warning that nothing signs, settles or expires on its own; never run a deployment that way. Each duty has its own interval, in seconds:

DutyWhat it doesRegisteredVariableDefault
engine.<mode>due signing, policy checkpoints, checkpoint-band checkpoints, finality pollingone per funding modeCORE_SCHEDULER_TICK_SECONDS15
verified.<mode>verified-mode settlements and finalitywith CORE_VERIFIED=1CORE_SCHEDULER_VERIFIED_SECONDS30
control.<mode>approval timeouts, freeze expiries, backend revoke retriesone per funding modeCORE_SCHEDULER_CONTROL_SECONDS5
cardcredit card rail: renew holds near expiry, settle each epochwith the credit card railCORE_SCHEDULER_CARD_SECONDS300
engine.reservationsrelease reservations held longer than the reservation TTLalwaysCORE_SCHEDULER_RESERVATIONS_SECONDS60
escrow.release-held.<mode>release matured contract version 2 holdback to payees through the relayerwith contract version 2CORE_SCHEDULER_RELEASE_HELD_SECONDS60
rails.payee-claimsclaim payees' vouchers on an adopted rail before its deadlineonly where an adopted rail is wired into the coreCORE_SCHEDULER_RAILS_SECONDS30
rails.payer-exitsrefuse a payer exit while a voucher is unclaimedonly where an adopted rail is wired into the coreCORE_SCHEDULER_RAILS_SECONDS30
alertsalert evaluationalwaysCORE_SCHEDULER_ALERTS_SECONDS30
mailsend queued alert email, retry failed sendsalwaysCORE_SCHEDULER_MAIL_SECONDS30
deliverywebhook deliverywith EVENTS_DBEVENTS_DELIVER_INTERVAL_SECONDS5
maintenancerequest-log retention, session pruning, escrow reconciliationalwaysCORE_SCHEDULER_MAINTENANCE_SECONDS3600

These duties run on the alerts interval (CORE_SCHEDULER_ALERTS_SECONDS): main-switch (finish an interrupted pause or resume, then the automatic check) and tasks, always; payees.connect and card.payee-held with the credit card rail; kya.refresh when KYA is on; ramps.onramps when ramps are on; and vault.guard with a stablecoin mode. The hosted entry prints every registered duty with its interval on the scheduler on: boot line. No adopted rail is wired in by either entry from the environment.

CORE_CARD_RENEW_WITHIN_SECONDS (default 86400) sets how close to expiry a hold is renewed, and CORE_CARD_EPOCH_SECONDS (default 86400) sets the settlement epoch.

CORE_RESERVATION_TTL_SECONDS (default 3600, minimum 60) is how long a reservation may stay held when it has no per-hold expiry. The reservations duty releases older holds and sends a reservation.expired webhook event for each. A confirm that arrives after the release is billed if the stream's cap still allows it. Otherwise it is refused with reservation_expired, nothing is billed, and an alert fires.

CORE_MAX_HOLD_SECONDS (default 7 days) is the ceiling on an optional holdUntilSeconds passed at reserve (owner-api 1.11.0). A hold with holdUntil stays held past the reservation TTL until that instant, a verified settlement, or stream close; the duty skips it while holdUntil is still in the future. Boot refuses when CORE_MAX_HOLD_SECONDS is below the reservation TTL.

Alert email goes through a queue in the host database. Each alert sends at most one email. A failed send is retried with exponential backoff, from 3 minutes, up to 8 attempts over about six hours, then marked failed. A restart never drops a queued email. An email is recorded as sent right after the mail server accepts it, so only a crash in the moment between the two can send it a second time.

hour, and raises an alert.

take the lease and refuses to start (scheduler_conflict), so one instance is one process over one data directory.

fair loop visits CORE_TENANT_SCHEDULER_VISITS tenant stores (default 16) every CORE_TENANT_SCHEDULER_POLL_MS (default 1000).

process exits.

POST /owner/v1/scheduler/run runs the duties once, now.

Holdback

CORE_HOLDBACK_DEFAULT_SECONDS (default 300, allowed 0 to 86400) is the holdback window for verified-mode streams that do not set their own. Inside the window a refund executes against held funds. After it the units are final: the scheduler marks them final and fires settlement.final, and a later refund is refused by the protocol.

Backup and restore

Run both commands with the instance's own CORE_DB, CORE_KEY_ENCRYPTION_KEY and, when the instance uses one, EVENTS_DB. They need no owner token or wire token.

cd "$DYNAMO_REPO"
pwd
mkdir -p "$PWD/.dynamo-backups" "$PWD/.dynamo-restored"
CORE_DB="$PWD/.dynamo-local/core" \
  node deploy/image/entry.mjs backup "$PWD/.dynamo-backups/core-backup.bin"
# Stop the original instance, then:
CORE_DB="$PWD/.dynamo-restored/core" \
  node deploy/image/entry.mjs restore "$PWD/.dynamo-backups/core-backup.bin" --original-stopped

In the image the commands are node /app/server.cjs backup <file> and node /app/server.cjs restore <file> [--original-stopped]. The plain core-host entry has the same two commands for a single-tenant instance only.

(each funding mode, the host store, events), taken while it runs. It carries a SHA-256 digest per file and is encrypted with the key-encryption key.

that is altered, truncated or encrypted under another key.

same budgets and keys. A restore drops the scheduler lease it copied. The restored copy's first boot refuses until you confirm the original is stopped, either with restore <file> --original-stopped or by setting CORE_RESTORE_ORIGINAL_STOPPED=1 on that first boot. The confirmation is recorded in the host store, and later boots need nothing.

boot while the marker exists, and the next restore clears exactly the files the marker names before it starts.

A multi-tenant deployment

Run the same commands with CORE_TENANCY=multi (and CORE_TENANCY_ROOT when you set one). The bundle holds the tenant registry, every tenant's own stores and the events database. Each tenant's data key stays wrapped under CORE_KEY_ENCRYPTION_KEY inside the bundle; the commands never unwrap one.

with rekey <new-key-file> --backup <file> when you rotated the key. A wrong key refuses before anything is written.

deployment's first boot refuses until you confirm the original is stopped (--original-stopped, or CORE_RESTORE_ORIGINAL_STOPPED=1 on that boot); the confirmation covers every tenant, including those first opened later.

the other way round.

The control console

The console is served at /console when the instance finds the console bundle: booted from the repository, or with CORE_CONSOLE_DIR pointing at a copy of it. The container image does not carry the bundle, and its boot prints console: not served. Set CORE_CONSOLE=off to turn the console off. Sign in with the owner token.

CORE_TLS=1. Sessions last CORE_CONSOLE_SESSION_SECONDS (default 8 hours). Every control action needs the session's CSRF token.

itself only, and no request or response bodies are shown.

network map (state, frozen, stopped and pending approvals per node), the approval inbox, the request log, the revenue switchboard, scheduler status, reconciliation and alert settings.

(you type the budget id to confirm), tighten a cap or mandate (widening is refused in the form and by the core), approve and reject. Each one writes a signed authority record.

file (GET /owner/v1/budgets/{id}/evidence), which anyone can check offline with dynamo-verify.

Alerts

Alerts fire on cap thresholds (50, 80 and 100 percent of a budget's cap), spend velocity, a new payee, a mandate refusal, a failed settlement, an approval request, and a late reservation confirm refused after the TTL. Each budget's settings are at GET and PUT /owner/v1/budgets/{id}/alerts and on the budget page.

Alerts go out as signed webhook events (the events system) and, when SMTP is set, as email:

VariableMeaning
CORE_SMTP_HOSTSMTP server; email is off when unset
CORE_SMTP_PORTdefault 25, or 465 with CORE_SMTP_SECURE=tls
CORE_SMTP_SECUREnone (default) or tls
CORE_SMTP_USER, CORE_SMTP_PASSoptional login
CORE_SMTP_FROMsender address (required with a host)
CORE_ALERT_EMAIL_TOcomma-separated recipients (required with a host)

POST /owner/v1/webhooks/test {"type": "cap.threshold"} sends a signed test event, and so does dynamoprotocol trigger cap.threshold. Alert email names budget and stream ids, amounts and reason codes only.

Request log

GET /owner/v1/logs?since=&after=&limit= returns the payload-free request log: time, method, route, status, duration, ids and reason codes. It never holds bodies, credentials, tokens or raw IP addresses. Entries are kept for CORE_REQUEST_LOG_RETENTION_SECONDS (default 7 days). dynamoprotocol logs tail --follow follows it from a terminal.

Refusal retention

A refused delegation (a mandate, depth or category refusal) is recorded as an unsigned diagnostic row and listed under refusals in a budget's tree and network views. Rows older than CORE_REFUSAL_RETENTION_SECONDS (default 30 days, at least 7 days; a smaller value refuses to boot) are deleted by the hourly maintenance duty, with their alert keys. A client pinned at Dynamo-Version 1.11.0 sees the window as refusalsWindowSeconds. Signed authority records and evidence bundles never include these rows, so nothing signed is lost.

Admin audit and privacy

GET /owner/v1/admin-audit (owner-api 1.9.0) is the per-actor trail of console and admin actions. Retention is CORE_ADMIN_AUDIT_RETENTION_SECONDS (default 90 days). Personal-data export and deletion, and what signed records are kept, are in privacy.

Service-node protections

Each payee (service node) is protected from a payer's unsettled spend by the engine defaults: an unsettled-exposure ceiling of 1,000 USD, and headroom bands on the budget. With more than half of the budget left, work runs in full; between a quarter and a half, each request is size-limited; between a tenth and a quarter, a stream must be checkpointed before it takes more work; below a tenth, new work is refused. On every engine pass the scheduler checkpoints each stream in the checkpoint band that has unsettled work of its own. The engine pass runs every CORE_SCHEDULER_TICK_SECONDS (default 15 seconds), so in that band a request refused with checkpoint_pending is admitted again within one engine pass, with no checkpoint call from the client. The refusal carries a Retry-After header; the SDKs and the OpenAI Agents plugin wait and retry it for up to 30 seconds by default.

CORE_NODE_PROTECTIONS sets these per node: a JSON object of service-node address to a full protection set (money as decimal strings, in micro-USD):

{
  "0x6a4d0000000000000000000000000000000006a4": {
    "maxUnsettledExposure": "5000000",
    "maxConcurrentUnpaidWork": 32,
    "finalityRequiredAboveAmount": "100000000",
    "degradation": { "fullAbovePercent": 0, "throttledAbovePercent": 0, "checkpointRequiredAbovePercent": 0 },
    "throttleMaxUnitsPerRequest": "1000000"
  }
}

Turn the bands off only for nodes you run yourself, where payer and payee are the same operator and every call is reserved before it runs (the guard does this for its own node). Unknown keys, missing fields or bad thresholds refuse boot. Nodes not listed keep the defaults.

Other settings

the in-repo error pages).

for the escrow backend; later entries are used when earlier ones fail.

or 1337), how often the escrow backends poll for receipts, 10 to 4000 ms (default: the client's 4 s). With it set, a backend connected to any other chain refuses to start.

metrics, with payload-free attributes only.

escrow stream. A mismatch raises an alert and is never corrected automatically.

Try the whole flow

One command runs the operator flow end to end on a fresh local instance (Control Mode, no money, no key, a local mail sink, nothing sent outside the machine): a three-level agent network on the OpenAI Agents plugin with a scripted model, the console, the alert email and signed webhook at 80 percent, a freeze, an approval, an emergency stop, a kill -9 of the core with a reboot, and scheduler-driven finality.

cd "$DYNAMO_REPO"
pwd
make demo-human-control

It needs Node 22.12 or later, pnpm and Python 3.10 or newer; the first run provisions the plugin's virtual environment. Its script, scripts/human-control.mjs, shows every environment variable and console call it uses.

See known limits before you rely on any of this.