litellm-dynamo

Meter and enforce allowances on every model call through a LiteLLM proxy you operate — without adding a network hop to the request path. Each caller presents an allowance credential as their API key; the plugin verifies it in-process against your pinned engine signer. No live credential, no completion: HTTP 402.

Payments are none of the plugin's business. The credential might be backed by a no-money Control ledger, USDC escrow, or a card authorization — the plugin neither knows nor cares (mode-agnostic by construction).

Install

Python ≥ 3.10; the LiteLLM pin is 1.61.20:

pip install litellm-dynamo

Two environment variables are mandatory (plus the wire bearer on hosted instances):

export DYNAMO_ENGINE_URL=https://your-instance-url   # your dynamo-core — ASYNC leg only
export DYNAMO_ENGINE_SIGNER=0xYourEngineSigner       # PINNED signer; verification is offline
export DYNAMO_ENGINE_TOKEN=your-plugin-wire-token    # bearer for /plugin/v1/* on hosted instances

Choose your token→units mapping — config, not code; your tokens map onto Dynamo's payload-neutral units:

export DYNAMO_UNITS_PER_PROMPT_TOKEN=1
export DYNAMO_UNITS_PER_COMPLETION_TOKEN=2
export DYNAMO_UNITS_PER_REQUEST=0

Then register the hook in a dynamo_hook.py beside your proxy config:

from litellm_dynamo import DynamoLiteLLMHook, config_from_env

dynamo_hook = DynamoLiteLLMHook(config_from_env())

The request path, precisely

  1. Pre-call (free): credential parsed and verified offline (JWS ES256K

against the pinned signer), expiry and revocation-at-last-sync checked, estimated units pre-checked against cached headroom. Any failure → 402 with an x402-shaped challenge body. The upstream is never touched.

  1. The model call proceeds only for a live, sufficient allowance.
  2. Post-call (async): actual usage maps to units and ticks to the engine

off the request path. Metering never blocks a response.

There is no fail-open configuration. Missing, stale, exhausted, expired, unverifiable, or revoked-at-last-sync → deny. The denial classes, their evaluation order, and the 402 challenge schema are identical across every gateway surface (Kong, APISIX) — enforced by a shared conformance vector set, so a caller cannot distinguish gateways by their denials.

Verify an install

The quickest live check is the guard — it composes exactly this plugin with a LiteLLM proxy against your instance and prints a capped base URL in one command. Under a minute: one completion serves (HTTP 200), spending past the cap answers HTTP 402, and the upstream is not touched after the halt.