Gateway outbox worker, MQ wrap, thirdweb adapter updates, and SolaceNet docs.

Made-with: Cursor
2026-04-12 06:11:37 -07:00
parent 6ebf71dda8
commit ae744e4be4
23 changed files with 775 additions and 16 deletions
--- a/docs/solacenet/DBIS_RAIL_SCOPE_DEFERRAL.md
+++ b/docs/solacenet/DBIS_RAIL_SCOPE_DEFERRAL.md
@@ -0,0 +1,13 @@
+# DBIS Rail on-chain scope — formal deferral (repository record)
+
+**Gap IDs:** PG-RAIL-001 through PG-RAIL-005 (and dependent mint path PG-RAIL-005).
+
+**Status:** **Deferred** in this codebase until a chartered program funds contract development, security review, and deployment to target chains.
+
+**What is not delivered here:** `DBIS_RootRegistry`, `DBIS_ParticipantRegistry`, `DBIS_SignerRegistry`, `DBIS_SettlementRouter`, `DBIS_GRU_MintController` (router-only mint) as live deployed contracts wired to production services.
+
+**Authoritative planning doc (Proxmox repo):** `docs/dbis-rail/DBIS_RAIL_AND_PROJECT_COMPLETION_MASTER_V1.md`.
+
+**Narrative rule:** Docs and UIs must **not** imply these contracts are live. Off-chain ISO / MintAuth work (PG-RAIL-006, PG-RAIL-007) may proceed in parallel where specs exist.
+
+**Last updated:** 2026-04-08
--- a/docs/solacenet/FILE_DROP_GATEWAY_INGRESS.example.md
+++ b/docs/solacenet/FILE_DROP_GATEWAY_INGRESS.example.md
@@ -0,0 +1,21 @@
+# File-drop ingress — SolaceNet gateway enforcement (example)
+
+**Gap:** PG-GW-W03 — file workers are app-specific; this pattern shows how to call the shared guard.
+
+```typescript
+import { requireGatewayMicroservicesForWorker } from '@/core/gateway/rails/gateway-rails-enforcement';
+import fs from 'fs/promises';
+
+async function processInboundFile(path: string, tenantId: string, adapterId: string) {
+  await requireGatewayMicroservicesForWorker({
+    tenantId,
+    ingressKind: 'file-drop',
+    detail: path,
+    adapterId,
+  });
+  const raw = await fs.readFile(path, 'utf8');
+  // ... validate / ingest ...
+}
+```
+
+Set `SOLACENET_GATEWAY_RAILS_ENFORCE=1` in the worker environment when policy should apply.
--- a/docs/solacenet/GATEWAY_ADAPTER_LIFECYCLE.md
+++ b/docs/solacenet/GATEWAY_ADAPTER_LIFECYCLE.md
@@ -0,0 +1,11 @@
+# Gateway rail adapter lifecycle (DI policy)
+
+**Gap:** PG-GW-W01 (singleton vs per-request).
+
+**Policy (until superseded by performance review):**
+
+1. **HTTP (`/api/v1/gateway/rails/*`)** — `createGatewayRailAdapter(id)` returns a **new instance per request** after `registry` lookup. Do **not** cache adapter instances on the Express router.
+2. **Long-lived workers** — if a worker holds a connector with sockets or session state, that worker process owns **one** instance per `adapterId`; document in the worker module.
+3. **Secrets** — `initialize(config, secretsRef)` is the hook for KMS/HSM-backed config; REST routes still pass `{}` until per-rail secrets routing is implemented.
+
+**Related:** `gateway-adapter-registry.ts`, `gateway.routes.ts`.
--- a/docs/solacenet/PROTOCOL_GAPS_CHECKLIST.md
+++ b/docs/solacenet/PROTOCOL_GAPS_CHECKLIST.md
@@ -8,6 +8,7 @@
 |--------|---------|
 | `open` | Not implemented or only stub/placeholder |
 | `partial` | Some path exists; production contract or wiring incomplete |
+| `deferred` | Explicit program deferral recorded (see linked doc); not “open” for sprint pickup |
 | `n/a` | Explicitly out of scope for this repo (document boundary only) |
 | `done` | Production-capable per defined gate (update row when true) |

@@ -83,11 +84,11 @@ Source of truth for deployment truth: `docs/dbis-rail/DBIS_RAIL_AND_PROJECT_COMP

 | ID | Protocol / contract layer | Status | Gap summary |
 |----|---------------------------|--------|-------------|
-| PG-RAIL-001 | **DBIS_RootRegistry** | `open` | Contract set not implemented in repo |
-| PG-RAIL-002 | **DBIS_ParticipantRegistry** | `open` | Same |
-| PG-RAIL-003 | **DBIS_SignerRegistry** | `open` | Same |
-| PG-RAIL-004 | **DBIS_SettlementRouter** | `open` | Same |
-| PG-RAIL-005 | **DBIS_GRU_MintController** (router-only mint) | `open` | Mint path still owner-mint on c* per master doc |
+| PG-RAIL-001 | **DBIS_RootRegistry** | `deferred` | [DBIS_RAIL_SCOPE_DEFERRAL.md](DBIS_RAIL_SCOPE_DEFERRAL.md) — not in active implementation until program charter |
+| PG-RAIL-002 | **DBIS_ParticipantRegistry** | `deferred` | Same |
+| PG-RAIL-003 | **DBIS_SignerRegistry** | `deferred` | Same |
+| PG-RAIL-004 | **DBIS_SettlementRouter** | `deferred` | Same |
+| PG-RAIL-005 | **DBIS_GRU_MintController** (router-only mint) | `deferred` | Same; owner-mint vs router mint per master doc when un-deferred |
 | PG-RAIL-006 | **Production ISO Gateway** matching EIP-712 MintAuth pipeline | `partial` | Spec/rulebook complete; production service + on-chain registry not aligned |
 | PG-RAIL-007 | **MintAuth relayer** operational hardening | `partial` | Documented; full ops gate open |

@@ -138,7 +139,7 @@ Below: rows that are **protocol, endpoint, or message-contract** heavy and **not
 |----|-----|--------|------------|
 | PG-SN-001 | `gateway-microservices` capability describes rails; **adapters remain scaffolds** | `open` | Critical rails reach `done` in section B or scope reduced in offering JSON |
 | PG-SN-002 | `solacenet_provider_connector` rows for each live rail provider | `partial` | **Seed:** `npm run seed:gateway-provider` (`dbis-gateway-rail-plane`). **Remaining:** per-institution connectors + bindings |
-| PG-SN-003 | smom-dbis-138-publish / tokenization docs: **SolaceNet policy** integration called out as future in places | `partial` | Code paths call `requireCapability` or documented exception |
+| PG-SN-003 | smom-dbis-138-publish / tokenization vs SolaceNet | `partial` | **Doc:** [TOKENIZATION_SMOM_AND_SOLACENET.md](TOKENIZATION_SMOM_AND_SOLACENET.md). **Close when:** smom flows hit `dbis_core` tokenization APIs and entitlements are enforced there |

 ---

--- a/docs/solacenet/REMAINING_TASKS_FULL_LIST.md
+++ b/docs/solacenet/REMAINING_TASKS_FULL_LIST.md
@@ -21,7 +21,7 @@
 |---|------|-----------|--------|
 | 1 | **Reconcile marketplace/offering copy** with live connector maturity — or scope down `gateway-microservices-offering.json` until rails are real | PG-SN-001, offering JSON | **Updated 2026-04-08:** description names scaffolds + KTT evidence semantics; keep row until production connectors ship. |
 | 2 | **Document KTT explicitly** as evidence-only in marketplace/UI copy; separate from symmetric send/receive rails | PG-GW-008, inconsistencies doc | Code already fails `send` by design |
-| 3 | **Decide DBIS Rail scope:** implement contracts + router-only mint **or** formally defer and update all docs that imply live Rail | PG-RAIL-001–005, master doc | See `docs/dbis-rail/DBIS_RAIL_AND_PROJECT_COMPLETION_MASTER_V1.md` |
+| 3 | **Decide DBIS Rail scope:** implement contracts + router-only mint **or** formally defer and update all docs that imply live Rail | PG-RAIL-001–005, master doc | **Deferred (recorded):** [DBIS_RAIL_SCOPE_DEFERRAL.md](DBIS_RAIL_SCOPE_DEFERRAL.md) + checklist `deferred`. Planning: `docs/dbis-rail/DBIS_RAIL_AND_PROJECT_COMPLETION_MASTER_V1.md` |
 | 4 | **Fix or quarantine Thirdweb adapter** type issues (`ethers` namespace) for clean `tsc` / CI | PG-GW-009 | **Partial:** `ethers` dependency + `import type { Provider, Signer }`; full `dbis_core` `tsc` may still fail elsewhere. |

 ---
@@ -30,12 +30,12 @@

 | # | Task | Gap ID(s) | Notes |
 |---|------|-----------|--------|
-| 5 | **DI / lifecycle policy** for rail adapters (singleton vs per-request) and document | PG-GW-W01 | Registry exists; avoid accidental state bleed |
-| 6 | **Orchestrated `send` path** (outbox, idempotency, DLQ) for rails that support outbound messages | PG-GW-W01, PG-GW-001–007 | KTT may stay receive-only |
-| 7 | **Extend `SOLACENET_GATEWAY_RAILS_ENFORCE`** to every rail ingress (file drop, MQ, future workers) + **audit log** export for denied/allowed | PG-GW-W03 | HTTP path partially done |
-| 8 | **Optional code facade** for PluginRegistry ↔ gateway adapters when one physical connector serves both | PG-GW-W02 | Doc exists; implement if needed |
+| 5 | **DI / lifecycle policy** for rail adapters (singleton vs per-request) and document | PG-GW-W01 | **Doc:** [GATEWAY_ADAPTER_LIFECYCLE.md](GATEWAY_ADAPTER_LIFECYCLE.md) |
+| 6 | **Orchestrated `send` path** (outbox, idempotency, DLQ) for rails that support outbound messages | PG-GW-W01, PG-GW-001–007 | **Partial:** [gateway-outbox.service.ts](../../src/core/gateway/control/gateway-outbox.service.ts) + [gateway-outbox.worker.ts](../../src/workers/gateway-outbox.worker.ts) (`npm run worker:gateway-outbox`); app-level CAS claim; production rails + optional DB `SKIP LOCKED` still TBD |
+| 7 | **Extend `SOLACENET_GATEWAY_RAILS_ENFORCE`** to every rail ingress (file drop, MQ, future workers) + **audit log** export for denied/allowed | PG-GW-W03 | HTTP + audit + [gateway-mq-wrap.ts](../../src/core/gateway/rails/gateway-mq-wrap.ts) + `MessageBusService.buildGatewayAwareHandler`; file-drop = app-specific |
+| 8 | **Optional code facade** for PluginRegistry ↔ gateway adapters when one physical connector serves both | PG-GW-W02 | **Stub:** [plugin-gateway-facade.ts](../../src/core/gateway/integration/plugin-gateway-facade.ts) |
 | 9 | **Register `solacenet_provider_connector`** rows per environment for each live provider | PG-SN-002 | Tie to capability bindings |
-|10 | **smom-dbis-138-publish / tokenization:** wire `requireCapability` or document explicit exception | PG-SN-003 | |
+|10 | **smom-dbis-138-publish / tokenization:** wire `requireCapability` or document explicit exception | PG-SN-003 | **Doc:** [TOKENIZATION_SMOM_AND_SOLACENET.md](TOKENIZATION_SMOM_AND_SOLACENET.md) |

 ---

@@ -166,6 +166,7 @@ Besu, Explorer, Caliper benchmarks, and other **non-protocol** RTGS matrix rows
 - [public/GAPS_AND_INCONSISTENCIES.md](public/GAPS_AND_INCONSISTENCIES.md) — narrative gaps  
 - [RAIL_AND_PROTOCOL_GOVERNANCE.md](RAIL_AND_PROTOCOL_GOVERNANCE.md)  
 - [PLUGIN_AND_GATEWAY_BRIDGE.md](PLUGIN_AND_GATEWAY_BRIDGE.md)  
+- [TASK_BACKLOG_COMPLETION_STATUS.md](TASK_BACKLOG_COMPLETION_STATUS.md) — repo-done vs external vs deferred  
 - Proxmox: `docs/03-deployment/DBIS_RTGS_E2E_REQUIREMENTS_MATRIX.md`  
 - Proxmox: `docs/dbis-rail/DBIS_RAIL_AND_PROJECT_COMPLETION_MASTER_V1.md`  
 - Proxmox: `docs/04-configuration/SOLACENET_PUBLIC_HUB.md`
--- a/docs/solacenet/SOLACENET_GATEWAY_RAILS_ENFORCE_RUNBOOK.md
+++ b/docs/solacenet/SOLACENET_GATEWAY_RAILS_ENFORCE_RUNBOOK.md
@@ -23,6 +23,11 @@
 4. **Smoke test with entitlement:** same call with an entitled tenant — expect **200** and health JSON.
 5. **Confirm list endpoint:** `GET /api/v1/gateway/rails` remains **200** (metadata only; still requires normal API auth).

+**Gateway outbox worker (send scaffold)**
+
+- `npm run worker:gateway-outbox` — processes `gateway_outbox` via rail `send` (scaffold until live wires). Env: `GATEWAY_OUTBOX_BATCH_SIZE`, `GATEWAY_OUTBOX_MAX_ATTEMPTS`.
+- Example systemd unit (parent repo): `config/systemd/dbis-core-gateway-outbox-worker.example.service` — copy, set `WorkingDirectory` + `EnvironmentFile` with `DATABASE_URL`, then `systemctl enable --now`.
+
 **Rollback**

 - Unset `SOLACENET_GATEWAY_RAILS_ENFORCE` or set to `0` / `false` and restart the API.
--- a/docs/solacenet/TASK_BACKLOG_COMPLETION_STATUS.md
+++ b/docs/solacenet/TASK_BACKLOG_COMPLETION_STATUS.md
@@ -0,0 +1,113 @@
+# Backlog completion status (honest matrix)
+
+**Purpose:** Map `REMAINING_TASKS_FULL_LIST.md` items (and related audit bullets) to **what the repository can and cannot close** without external systems, contracts, or multi-sprint hardening.
+
+**Legend**
+
+| Status | Meaning |
+|--------|--------|
+| **Repo-done** | Addressed in code or canonical docs in this monorepo / `dbis_core`. |
+| **Repo-partial** | Scaffolding, policy doc, or automation exists; production gate still open. |
+| **External** | Requires SWIFTNet, DTCC, bank APIs, carrier CPE, jurisdiction endpoints, or paid cloud contracts. |
+| **Deferred** | Explicit product/program deferral recorded in repo (implement later or never). |
+
+---
+
+## P0 (tasks 1–4)
+
+| # | Status | Evidence / blocker |
+|---|--------|-------------------|
+| 1 | Repo-partial | Marketplace copy updated; **close** when live connectors exist or offering scoped further. |
+| 2 | Repo-partial | Narrative in offering + public gaps; **close** when all UI surfaces reviewed. |
+| 3 | **Deferred** | [DBIS_RAIL_SCOPE_DEFERRAL.md](DBIS_RAIL_SCOPE_DEFERRAL.md) — on-chain Rail contracts not in scope until chartered. |
+| 4 | Repo-partial | `ethers` + types on Thirdweb adapter; **full `tsc`** still blocked by unrelated modules (see `npx tsc --noEmit`). |
+
+---
+
+## P1 Gateway core (5–10)
+
+| # | Status | Evidence / blocker |
+|---|--------|-------------------|
+| 5 | Repo-done | [GATEWAY_ADAPTER_LIFECYCLE.md](GATEWAY_ADAPTER_LIFECYCLE.md) |
+| 6 | Repo-partial | [gateway-outbox.service.ts](../../src/core/gateway/control/gateway-outbox.service.ts) + [gateway-outbox.worker.ts](../../src/workers/gateway-outbox.worker.ts) + `npm run worker:gateway-outbox` (CAS claim + `adapter.send`). |
+| 7 | Repo-partial | HTTP + audit + MQ wrap + [FILE_DROP_GATEWAY_INGRESS.example.md](FILE_DROP_GATEWAY_INGRESS.example.md). |
+| 8 | Repo-partial | [plugin-gateway-facade.ts](../../src/core/gateway/integration/plugin-gateway-facade.ts) stub — implement when dual-surface connector exists. |
+| 9 | Repo-partial | `npm run seed:gateway-provider`; per-institution rows + bindings = **External / ops**. |
+| 10 | Repo-partial (doc) | [TOKENIZATION_SMOM_AND_SOLACENET.md](TOKENIZATION_SMOM_AND_SOLACENET.md) — exception until cross-service API calls exist. |
+
+---
+
+## P1 Rail scaffolds (11–19)
+
+| # | Status | Blocker |
+|---|--------|--------|
+| 11–17 | **External** | Institution contracts, network access, certification. |
+| 18 | **External** | SoR integration, persistence design, bank agreement. |
+| 19 | Repo-partial | Thirdweb encode path still placeholder — mark non-prod in adapter doc / complete encode = **External** (ABI per deployment). |
+
+---
+
+## P1 DBIS Rail off-chain (20–21)
+
+| # | Status | Blocker |
+|---|--------|--------|
+| 20–21 | **External** | Production deployment, keys, monitoring, counterparty test environments. |
+
+---
+
+## P1 RTGS spine (22–34)
+
+| # | Status | Blocker |
+|---|--------|--------|
+| 22–34 | **External** | See `docs/03-deployment/DBIS_RTGS_E2E_REQUIREMENTS_MATRIX.md` — live endpoints, legal packages, multi-party acceptance. |
+
+---
+
+## P2 Hyperledger / institutional (35–50)
+
+| # | Status | Blocker |
+|---|--------|--------|
+| 35–50 | **External** | Topology decisions, vendors, custody/CSD/FX counterparties. |
+
+---
+
+## P2 Quantum / QPS / admin (51–56)
+
+| # | Status | Evidence / blocker |
+|---|--------|-------------------|
+| 51–52 | Repo-partial | Types + example telecom schema; **real scoring / boundary** = External. |
+| 53–56 | **External** | Real telemetry and bank data feeds into dashboards. |
+
+---
+
+## P3 Carrier (57)
+
+| # | Status | Blocker |
+|---|--------|--------|
+| 57 | **External** | Carrier boundary service + regulated interconnect. |
+
+---
+
+## P3 Hygiene (58–63)
+
+| # | Status | Evidence / blocker |
+|---|--------|-------------------|
+| 58 | Repo-done | Labeled in [public/GAPS_AND_INCONSISTENCIES.md](public/GAPS_AND_INCONSISTENCIES.md) §2. |
+| 59–62 | Repo-partial / Ops | Heuristics remain until data model exists; verify scripts are ongoing. |
+| 63 | Ongoing | Process — update [PROTOCOL_GAPS_CHECKLIST.md](PROTOCOL_GAPS_CHECKLIST.md) as gates close. |
+
+---
+
+## Audit “remaining” bullets (post-2026-04-08)
+
+| Topic | Status |
+|-------|--------|
+| Outbox / DLQ / idempotency (full) | Repo-partial — DB tables + enqueue helper; **worker + exactly-once** = further work. |
+| Per-tenant rate limits | Repo-partial — rail mutate limiter keys on **`ip:tenant`** (`gateway.routes.ts`). |
+| Full `dbis_core` `tsc` | **External** to this sprint — hundreds of errors across unrelated modules; fix incrementally. |
+| SIEM-signed audit | **External** — log shipping + tamper-evident store. |
+| Go ↔ Node single edge | Repo-partial — documented; shared artifact optional. |
+
+---
+
+**Last updated:** 2026-04-08
--- a/docs/solacenet/TOKENIZATION_SMOM_AND_SOLACENET.md
+++ b/docs/solacenet/TOKENIZATION_SMOM_AND_SOLACENET.md
@@ -0,0 +1,11 @@
+# Chain 138 tokenization (smom-dbis-138) vs SolaceNet (`dbis_core`)
+
+**Gap:** PG-SN-003 — smom-dbis-138 publish / tokenization paths do not call `dbis_core` `requireCapability`.
+
+**Status:** **Documented exception.** Token mint/deploy and token-aggregation flows live primarily in **`smom-dbis-138`** (contracts, scripts, services). SolaceNet capability checks apply to **`dbis_core` HTTP/API** surfaces (e.g. `/api/v1/solacenet/tokenization/*`, gateway rails).
+
+**When to add checks:** If a future **cross-service** flow posts from smom automation into `dbis_core` tokenization APIs, add `requireCapability` at those API boundaries and record entitlements per tenant.
+
+**References:** `smom-dbis-138/docs/tokenization/`, `dbis_core/src/core/solacenet/capabilities/tokenization/`.
+
+**Last updated:** 2026-04-08
--- a/docs/solacenet/public/GAPS_AND_INCONSISTENCIES.md
+++ b/docs/solacenet/public/GAPS_AND_INCONSISTENCIES.md
@@ -24,7 +24,7 @@
 |--------|--------|
 | **Dual adapter systems** | **Mitigated (doc):** [PLUGIN_AND_GATEWAY_BRIDGE.md](../PLUGIN_AND_GATEWAY_BRIDGE.md). **Remaining:** optional facade in code if one physical connector serves both shapes. |
 | **KTT vs other rails** | Marketplace lists **KTT Legacy** next to full **send/receive** rails; code is **evidence-only** and **send** fails by design. |
-| **Synthetic success** | SWIFT FIN / TT scaffolds return **SENT** without network I/O; **KTT** returns **FAILED** on send—mixed semantics for “outbox” testing. |
+| **Synthetic success** | SWIFT FIN / TT scaffolds return **SENT** without network I/O; **KTT** returns **FAILED** on send—mixed semantics for “outbox” testing. **Operator rule:** dashboards and demos must label these paths **scaffold / non-wire** so they are not read as production ACKs. |
 | **Thirdweb in gateway folder** | Shares adapter plane with **bank rails**; not a messaging rail—boundary should stay documented for integrators. |
 | **Offering copy vs reality** | `gateway-microservices-offering.json` describes a regulated fabric; **implementations are not production-grade** connectors yet. |
 | **QPS dashboard “SWIFT” volume** | Derived from `iso_messages` heuristics (`messageType.includes('SWIFT')`)—**not** true SWIFT FIN volume. |