Market ingest (ops)

Status: slice 34 · October 2026.

UI: locus-ops /market — paste or upload JSON, set the target organizationId, and post it to the SoR for that tenant. This is how official market publications (BEAC / BVMAC bulletins, rate observations, treasury auction results, fitted curves) reach a bank's Market Radar, Morning brief, and curve views.

There is no built-in demo radar pack: a fresh tenant sees empty states until something is ingested. See the bank-side consumer in Monitoring zone and the producer-side flow in Market ingest flow.

Data path

Producer (ops UI | Python script | workflow_dispatch)
  → [ops] POST /api/market/ingest       (requireOpsSession, shape validation, organizationId required)
  → [SoR] POST /api/market/ingest        (MarketController.Ingest, Bearer session)
  → MarketBulletins · MarketObservations · AuctionLines · CurvePublications   (all tenant-scoped)
  → audit event MARKET_INGESTED
  → bank: GET /api/locus/market → GET /api/market/overview  (financial:read)

Who may call it

Caller	Permission	Target org
Tenant user	financial:write or tenant:manage	Own tenant only. Omitting organizationId is fine; passing the own id is fine; passing another id → 403 PERMISSION_DENIED
PLATFORM_OPERATOR (ops UI)	system:manage + tenant:manage	Any existing organization via organizationId; unknown id → 404 Target organization not found.

The ops UI additionally hides the form unless the session role is PLATFORM_OPERATOR or holds system:manage.

Payload (MarketIngestSchema)

Top-level object with an organizationId (ops route requires it) and at least one of four collections, each capped at 1000 rows:

Collection	Required fields	Optional	Persists to
bulletins[]	title	source (default BEAC), sha256Hash, publishedAt, payload	MarketBulletins
observations[]	seriesKey, value (number)	country (default CM), observedAt, meta	MarketObservations
auctions[]	tenor, marginalYield (number)	country (default CM), instrumentType (default BTA), auctionDate, payload	AuctionLines
curves[]	—	country, model (default NelsonSiegel), parameters, rmse, publishedAt	CurvePublications

Minimal example:

{
  "organizationId": "org_…",
  "bulletins": [
    {
      "title": "BEAC bulletin 2026-10",
      "source": "BEAC",
      "sha256Hash": "<64 hex chars>",
      "publishedAt": "2026-10-01T00:00:00Z",
      "payload": { "url": "https://www.beac.int/..." }
    }
  ],
  "observations": [
    { "seriesKey": "TIAO", "country": "CM", "value": 3.5, "observedAt": "2026-10-01T00:00:00Z" }
  ]
}

Normalization on the SoR: country codes upper-cased and clipped to 5 characters, strings clipped to column widths (titles 255, series keys 64, tenors 32), missing timestamps default to now (UTC), non-object payload/meta become {}.

Validation layers

1. Ops UI — must be valid JSON whose top level is an object.
2. Ops route (locus-ops/src/app/api/market/ingest/route.ts) — organizationId required (400 ORGANIZATION_REQUIRED); array/row-shape checks mirroring the bank's MarketIngestSchema (ops has no Zod dependency); 400 INVALID_PAYLOAD with details[] on failure; 502 SOR_UNAVAILABLE if the SoR cannot be reached.
3. SoR (MarketController) — re-validates required fields, the 1000-row cap, and "nothing to ingest".

Idempotency and dedupe

• Bulletins are de-duplicated on sha256Hash per tenant. A repeated hash is skipped and counted in skippedBulletins. Bulletins with an empty hash are always inserted, so always send the real digest of the source document.
• Observations, auctions, and curves are append-only. Re-running a producer duplicates them; use distinct files per run.

Successful response (201):

{ "organizationId": "org_…", "bulletins": 1, "skippedBulletins": 0, "observations": 1, "auctions": 0, "curves": 0 }

The ops page prints HTTP <status> followed by the response body in the Result panel.

Producers

Producer	Role
Ops UI /market	Manual paste/upload, Load sample button, per-tenant targeting
locus-workers/scripts/market_ingest_producer.py	CLI / cron. Reads a JSON file or stdin; --dry-run validates without network; --target-org org_… for operator ingestion. Env: LOCUS_API_BASE_URL, LOCUS_API_TOKEN (Bearer session), LOCUS_ORGANIZATION_ID. Exit codes: 0 success, 1 HTTP/request error, 2 usage/validation
.github/workflows/market-ingest.yml	workflow_dispatch only (inputs: repo-relative bulletin_file, dry_run default true). Secrets: LOCUS_API_BASE_URL, LOCUS_API_TOKEN, LOCUS_ORGANIZATION_ID. Scheduled ingest is intentionally cron on a trusted host, not GitHub Actions

Cron example (daily 06:00 UTC):

0 6 * * * LOCUS_API_BASE_URL=https://api.example LOCUS_API_TOKEN=... \
  LOCUS_ORGANIZATION_ID=org_... python3 /srv/locus/locus-workers/scripts/market_ingest_producer.py \
  /srv/locus/data/market/latest.json

What the bank sees

GET /api/locus/market (→ GET /api/market/overview, financial:read, tenant fail-closed) returns that tenant's bulletins, observations, auctions, curves, blotter, and alerts. The workspace turns them into:

• Market Radar / Research: morning brief, anomalies (z-score screen: |z| ≥ 2 watch, |z| ≥ 3 investigation signal; fewer than 3 points or σ = 0 → no score), auction ledger, official sources with SHA-256 fingerprints.
• Sovereign curves and the fixed-income terminal: published Nelson–Siegel parameters and RMSE, INSUFFICIENT_OBSERVATIONS when fewer than 4 liquid points.
• Alerts: MARCHÉ-severity items appear once market rows exist.

Market data is tenant-scoped: there is no global shared market table, so the same bulletin must be ingested for each tenant that should see it.

Quarantine and consistency rules (domain layer)

The TypeScript market domain (src/domains/market/) additionally quarantines malformed bulletins with structured diagnostics and enforces auction consistency (allotted ≤ tendered, marginal yield ≥ weighted-average yield). Those rules protect the radar logic; the raw SoR ingest accepts any well-formed row.

Deferred

Automatic download and parse of live BEAC / BVMAC bulletin URLs is still open. The ingest API, ops UI, Python producer, and manual workflow are ready to receive that producer.

Related: Ops overview · Market ingest flow · Data stores · Doc map.