Legal vault & Watchtower

Slices 33/34. The legal vault is the platform corpus of statutes (OHADA, SYSCOHADA/AUDCIF, BEAC, COSUMAF, …) managed by PLATFORM_OPERATOR in locus-ops. Banks only read it.

Principle

Discovery ≠ legal validity. Scraped PDFs enter review; only human APPROVE runs VaultPublisher.

Bank never writes the vault. Tenant uploads (deal-room PDFs) never become legal authority for other tenants, and LegalVaultController exposes only regulatory:read endpoints.

flowchart LR
  Src[RegulatorySources] --> Mon[SourceMonitor fingerprint]
  Mon -->|changed| Rev[ReviewQueue]
  Src --> Fetch[DocumentFetcher PDFs]
  Fetch --> Ext[DocumentExtractor PdfPig]
  Ext --> Rev
  Fetch -->|SHA mismatch same URL| Hash[HASH_CHANGE review]
  Rev -->|APPROVE| Pub[VaultPublisher]
  Pub --> Ed[Edition in_force / legal_reviewed]
  Ed --> Analyze[analyzeDealWithVault]
  Analyze --> Findings[DossierRegulatoryFindings]

Lifecycle: discovery → review → publish

sequenceDiagram
    autonumber
    actor Ops as PLATFORM_OPERATOR
    participant UI as locus-ops /regulatory
    participant SoR as locus-backend /api/admin/regulatory
    participant Web as Listing / PDF host
    participant DB as Postgres (platform tables)
    Ops->>UI: Check now / Check all
    UI->>SoR: POST sources/{id}/check · POST sources/check-all
    SoR->>Web: GET ListingUrl (45 s timeout)
    SoR->>SoR: SHA-256(content) vs LastContentFingerprint
    alt changed after a prior fingerprint
        SoR->>DB: LegalReviewItem Kind=WATCHTOWER_CHANGE, Status=PENDING
    else first fingerprint
        SoR->>DB: baseline only (no review spam)
    end
    Ops->>UI: Fetch PDFs
    UI->>SoR: POST sources/{id}/fetch-documents
    SoR->>Web: discover href *.pdf (≤ 20 per run), download (≤ 80 MB)
    SoR->>DB: LegalSourceArtifact (sha256, storage) + review Kind=WATCHTOWER_PDF
    opt known URL, new SHA-256
        SoR->>DB: review Kind=HASH_CHANGE (previous → new artifact)
    end
    Ops->>UI: Approve & publish (instrumentId or editionId)
    UI->>SoR: POST review/{id}/decide {decision: APPROVE}
    SoR->>DB: LegalVaultPublisher.PublishApprovedAsync
    SoR->>DB: new Edition in_force + legal_reviewed; prior in_force → historical
    SoR->>DB: LegalIngestionLogs PUBLISH + NOTIFY (+ EmailOutbox legal_vault_published)

Stage rules

Stage	Component	Rule
Monitor	LegalVaultWatchtower.CheckSourceAsync	manual method sources are not polled. HTTP error → Health=ERROR. Success → Health=HEALTHY, store LastContentFingerprint. Review is only enqueued when the fingerprint changed after a prior one
Stale	MarkStaleSourcesAsync	Enabled source with last success older than max(CheckIntervalMinutes × 2, 360) min → Health=STALE
Fetch	LegalVaultDocumentFetcher	Parses href="….pdf" on the listing; maxDownloads clamped to 1–20; max file 80 MB; user agent LocusWatchtower/1.0; never publishes in-force editions
Dedupe	fetcher	Known URL with only a review (no artifact) → skipped (ops hasn't decided); known URL with stored artifact → re-downloaded and compared by SHA-256
Drift	LegalVaultChangeDetector	Same URL, different SHA-256 → HASH_CHANGE review referencing previous + new artifact (once per new artifact)
Extract	LegalVaultDocumentExtractor (PdfPig)	Pages + candidate article headings stored in the review PayloadJson
Decide	POST review/{id}/decide	APPROVE, REJECT, NEEDS_CORRECTION; APPROVE requires instrumentId or editionId (or proposedInstrumentId on the item) else 400
Publish	LegalVaultPublisher	Creates …__edition_{yyyyMMddHHmm} with LegalStatus=in_force, VerificationStatus=legal_reviewed; links SupersedesEditionId; prior in_force → historical with EffectiveUntil (never deleted); unknown instrument → auto-stub (Authority=PLATFORM)
Provisions	publisher	If the edition has no provisions, up to 25 candidate provisions (VerificationStatus=imported, "awaiting legal review of full text") are attached from extract
Notify	LegalVaultNotificationService	NOTIFY log row; when LEGAL_VAULT_NOTIFY_EMAIL set, queues EmailOutbox template legal_vault_published

Review statuses: PENDING, NEEDS_CORRECTION, APPROVED, REJECTED. Review kinds: WATCHTOWER_CHANGE, WATCHTOWER_PDF, HASH_CHANGE.

Ops UI

locus-ops /regulatory: Source Registry · Legal Vault (upload PDF) · Review · Applicability · Logs. Operator guide: Regulatory admin.

Admin API (api/admin/regulatory, system:manage)

Method + path	Purpose
GET overview	Counts: sources, enabled, stale, instruments, editions, artifacts, provisions, amendments, applicability rules, reviewPending
GET sources · PATCH sources/{id}	Registry; enable/disable
POST sources/{id}/check · POST sources/check-all	Monitor (discovery only)
POST sources/{id}/fetch-documents	Download PDFs → review items
GET instruments · GET instruments/{id}	Vault instruments / editions
GET editions/{editionId}/provisions	Provisions of an edition
POST editions/{editionId}/artifact	Editions artifact API — register an immutable PDF already stored in R2/local vault
POST artifacts/{id}/extract · GET artifacts/{id} · GET artifacts/{id}/content	PdfPig extract; metadata; content
GET amendments · GET editions/{editionId}/amendments · POST amendments	Amendments
GET/POST applicability · PUT/DELETE applicability/{id}	LegalProvisionApplicability CRUD
GET review · POST review · POST review/{id}/decide	Review queue (manual enqueue for ops testing)
GET logs · POST logs/{id}/retry	Ingestion logs

Editions artifact API

POST /api/admin/regulatory/editions/{editionId}/artifact — the ops BFF uploads bytes (legal-vault/{sha256}/{file} on R2, or local: path), then registers metadata:

{
  "sha256": "<64 lowercase hex>",
  "storageKey": "legal-vault/<sha256>/ohada-aus-2010.pdf",
  "filename": "ohada-aus-2010.pdf",
  "byteSize": 1234567,
  "officialUrl": "https://…",
  "discoveredFromSourceId": null,
  "pagesCount": 42
}

Rule	Detail
Validation	sha256 must be 64 lowercase hex (400); storageKey required
Idempotency	Duplicate SHA-256 reuses the existing artifact row (keeps its original storageKey, logs a warning)
Artifact id	art_{first 16 hex of sha}
Side effects	edition.SourceArtifactId set; VerificationStatus imported → source_verified; PdfPig extract attempted for local artifacts; LegalIngestionLogs INGEST; audit LEGAL_ARTIFACT_REGISTER
Review decide audit	REGULATORY_REVIEW_DECIDE

Edition status model

Field	Values (as seen in code)
LegalStatus	imported / under_review → in_force → historical
VerificationStatus	imported → source_verified → legal_reviewed

Catalog evaluation still runs on catalog text until vault text is legal_reviewed.

Tenant consume (read-only)

BFF route	SoR route	Permission
GET /api/locus/legal-vault/resolve?instrumentId&provisionNumber	GET /api/legal-vault/resolve	regulatory:read
GET /api/locus/legal-vault/artifacts/[id]/content	GET /api/legal-vault/artifacts/{id} + /content	regulatory:read

• resolve returns instrumentId, instrumentTitle, authority, editionId, legalStatus, verificationStatus, provisionId/Number/Title/Text, pageStart/End, artifactId, sha256, officialUrl, hasArtifact, openStatutePath. It prefers the edition with an artifact, then the most recent EffectiveFrom.
• Open-statute BFF streams the artifact (hash verified): SoR streams local: files and returns 409 HASH_MISMATCH if the digest differs; for object storage it returns USE_BFF_PROXY and the BFF fetches from R2 with shared credentials, verifying SHA-256.
• Catalog/analyzeDeal still evaluates rules; vault IDs overlay evidence (Regulatory analyze).
• Findings persistence: PUT /api/deals/{dealId}/regulatory-findings (replace-all, dossier:write) and GET (regulatory:read) via BFF /api/locus/deals/[id]/regulatory-findings.

DossierRegulatoryFindings columns

FindingKey, Type, Severity, Status, Title, Detail, Framework, RegulationId, ArticleNumber, StatutoryExcerpt, VaultEditionId, VaultProvisionId, VaultArtifactId, VaultSha256, EvidenceJson — Vault* ids let an audit replay show exactly which edition/artifact was cited.

Scheduler

LEGAL_VAULT_WATCHTOWER_INTERVAL_MINUTES > 0 → hosted check-all loop on SoR (LegalVaultWatchtowerHostedService). Default off (0). First tick after one full interval. Each tick: bootstrap ensure → check every enabled source → MarkStaleSourcesAsync; discovery only, never publishes. Alternative: external cron calling POST /api/admin/regulatory/sources/check-all with a PLATFORM_OPERATOR session.

LEGAL_VAULT_* environment

Variable	App	Role
LEGAL_VAULT_STORAGE_PATH	locus-backend	Root for fetched PDFs (default ./data/legal-vault)
LEGAL_VAULT_WATCHTOWER_INTERVAL_MINUTES	locus-backend	Scheduler interval; 0/unset = disabled
LEGAL_VAULT_NOTIFY_EMAIL	locus-backend	Recipient for publish notifications (needs a real org)
LEGAL_VAULT_LOCAL_DIR	locus-ops	Local PDF upload dir when R2 is not configured
R2_*	locus-ops	Object storage for uploaded vault PDFs

Bootstrap

LegalVaultBootstrap.EnsureAsync (run by admin overview and resolve) idempotently seeds monitor sources, instrument stubs for ids used by analyzeDeal (e.g. ohada-aus-2010), and imported applicability rows for key provisions (e.g. cosumaf-rg-2014-art-4, beac-reg-changes-2018-art-42, syscohada-audcif-2017-art-17, ohada-aus-2010-art-138). It never uploads PDFs and never overwrites ops edits.

Fixtures

CI synthetic PDFs: Regulatory fixtures. Production: upload official PDFs in ops.