zeromodel

Post-c203e7a7 remediation — Stage C: closed report aggregate, complete report recovery, and auditable trust receipts

Baseline SHA: c203e7a7dddd2f8d3bad1a41e46c18a283c6b163 on main (“fix: make compiled report artifacts reloadable and auditable”). Final working-tree state: uncommitted — every change below is staged in the working tree, pending an explicit commit decision. Objective: complete the report-artifact architecture so a compiled report can be loaded as one coherent, fully recoverable and auditable aggregate, and give Trust decisions a real content-derived signature-envelope identity instead of the raw signature value.

1. Why this stage was required

Stage B (adapter-first report compilation) and its e101b06 review response fixed VPM view-coordinate bindings, Core artifact persistence, compiled-artifact internal closure, deep manifest freezing, metric compatibility schema identity, and Trust decision audit fields. But AdaptedReportDTO — the object carrying raw values, per-value confidence, per-value importance, source bindings, and parent-report lineage — still only ever existed as a discarded local Python variable inside compile_report(). CompiledReportArtifactDTO named it by a bare digest string (adapted_report_id: str), not a resolvable reference. And even with all four objects (adapted report, ScoreTable, LayoutRecipe, VPMArtifact) individually persisted and digest-valid, nothing proved they described the same report — a self-validating compiled report could reference a ScoreTable from report A, a LayoutRecipe from report B, and a VPMArtifact from report C, every one individually valid.

This stage distinguishes individual artifact integrity (a correct digest) from aggregate semantic closure (the collection actually describes one report), and closes the gap between them.

2. Files added

3. Files modified

4. Public API changes

New exports from zeromodel.artifacts:

ADAPTED_REPORT_ARTIFACT_KIND, ResolvedCompiledReportAggregateDTO,
CompiledReportClosureReceiptDTO, store_adapted_report, load_adapted_report,
load_compiled_report_aggregate, validate_compiled_report_aggregate,
build_compiled_report_closure_receipt, compute_report_semantics_id

CoreArtifactRefs was already public (Stage B/e101b06); ReportSemanticsInfo and CompatibilityInfo remain deliberately unexported implementation detail, reachable via the compiled_artifact submodule for tests only — the same restricted-public-API pattern used throughout this workspace.

New exports from zeromodel.trust:

compute_signature_envelope_id, signature_envelope_identity_payload

docs/architecture/package-public-api-1.0.13.csv was hand-updated (not regenerated via scripts/validate_release_candidate.py — see §12) to add these rows in the file’s existing (distribution, exported_symbol) sort order; tests/test_public_api_manifest.py’s fast subset (15 tests, all passing) verifies every row against the real __all__ and rejects any manifest/__all__ mismatch, so a hand-edit that drifted from reality would have failed loudly.

5. Artifact-kind and spec-version decisions

ADAPTED_REPORT_ARTIFACT_KIND = "zeromodel.artifacts.adapted-report/v1" is a new kind — no existing persisted v1 payload’s identity semantics changed. CompiledReportArtifactDTO keeps its existing kind and spec version (zeromodel.artifacts.compiled-report/v1) even though its canonical identity payload gained fields (adapted_report_ref in place of adapted_report_id, plus the five report-semantics fields): this is judged safe without a version bump because no compiled-report artifact from this kind has ever been durably persisted outside a test process — InMemoryArtifactStore is explicitly non-durable, and this repository has shipped no production consumer of the prior shape. The identity function changing what its digest covers is therefore not “silently reinterpreting an existing persisted v1 payload” (the case §4.5 of the brief prohibits without a version bump); it is completing a v1 contract that was never actually durable. This decision is recorded here explicitly per that same section’s requirement to explain, not silently make, this call.

zeromodel-artifacts-compiled-report-aggregate/v1 is the new spec_version for CompiledReportClosureReceiptDTO (aggregate.py) — a wholly new artifact kind, not a migration.

6. Adapted-report canonical format

The canonical payload stored for an adapted report is exactly adapted_report_signing_payload(adapted_report) — the same payload AdaptedReportDTO.adapted_report_id is already computed over, with the self-referential id field excluded. Because ArtifactStore.put() computes ref.artifact_id = sha256_digest(canonical_bytes) over those same canonical bytes, ref.artifact_id == adapted_report.adapted_report_id holds unconditionally — the store’s content identity and the DTO’s own self-validated identity are literally the same digest, not two independently-computed values that happen to agree (this was the brief’s §5.1 requirement). load_adapted_report follows the established decode-and-verify pattern: resolve canonical bytes → recompute digest → require equality with the requested ArtifactRef.artifact_id → decode → reconstruct via AdaptedReportDTO’s own __post_init__ self-validation. The manifest is never read (proven by test_loading_does_not_use_the_manifest_as_authority, using the same “manifest-forbidden resolver” pattern established in Stage B).

7. Compiled-report identity changes

CompiledReportArtifactDTO.adapted_report_ref: ArtifactRef replaces the former adapted_report_id: str; a computed adapted_report_id property (self.adapted_report_ref.artifact_id) preserves the old read-only access pattern for any caller that only wants the digest. The identity payload now binds the adapted report’s full ArtifactRef (kind + id via _artifact_ref_payload), not just a digest string — matching how the three Core refs were already bound.

Five new fields close the “report/subject semantics” gap the brief’s §9.8 identified: report_kind, subject_kind, dimension_namespace, duplicate_value_semantics, and a self-validated report_semantics_id digest over those four. Design rationale in §8 below.

Regression, strengthened per the brief’s §13: test_cell_bindings_use_view_coordinates_not_source_order (in test_report_compilation.py) previously used ai_artifact_family, whose source order already happened to coincide with the sorted (descending) order — view_row == source_row_index held for every cell even with the exact bug the test claimed to catch, because the fixture’s row 0 already had the highest score. It has been rewritten to use quality_family (sentence-001=0.4, sentence-002=0.95 — a genuine inversion under a descending sort) and an explicit reversed column order (source declares quality, clarity; the recipe requests clarity, quality). The test now asserts view_row != source_row_index and view_column != source_metric_index explicitly for concrete cells, not merely infers correctness from subject names.

8. Aggregate object model

@dataclass(frozen=True, slots=True)
class ResolvedCompiledReportAggregateDTO:
    compiled_report: CompiledReportArtifactDTO
    adapted_report: AdaptedReportDTO
    score_table: ScoreTable
    layout_recipe: LayoutRecipe
    vpm_artifact: VPMArtifact

load_compiled_report_aggregate(*, ref, resolver) resolves compiled_report via load_compiled_report_artifact, then each of the other four via their own already-established loader (load_adapted_report, load_score_table, load_layout_recipe, load_vpm_artifact) — each independently digest-verified and self-validated — then calls validate_compiled_report_aggregate before returning. It never returns a partially-valid aggregate: any check failure raises before the function returns, and no failed reference is silently skipped or rebuilt from another object.

9. Aggregate-validation algorithm

validate_compiled_report_aggregate runs five checks, each a separate function so no single function mixes more than one concern:

  1. _check_adapted_report_matches_compiled — compiled.adapted_report_ref.artifact_id == adapted.adapted_report_id; compiled.adapter_contract_id, compiled.compatibility_id, compiled.report_kind all equal the adapted report’s own; compiled.subjects == adapted.subjects and compiled.dimensions == adapted.dimensions (full tuple equality — identity, order, and every field, since these are literal embedded copies on both sides).
  2. _check_score_table_matches_adapted_report — first, and critically, score_table.metadata["adapted_report_id"] == adapted.adapted_report_id (see §10 below — this is the check that actually catches identical-matrix substitution). Then score_table.row_ids/metric_ids equal the adapted report’s declared subject/dimension order exactly, and every score_table.values[i, j] equals the corresponding AdaptedValueDTO.raw_value with exact float64 equality (no tolerance — the brief is explicit that tolerance-based equality is wrong here, since both were compiled from identical canonical numeric input).
  3. _check_layout_recipe_matches_vpm — the independently loaded LayoutRecipe’s digest and canonical payload (to_dict()) both equal vpm_artifact.recipe’s.
  4. _check_score_table_matches_vpm_source — the independently loaded ScoreTable’s digest, row/metric ids, and full array values (np.array_equal) all equal vpm_artifact.source’s.
  5. _check_cell_bindings_match_vpm_and_values — for every CellBindingDTO, resolves vpm_artifact.cell(view_row, view_column) (Core’s sole coordinate-resolution authority) and requires source_row_index/source_metric_index/subject_id(row_id)/dimension_id(metric_id) to agree; then resolves adapted.values[cell.value_index] and requires its subject_id/dimension_id/source_binding to match the cell’s, and its raw_value to equal the VPM cell’s raw_value — giving value_index a real, checked referent instead of an unchecked integer (the brief’s §9.7 requirement).

VPMArtifact’s own internal identity (source/recipe/permutation/normalized-value bounds/artifact-id/provenance) is not re-validated here — it is already proven by VPMArtifact.__post_init__ when load_vpm_artifact reconstructs it. This module only adds the cross-object checks Core’s own authority cannot know about (per the brief’s §9.5/§11 instruction not to duplicate Core validation).

Core-identity layering is preserved per §11 of the brief: ScoreTable.digest/VPMArtifact.artifact_id (Core’s own identity scheme) and ArtifactRef.artifact_id (the store’s content identity) are never required to be numerically equal anywhere in this module — only that an ArtifactRef resolves to canonical storage bytes that reconstruct a Core object that independently validates its own Core identity.

10. The identical-numeric-matrix gap, found and closed during this stage

Working through adversarial case 14.4 (two adapted reports sharing identical subjects/dimensions/raw values but differing confidence, importance, source finding, and parent lineage) surfaced a real design gap in the first draft of validate_compiled_report_aggregate: neither _check_adapted_report_matches_compiled (which only proves the compiled report’s own ref resolves to whichever adapted report was loaded, not that it’s the right one relative to a substituted ScoreTable) nor a naive raw-value comparison (identical by construction in this adversarial case) would have caught the substitution. The fix: compile_report’s _build_score_table already stamps metadata={"adapted_report_id": adapted.adapted_report_id} onto the ScoreTable it builds; _check_score_table_matches_adapted_report now requires that stamped id to equal the resolved adapted report’s own id — directly satisfying the brief’s §9.1/§9.2 requirement (“Require ScoreTable metadata to bind the adapted report identity”) and turning an adversarial test that would otherwise have silently passed into a real regression guard (test_adapted_report_substitution_with_identical_matrix_is_rejected).

11. Compatibility identity design

Design B (separate identities) was chosen per the brief’s own two offered options, over Design A (extend the existing compatibility_schema_id). compute_compatibility_schema_id (dimension ids/order, score semantics, value/target ranges, missing-value semantics) is unchanged — its v1 semantics are not reinterpreted, so no version bump is needed there. A new, additive compute_report_semantics_id(*, report_kind, subject_kind, dimension_namespace, duplicate_value_semantics) closes the layer the dimension-only digest cannot: two reports over sentences and claims with an identical dimension schema now produce different report_semantics_ids even though compatibility_id and compatibility_schema_id could coincide. Full compatibility between two compiled reports now requires all three to agree: the human label (compatibility_id), the dimension schema digest (compatibility_schema_id), and the report/subject schema digest (report_semantics_id). report_kind/subject_kind/dimension_namespace/duplicate_value_semantics are stored as literal fields on CompiledReportArtifactDTO (not merely referenced) so report_semantics_id can be self-validated in __post_init__ the same way compatibility_schema_id already is — recomputed from the record’s own fields, not trusted from an external contract that isn’t part of the aggregate.

12. Signature-envelope identity design

“Compatible design” was chosen (the brief’s own second offered option) over adding a self-validating signature_envelope_id field directly to SignatureEnvelopeDTO: the preferred design would require every existing construction site (test fixtures across two packages, the example scripts, verify.py’s callers) to be touched to supply a correctly-computed id, for a DTO that is never itself persisted through an ArtifactStore — it’s constructed dynamically at verification time from a raw sign_digest() output. Instead, compute_signature_envelope_id(*, authorization_id, signer_id, signature_hex, key_algorithm, spec_version) computes the identity centrally; verify.py’s _resolve_signature_envelope_id calls it once and the result feeds both TrustDecisionDTO.signature_envelope_id (the audit-receipt field) and _check_revocations’s "signature_envelope" target — the same function, so the two can never silently disagree on what “this envelope’s identity” means. TrustDecisionDTO.signature_envelope_id’s validation was tightened from “non-empty string” to _require_sha256 (matching every other evidence field on that DTO) when present.

13. Revocation migration behavior

Before this stage, RevocationRecordDTO(target_kind="signature_envelope", target_id=...) was checked against the raw signature_hex — an already-existing but completely untested code path (TrustFailureCode.REVOKED_ENVELOPE had zero test coverage before this stage; grep of the pre-stage test suite confirms it). After this stage, the same target_kind="signature_envelope" now targets compute_signature_envelope_id(...)’s output instead. This is a genuine behavior change for that one target kind, not a new target kind added alongside the old one (per the brief’s explicit instruction not to silently support two meanings for the same target kind) — a revocation record authored against the old raw-hex value will no longer match. test_revoking_raw_signature_hex_does_not_revoke_the_envelope proves this migration is real (not merely additive) by constructing exactly that stale record and asserting the artifact is still authorized.

14. Tests added

15. Validation run this session

python -m pytest -q packages/artifacts/tests                 # 88 passed
python -m pytest -q packages/trust/tests                     # 51 passed
python -m ruff format --check packages/artifacts packages/trust   # passed (after one auto-format pass)
python -m ruff check packages/artifacts packages/trust            # passed (after one auto-fix pass: unused/duplicate import)
python -m mypy packages/artifacts/src packages/trust/src           # Success: no issues found in 22 source files
python scripts/check_package_boundaries.py                         # passed: 141 production modules
python scripts/check_architecture.py                                # passed: 141 production modules inspected
python scripts/check_quality.py                                     # Quality checks passed (all governed packages)
python scripts/run_fast_tests.py                                    # 975 passed, 1 skipped, 0 failed, 69.58s / 120s budget
python -m pytest -q tests/test_public_api_manifest.py -k "not slow" # 15 passed

Two hard-limit violations were introduced along the way and both fixed within this session, per the repository’s stated policy of not silently exceeding quality gates:

16. Explicitly not run without further authorization

17. Known limitations (claims boundary)

Supported claim: ZeroModel can persist and reload a complete adapted report and its compiled ScoreTable, LayoutRecipe, and VPMArtifact as one content-addressed aggregate, and can verify that every subject, dimension, value, coordinate, source binding, and compatibility contract agrees across all representations. Trust decisions identify the exact signature envelope used through a content-derived envelope identity, and signature-envelope revocation targets that same identity.

Not supported / explicitly out of scope, per the brief’s own non-goals and this implementation’s actual reach:

18. Next safe architectural stage

Per the brief’s own instruction, this report does not generate the next implementation prompt. The brief itself and the ADR both note that Search is the likely later consumer, contingent on Stage 1A, Stage B, and this stage (Stage C) all being complete — which they now are.

This site is open source. Improve this page.