Scope note: this audit predates the nine-package split and covers only the legacy zeromodel root package’s claims (VPM artifacts, policy lookup, visual addressing, learning/training traces, and related consumers). The newer zeromodel-artifacts/zeromodel-trust/zeromodel-navigation distributions carry their own claims-boundary sections in their package READMEs and stage completion reports (see package-system-next.md and, for the current compiled-report aggregate guarantee, post-c203e7a7-aggregate-closure.md); they are not yet folded into this document’s claim matrix below.
This document compares the public ZeroModel claims against the current clean zeromodel package.
Sources reviewed:
zeromodel/tests/docs/research/ and docs/results/The audit uses a five-state evidence ladder. Qualifiers such as “for the committed fixture” or “at the tested operating point” are part of the status.
| Status | Meaning |
|---|---|
| Not implemented | The repository does not contain the claimed mechanism. |
| Implemented / unmeasured | Code exists, but no committed benchmark or evidence package measures the claimed capability. |
| Measured / unsupported | A measurement exists, but it does not support the stronger claim or does not yet identify a useful operating region. |
| Measured / refuted within stated conditions | A declared experiment produced evidence against the claim under the recorded fixture, representation, calibration, or operating point. |
| Validated within bounded conditions | Implemented and covered by explicit tests or reproducible evidence for a precisely bounded claim. |
Historical wording such as “thin evidence,” “not validated,” and “reframe” should be translated into this ladder when the affected row is next edited.
| Public claim | Current repo evidence | Status | Notes / required next proof |
|---|---|---|---|
| A VPM is a deterministic spatial view over a table of scored items. | ScoreTable, LayoutRecipe, VPMArtifact, deterministic artifact_id, binary-canonical identity bytes, and golden artifact-id test. |
Validated within bounded conditions | This is the strongest centre of the project. Keep this as the primary claim. |
| VPM cells map back to source evidence. | VPMArtifact.cell() returns source row, metric, raw value, and normalized value. Covered by test_cell_maps_view_coordinates_to_source_coordinates. |
Validated within bounded conditions | This supports inspectability. It does not by itself prove causal explanation. |
| Layout recipes reorganize the matrix task-by-task. | LayoutRecipe supports source, lexicographic, weighted, and complete explicit row ordering plus source/explicit column ordering. Tests cover lexicographic ordering, tie-breaking, and rejection of incomplete explicit permutations. |
Validated within bounded conditions | Explicit orders allow the outcome of an external or learned ordering process to become a normal deterministic VPM view. |
| The same dense table can support multiple policy views. | ViewProfile, ViewSet, build_view(), and build_views() build multiple deterministic VPMs from the same ScoreTable. Tests verify people/tree/car/risk views move different rows top-left while preserving the same source digest and cell/source mapping. |
Validated within bounded conditions | Valid wording: “ZeroModel can build multiple deterministic policy views from the same dense source table while preserving source mapping.” |
| A bounded policy can be compiled into an addressable artifact and read without a model call. | VPMPolicyLookup treats rows as discretized states and metrics as actions. Tests cover action argmax, cell/source proof, action-column limiting, alias SignReader, deterministic replay, and a tiny arcade-shooter example that clears a wave and beats a random baseline. |
Validated within bounded conditions | Valid wording: “ZeroModel can compile a bounded policy into a deterministic VPM artifact and use runtime state to read the action sign without invoking a model.” Do not claim open-world generalization. |
| A bounded visual observation can address a compiled policy without a symbolic runtime state ID. | VisualFeatureSpec, build_visual_index(), VisualIndexCalibration, and VisualSignReader compile a separate visual index linked to the policy by addresses. The corrected v2 calibration enforces a mathematically effective absolute-gap ambiguity rule, and tests cover caller-memory ownership, a reachable ambiguity rejection, all 112 canonical frames, and all 2,401 four-target waves / 31,213 visual decisions. |
Validated within bounded conditions | Valid wording: “ZeroModel can recover the committed arcade policy rows and actions from canonical observations with exhaustive equivalence to the state-addressed path.” This is exact feature-codeword addressing, not accepted non-exact tolerance, open-world perception, or general image understanding. |
| Different visual-address engines can share one governed seam while the policy remains independently identified. | ImageObservation, VisualAddressContract, VisualAddressDecision, VisualAddressProvider, DeterministicVisualAddressProvider, and VisualPolicyReader expose score polarity, replay semantics, provider identity, rejection evidence, and a separate policy lookup. Tests use the current deterministic reader as the conformance proof. |
Validated within bounded conditions | Valid wording: “ZeroModel defines a provider-neutral contract through which a visual address can be verified and then delegated to an exact VPM policy.” This does not validate learned visual generalization. |
| External provider evaluations can be preserved as immutable evidence while separating exact observation correctness from policy-level behavioural correctness. | ProviderConfigurationDTO, ProviderEvaluationCaseDTO, ProviderEvaluationSummaryDTO, ProviderEvaluationRunDTO, MaterializedProviderEvaluationRunDTO, the DTO-only VideoActionSetStore surface, and matching in-memory/SQLAlchemy implementations persist provider configuration, observation references, expected and predicted states, policy decisions, response evidence, and deterministic summaries. Aggregate validation distinguishes exact, action_equivalent, action_changing, and rejected outcomes and rejects unrelated cases or summaries. Tests cover derived-field fabrication, aggregate substitution, SQL tampering, rollback, memory/SQL parity, and fresh-session reload. |
Validated within bounded conditions | Valid wording: “ZeroModel can persist and reload bounded provider-evaluation runs that distinguish exact-state correctness from policy-action correctness while binding the evidence to an identified provider configuration, observation, and compiled policy.” This validates evidence accounting and aggregate closure, not provider accuracy, visual understanding, confidence calibration, or safe deployment. |
| Controlled visual-representation changes can be recorded and compared while holding the provider and policy identities fixed. | docs/results/controlled-png-representation-v1/ records six real-provider runs sharing one provider configuration, prompt digest, protocol, compiled policy, fixture identity and case mode while varying only the identified representation recipe. The package preserves content-addressed PNGs, operation provenance, complete case outcomes, summaries and compiled-report identities. |
Validated within bounded conditions | Valid wording: “ZeroModel can execute, persist, reload and compare a bounded real-provider PNG-representation experiment while holding the provider configuration, prompt, parser, compiled policy, fixture and inference settings fixed.” This validates evidence accounting and controlled comparison, not that a representation improves perception or that the result generalizes. |
| The tested implicit PNG interventions improve the fixed local provider’s reliability over the unlabelled representation. | docs/results/controlled-png-representation-v1/ records an eight-state real-Ollama smoke experiment. cooldown-shape-v1 and cooldown-dual-v1 produced no targeted cooldown improvement; cooldown-redundant-v1 increased action-changing errors from 1 to 2; lane-enhanced-v1 increased action-changing errors from 1 to 2 and produced one rejected TANK_COLUMN: 7 response. The labelled control remained 8/8 exact. |
Measured / unsupported | Unsupported for the recorded model blob, Ollama version, fixed prompt, parser, policy and smoke fixture. No independent repetition or 112-state canonical run was performed because no implicit intervention passed the predefined smoke promotion gate. Do not generalize this result to other models, recipes or visual domains. |
| Dense visual representations can be identity-bearing without being treated as VPM policy metrics. | MatrixBlob provides canonical dtype/shape/bytes, quantization metadata, deterministic identity, tamper detection, and round-trip reconstruction. VisualAddressManifest binds each vector row to a prototype ID and policy row without creating fake policy rows. Phase 1 retrieval builders use these contracts for medoid and all-example vectors. |
Validated within bounded conditions | MatrixBlob is a tensor contract, not a VPM and not evidence that a representation is useful. Exact float-vector digests identify bytes under the recorded encoder/runtime scope; they are not a cross-device reproducibility claim. |
| A deployed visual address can be pinned to one policy, calibration, source scope, and optional encoder identity. | DeploymentBinding has deterministic identity, verifies an exact VisualAddressContract, and blocks research bindings by default. |
Validated within bounded conditions | The binding is not a signature, authorization record, trust attestation, or persistence service. Signed deployment lineage remains future work. |
| Visual-address benchmark inputs can enforce family-held-out prototype, calibration, and test splits. | CorruptionFamilySpec, VisualExampleRecord, VisualDatasetManifest, VisualBenchmarkMetrics, VisualBenchmarkReport, deterministic corruption operators, and evaluate_visual_provider() validate and execute split isolation, identical row coverage, OOD semantics, critical-intervention semantics, explicit FAR/FRR denominators, per-family counts, top-1 ranking counts, and research status. |
Validated within bounded conditions | This validates benchmark machinery, not a successful learned visual reader. Observation-level Wilson intervals are descriptive; paired and state/family-clustered analyses remain required for system comparisons. |
| A pinned frozen image encoder can feed prototype, k-NN, and linear-probe address baselines. | EncoderManifest, HuggingFaceDinoV2Encoder, VectorAddressIndex, FrozenVectorAddressProvider, LinearProbeIndex, and examples/arcade_visual_address_benchmark.py implement an optional DINOv2-small path plus normalized-pixel, medoid, all-example k-NN, and ridge linear-probe systems. The committed evidence shows the tested global DINOv2 CLS systems do not materially improve governed policy addressing on the bounded arcade fixture. |
Measured / refuted within stated conditions | Refuted for the recorded fixture, global CLS representation, preprocessing, and measured operating conditions. This does not prove every learned or local representation will fail. |
| Normalized pixels provide a safe approximate visual-address reader. | System B v2 preserves strong pre-rejection ranking signal, but the repaired calibration protocol selects only quantile 1.0, which yields calibration acceptance 1 / 1344, final benign coverage 0 / 1344, FAR 0 / 248, FRR 1344 / 1344, raw top-1 exact-row accuracy 1008 / 1344, and raw top-1 action accuracy 1302 / 1344. |
Measured / unsupported | System B has useful pre-rejection ranking signal but no useful zero-distinguishable-FAR operating region on the corrected bounded arcade fixture. Accepted precision is undefined on final evaluation because no benign observations were accepted. |
| Frozen DINOv2 global retrieval materially improves governed policy addressing over normalized pixels. | C and D reuse the same pinned DINOv2-small CLS extraction. D is 1.34 action points above B but about 32 exact-row points below B, with 73.79% FAR and 292 conflicting-action errors. C is below B on benign action, exact row, and FAR. | Measured / refuted within stated conditions | Refuted for this fixture, representation, preprocessing, calibration method, and measured operating point. This does not prove every learned or local representation will fail. |
| A rejection-equipped ridge linear probe matches retrieval at lower complexity. | G records 59.45% benign action accuracy, 28.79% exact benign row accuracy, 100% FAR, and 521 conflicting-action errors. | Measured / refuted within stated conditions | G is eliminated for this fixture and operating point. Curves may characterize the failure but do not currently justify further development. |
| Registration/locality already provides a validated next-step visual reader. | zeromodel.visual_registration, zeromodel.visual_local_baselines, and docs/results/visual-local-baseline-showdown-v1/ implement and measure bounded integer registration plus local normalized-pixel comparison against the frozen System B v2 comparator. The selected q=0.0 operating point preserves zero observed false accepts and zero accepted conflicting-action errors, but final benign coverage remains 0 / 1344 with FRR 1344 / 1344, while raw top-1 remains materially informative at 1176 / 1344 exact-row and 1323 / 1344 action accuracy. |
Measured / unsupported | Registration repairs part of the translation-locality ranking failure, but it still does not yield a useful governed accepted operating point on the bounded arcade fixture. Keep the next step on translation-equivariant local methods rather than promoting this reader. |
| Frame-local discriminative evidence yields a safe governed visual reader. | zeromodel.video_discriminative_evidence, the Stage 3 benchmark harness, and docs/results/video-discriminative-local-evidence-v1/ implement bounded region extraction, discriminative mask payloads, architecture A/B/C/D evaluation, frozen split/region/mask/grid manifests, and pre-final architecture/operating-point selection artifacts. The committed pre-final selection records no_safe_architecture and no_feasible_operating_point for the tested fixture and protocol. |
Measured / refuted within stated conditions | The machinery and evidence package exist, but the current bounded experiment does not justify promoting a discriminative local-evidence reader. This is a negative result for the recorded fixture, regionization, architecture family, and selection protocol, not a proof that every local method will fail. |
| A fixed-camera runtime is implemented for the visual-address path. | docs/research/fixed-camera-status-panel-benchmark.md records only a protocol direction and explicitly says no empirical fixed-camera captures are committed as of July 17, 2026. |
Not implemented | Protocol documentation alone is not an implementation or measurement. |
| Governance-parity runtime comparisons have been implemented and measured. | docs/research/visual-governance-parity.md records the question and notes that stronger System B v2 artifacts are preparatory only. |
Not implemented | The repository does not yet contain a completed governance-parity runtime experiment. |
| A compiled VPM policy can be consumed outside Python. | VPMPolicyLookup.to_compiled_plan(), compiled_plan_id(), and write_lua_policy() export the bounded arcade policy as a dependency-free Lua module. CI installs Lua 5.4, exports the policy, executes the complete wave, and verifies the VPM artifact ID and separate consumer-plan ID. |
Validated within bounded conditions | Valid wording: “the same identified bounded policy can be consumed by Python and standard Lua runtimes.” This is not a microcontroller memory, latency, energy, or real-time benchmark. |
| A Q-bearing policy can preserve criticality and decision margin as non-action evidence. | with_q_diagnostics() adds best-minus-worst criticality and best-minus-second-best decision_margin; VPMPolicyLookup.evidence_metric_ids returns those values while excluding them from argmax. Unit tests verify exact calculations and unchanged action selection across the 112-state example. |
Validated within bounded conditions | Use “VIPER-style criticality” only when the action columns carry Q-values or an equivalent consequence-bearing teacher signal. |
| Named row-level properties can be checked exhaustively over a finite policy. | PolicyPropertyChecker, PolicyPropertySpec, and the key-value-row-id/v1 decoder evaluate versioned declarative expressions across every source row. Tests cover a passing policy, an exact seeded FIRE counterexample, a repaired policy, typed row-ID scalars, JSON-null semantics, and contextual comparison errors. |
Validated within bounded conditions | This is not general formal verification of temporal dynamics, continuous state spaces, liveness, or universal safety. |
| Verification results can become deterministic artifacts linked to the exact checked policy. | PolicyVerificationReport.to_vpm() stores checker version, property-spec digest, rows checked, violations, candidates, evidence, and coordinates. Provenance includes a verifies parent relation; tests cover deterministic identity and .vpm round-trip. |
Validated within bounded conditions | The artifact identifies which policy and properties were checked. It does not prove authorship, authorization, or that the property set is sufficient. |
| Counterexample, repair, and re-verification can be represented as artifact lineage. | examples/criticality_verification.py creates original, unsafe, and repaired policy identities plus failed and passing verification artifacts. The seeded violation localizes to one declared row and action cell. |
Validated within bounded conditions | Automatic repair is not implemented. The example demonstrates a recorded review loop, not autonomous policy correction. |
| A spatial optimizer can derive a view profile for a geometric objective. | SpatialOptimizer, SpatialOptimizationResult, optimize_view_profile(), and build_optimized_view() learn non-negative metric weights for an explicit top-left mass objective and emit a normal ViewProfile. Tests cover improved objective mass, source digest preservation, table-series input, and validation errors. |
Validated within bounded conditions | Do not claim it learns the semantically best view for every task. |
| ZeroModel can discover and freeze latent row structure in a scored matrix. | MatrixPatternDetector, PatternAnalysisSpec, and PatternReport perform deterministic spectral seriation, score adjacent coherence and anti-Robinson structure, rerun the complete pipeline on every permutation-null sample, and materialize linked report and discovered-view artifacts. |
Validated within bounded conditions | Do not claim general semantic regime discovery, anomaly detection, or real-world validity from the synthetic fixture. |
| A sequence of dense panels can become a temporal decision manifold. | DecisionManifold, ManifoldFrame, ManifoldTransition, ManifoldSummary, build_decision_manifold(), and find_inflection_points() build optimized VPM frames across a consistent panel sequence and surface large spatial-view changes. |
Validated within bounded conditions | Do not claim semantic cause or universal change-point discovery. |
| No model at decision time. | TopLeftGate, VPMPolicyLookup, and VisualSignReader consume prepared artifacts without invoking a learned model. |
Validated within bounded conditions | HuggingFaceDinoV2Encoder performs model inference before deterministic policy lookup. The learned path must never be described as “no model at decision time.” |
| Learning can be made visible. | LearningObservation and build_learning_vpm() require train, held-out, and regression observations before learned=True. |
Validated within bounded conditions | This is deterministic trace artifactization, not proof of internal model causality. |
| Training progress can be visualized. | TrainingCheckpoint and build_training_progress_vpm() convert checkpoint telemetry into progress VPMs. |
Validated within bounded conditions | This does not replace experiment trackers or prove internal model causality. |
| Tracker exports can feed training progress artifacts. | zeromodel.adapters parses generic JSON/JSONL/CSV, TensorBoard scalar CSV, W&B history JSONL/CSV/JSON, and Trackio JSON/JSONL/CSV exports. |
Validated within bounded conditions | This is exported-file ingestion, not live SDK/API integration. |
| End-to-end fixtures can reproduce progress artifacts. | tests/fixtures/training/ contains deterministic tracker-style fixtures and tests verify selection, warnings, mapping, round-trip, and rendering. |
Validated within bounded conditions | This is synthetic research-readiness evidence, not validation on real training runs. |
| Critic/evidence scores can become risk-first artifacts. | CriticObservation, build_critic_vpm(), and observations_from_critic_lines() convert scored critic outputs into inspectable VPMs. |
Validated within bounded conditions | ZeroModel does not detect hallucinations by itself. |
| Task-aware top-left concentration. | phos_sort_pack, pack_artifact, guarded_pack_artifact, top_left_concentration, ViewProfile, SpatialOptimizer, and DecisionManifold. |
Implemented / unmeasured | Need task benchmarks showing automated or human inspection improvement. |
| Compositional visual logic: AND/OR/NOT/XOR/add/subtract. | zeromodel.compose implements shape-checked fuzzy operators; tests cover AND/OR/XOR and comparison. |
Validated within bounded conditions | Reframe as explicit fuzzy field composition, not general symbolic reasoning. |
| Deterministic reproducible provenance. | Artifacts include source and recipe digests, parents, provenance payload, cached identity bytes, provider decisions, manifests, calibrations, benchmark reports, and deployment-pair identities. | Validated within bounded conditions | Identity and declared lineage are not signatures or external trust attestations. The original local full-run PowerShell script contained a disclosed command/device capture defect; benchmark provenance must be tested end to end. |
Lossless .vpm artifact bundles. |
to_bundle() writes a zip with manifest.json; from_bundle() validates bundle version and reconstructs VPMArtifact. |
Validated within bounded conditions | Visual-address manifests, matrices, reports, and bindings do not yet have one combined persisted bundle format. |
| PNG as universal self-describing artifact. | render.png_bytes() writes a grayscale PNG image from a field. |
Measured / unsupported | Current PNG is a rendering/transport image only. It is not self-describing. |
| PNG survives image pipelines. | No current tests for resize/crop/JPEG/social-media pipelines. | Implemented / unmeasured | Avoid this claim. Lossy transforms will likely destroy exact numeric semantics. |
| Hierarchical pyramids / zoomable navigation. | build_pyramid() reduces fields into levels with mean/max/sum. |
Implemented / unmeasured | Need a pointer model, child tile IDs, resolver, and traversal benchmark. |
| Planet-scale / infinite memory / 40-hop traversal. | No current benchmark or fixture proves the scale and latency claims. | Not implemented | Treat as research ambition. |
| Milliseconds on tiny hardware / 25KB RAM. | The optimized Python consumer and dependency-free Lua exporter exist, but there is no named constrained-device benchmark. | Implemented / unmeasured | Required: named target hardware, implementation, timing, and memory report. |
| Edge ↔ cloud symmetry. | The same bounded policy identity can be consumed by optimized Python and standard Lua plans. | Implemented / unmeasured | Language/runtime portability is validated. Edge/cloud symmetry still requires named edge hardware and a cloud consumer over the same artifact bytes. |
| Multi-metric, multi-view by design. | ScoreTable, LayoutRecipe, ViewProfile, SpatialOptimizer, MatrixPatternDetector, and DecisionManifold operate over the shared artifact model. |
Validated within bounded conditions | Next proof: task-level automated or human inspection benefit. |
| Storage-agnostic routing via pointers. | No current pointer/resolver abstraction. | Not implemented | Add resolver interfaces and tests before making the claim. |
| Traceable “thought” / 40+ levels. | The code has many artifact and trace types but no router frame chain or step graph implementation. | Measured / unsupported | Use the specific artifact name rather than “thought.” |
| Human-compatible explanations. | Cell/source mapping, decisions, counterexamples, reports, and renderings allow inspection. | Implemented / unmeasured | Reframe as inspectable evidence mapping. Dense vector dimensions are not human explanations. |
| Cheap to adopt. | The core package remains NumPy-only; learned vision is optional. | Implemented / unmeasured | Measure install, model distribution, runtime, hardware, licence, and calibration costs. |
| Works with your stack. | Adapters and artifact builders cover exported telemetry, critic traces, finite policies, the arcade fixture, synthetic patterns, views, spatial objectives, and temporal panels. | Implemented / unmeasured | Broader wording requires real fixtures and optional live integrations. |
| Great fits: anomaly detection, safety gates, code review traces, search triage, closed-world policy lookup. | The package has primitives and early examples for these domains. | Implemented / unmeasured | Create domain-specific fixtures before claiming domain success. |
| Viewer you’ll actually use. | A static site exists, but the package has no bundled viewer or click-through pointer graph. | Implemented / unmeasured | Add a tested demo fixture and pointer graph. |
The repo can honestly claim:
ZeroModel turns scored data into deterministic, inspectable Visual Policy Map artifacts. For closed enumerable policies, those artifacts support exact state-addressed lookup, finite property checks, linked verification lineage, and portable consumers. In the committed arcade fixture, a separately identified deterministic visual index recovers all canonical feature codewords, policy rows, and actions; this is exact closed-world addressing, not general visual AI. The package also implements a provider-neutral visual-address seam and conventional pixel, frozen-embedding, k-NN, linear-probe, and frame-local discriminative research baselines. The corrected System B v2 adjudication shows that normalized pixels preserve useful pre-rejection ranking signal but do not provide a useful zero-distinguishable-FAR operating region on the corrected bounded arcade fixture. The current Stage 3 discriminative local-evidence selection artifacts record no safe architecture and no feasible operating point under the committed protocol. The tested global DINOv2 CLS, linear-probe, and current discriminative-local paths are not promoted. The video action-set research path also implements a closure-validated provider-evaluation aggregate that separates exact-state correctness from policy-action correctness. It can preserve provider configuration, PNG observation identity, transformation provenance, expected and predicted states, compiled-policy decisions, and deterministic run summaries. This is evidence infrastructure, not evidence that a provider is accurate or that changing a PNG improves model performance; those claims require a committed controlled representation-intervention benchmark.
The repository validates the core deterministic artifact and bounded-policy mechanisms, exact canonical visual codeword addressing, provider-neutral visual contracts, and executable research machinery. It has now also produced a meaningful negative visual result.
It does not yet have a validated visual AI.
The open problem is trustworthy observation-to-symbol compilation. The next work should determine whether further bounded local methods are warranted after the current negative discriminative result, whether a factorized state compiler is preferable, or whether no visual path at all is the correct architecture.
The repository now also validates the integrity of bounded provider-evaluation evidence: it can identify and persist which provider configuration observed which PNG-derived observation, which compiled policy interpreted the predicted state, and whether an error was exact-state, action-equivalent, action-changing, or rejected. This does not validate the provider itself, and it does not yet show that visual-representation changes improve performance.