This is a functioning prototype of the evaluation layer behind the TargetSpace benchmark: passive longitudinal evidence becomes sealed prediction instances; model predictions are hashed and sealed before outcomes exist; deterministic rules resolve outcomes; and a scoring pipeline reports calibrated, target-specific skill against R1/R2 baselines, controls, and evidence costs.
A system earns target-specific credit only when its sealed predictions beat the population prior (R1) and the target's own routine (R2), stay calibrated, collapse under wrong-target permutation, and disclose what the evidence cost.
Define target type, evidence streams, horizon, answer spaces, resolution rules, baselines, and cost axes.
Watch passive timestamped evidence become sealed prediction instances with cutoffs and deterministic rules.
Submit a probability distribution over the operational possibility space — the answer states still open; it is hashed and sealed before resolution.
Tamper-evident registry: hashes, timestamps, statuses, audit trail, and chain verification.
Deterministic rules resolve future evidence into outcomes — never self-report, never post-hoc judgment.
Log skill in bits vs R1/R2, Brier, calibration, wrong-target control, ablation, and validity-stack gates.
Incremental lift per evidence-cost axis: which observation earns its cost, and which does not — toward the minimum sufficient observation.
Architecture for keeping raw personal data local: only sealed predictions, outcomes, and aggregates leave.
Twelve JSON schemas define every record: runs, instances, sealed predictions, outcomes, reports.
| Component | Status | Notes |
|---|---|---|
| Sealed prediction registry — tamper-evident ledger (hash-chained) | implemented | Local (browser storage); production would add an external timestamp witness and database |
| Instance constructor from synthetic evidence | implemented | Deterministic rules pre-registered per instance |
| Scoring pipeline (log skill, Brier, R1/R2, controls) | implemented | Pure functions; runs end-to-end in the browser and in CI |
| Evidence-efficiency reporting (per-axis) | implemented | Ordinal 0–3 cost axes in the prototype; measured units in production |
| Validity-stack gates & certification-style summary | implemented | Summary of gate outcomes only; no certification marks exist |
| Live model inference | conceptual | Demo distributions are hand-authored synthetic submissions |
| Federated local runner & encrypted audit package | conceptual | Architecture specified on the federated page |
| External timestamp witness | conceptual | Prototype witness is the local ledger chain itself |