Proof center

The proof center is the console's view of the proof engine. A protocol states, before the data exists, what a model must beat. An evaluation judges a dataset build against a protocol: the protocol's internal quality safeguards alone, before any model exists, or a model against them. A backtest replays a model over cutoffs in time and records how far ahead of a positive outcome it first alerted. A comparator is a file of your own scores that a protocol can name as a bar. Every screen shows what the API recorded. Nothing is counted, rounded, re-ordered or combined in the browser, and a loss is shown as a loss. The API behind the screens is described in Protocols & verdicts and Baseline evaluations.

The proof center is in the console's Modeling group. It appears only in deployments that serve the protocol, evaluation and backtest operations and have the proof center switched on; with it off, its routes do not exist.

Protocols

A protocol is declared once and cannot be edited or deleted. To change the bar, declare a new protocol. It records:

• the labels it judges, the metrics it judges them on and a margin for each metric;
• the internal quality safeguards the model must beat (see Decision metrics & comparators) and the comparators (scores you uploaded, below);
• optionally, lead-time metrics with margins for backtests, detection-quality thresholds, and a holdout salt;
• the banding policy: bands, list size and alert budget, under which the decision metrics are computed;
• your claim about the data (fixture, design-partner or production), copied into every verdict and backtest the protocol produces.

The server fingerprints the declared content and records when it was declared (declared_at) with an audit event; the protocol page shows both.

Margins. A success margin is declared per metric. When none is declared for a metric the server applies 0.0, and the protocol page says "Not declared (0.0 applies)" instead of showing a zero nobody typed. In the declaration form a blank margin is not sent, so that default applies. Margins declared for names the protocol does not list are shown beside the listed ones.

Declared before the data. A protocol can judge only a dataset build that was created strictly after the protocol was declared. Judging an earlier build is refused with the reason protocol_after_data; the console names both instants and says what to do next for what was being judged: for a dataset build, build one after the protocol and judge that; for a model or a backtest, the build that matters is the one the model was trained on, so declare a new protocol, build a dataset after it and train a model on that build.

Declaring one. Members and above can declare a protocol in the console (POST /protocols through the API). The form reads the declaration back, field by field, before anything is sent, because it cannot be undone. A comparator that is still validating can be declared; it must be ready when an evaluation runs.

Comparators

A comparator is a file of your own scores, such as a predicted-value export from a marketing tool, a risk score from a security product or an in-house model, that a protocol can name as a bar the model has to beat. It is an evaluation input only: it is never ingested, bundled or served. Decision metrics & comparators says how a comparator is graded, including the point-in-time rules.

Registering one. The Comparators screen (/proof/comparators) lists the comparators and, for members and above, registers a new one. The form takes a name, a file (.csv or .parquet, up to 64 MiB) and the columns that hold the entity id, the time each score was computed (as_of) and the score, plus an optional class column. The console sends the file straight to storage, and the upload can be paused and resumed; the upload is never completed into a source. Whether the named columns exist and the values parse is checked by the server, not by the browser.

Pending, then ready or failed. A new comparator is Validating (pending) while a validation job reads the file. It settles to Ready, with its row and entity counts, the range of times it was scored between and the file's fingerprint, or to Failed, with the reason in words, the next step and the per-row violations the job found (row, column and problem, under "Show violations"). The list re-reads a validating comparator until it settles; if that takes long it says "Still validating" and offers Check again.

Must be ready when an evaluation runs. A protocol names up to four comparators. The declaration form lets you name one that is still validating, but each one must be Ready when an evaluation runs, and a failed comparator cannot be named.

Evaluations

An evaluation records its verdict as it comes out.

Overall verdict	Meaning
Pass	Every declared criterion passed.
Fail	At least one declared criterion failed.
Incomplete	No criterion failed, but not every criterion passed. Missing evidence is shown as neither a win nor a loss.
Bar only	The bar was measured and no model exists yet, so there is no win or loss to report.

Each declared criterion is one label, one metric and one bar (an internal quality safeguard or a comparator). The table shows the result, the declared margin, the difference between the model and that bar and the 95% interval of the difference. A criterion passes only when the lower bound of that interval is strictly above the declared margin. A criterion with no evidence is not evaluated, with its reason. A failure carries an icon and a word, never colour alone, and the rows are in the order the report states them. A value the console does not recognize is shown exactly as served and is never read as a pass.

The console lists the criteria against a comparator. A criterion against one of the internal quality safeguards is not listed, in evaluations and in backtests alike, and neither is a backtest's curve or paired comparison for one. The "Criteria met" count covers the comparisons shown, and a line under it says so whenever criteria were left out. The API, the stored report and its json download keep every criterion; dossier renders follow the same rule (governance evidence).

A comparator reads by one name on every page: the name it was uploaded under, or "Comparator 1", "Comparator 2" and so on, numbered among the comparators the page shows. Its id is under About beside the name.

A protocol page lists the labels, metrics and comparators a protocol declares. It never lists the internal quality safeguards the protocol also measures against; the API and the protocol document keep them.

A baseline evaluation has no model: each row is the bar's own value and interval, which is the bar a model will have to clear. The page links to the run launcher with the dataset selected; nothing is trained from the proof center.

When a baseline evaluation and a model evaluation of the same protocol exist, the console lines them up side by side, but only when the reports state the same target-set fingerprint and the same protocol fingerprint, because only then were both judged on the same masked test targets. Otherwise it says they were not judged on the same targets and does not align them. In the side by side, a bar that has no value shows the reason its report gave, the declared margin beside the model's result is the margin the model's own row states (it says so when that row states none, and never shows the bar's instead), and a criterion that either report states more than once is shown once per statement and not lined up, so that a repeat cannot hide a failure.

Every report ends with a recompute appendix: the input fingerprints, the seeds and the exact API calls that reproduce it. Verdicts are advisory evidence: they are never a promotion, serving or delivery gate.

Running one. Members and above can run an evaluation (POST /evaluations:baseline for the bar alone, POST /evaluations for a model). You choose the protocol; for the bar alone, the dataset build to measure (finished builds only) and, optionally, a subset of the protocol's comparators (none chosen means all of them); for a model, the model to judge; and, optionally, a seed (the server's default is 0). The form sends only what was chosen, because the criteria come from the protocol.

A run that is still going is re-read for a while, counting only the time the page is visible, and then the page says it stopped updating and offers to check again. Asking for an identical evaluation again returns the one that already exists, and the console says what happened: "Started" only for an evaluation that is new, "Already running" or "Already recorded" for one that existed, "Relaunched" when the earlier run had failed and the server started it again, and "Accepted" when the console could not tell. An evaluation the API records as succeeded without sending its report is shown as that, with no verdict inferred. The pickers on the form list up to 200 protocols, dataset builds or models and say so when more exist.

Backtests

A backtest asks how far ahead of a positive outcome each arm first alerted at the protocol's alert budget. It is replayed over cutoffs in time, either on the dataset's test split or on the full entity book after the training data. It refreshes no slot and writes no served state.

For each label the console draws, for every arm, the share of eligible positives alerted at least d days ahead, from the server's own points: there is no smoothing and no interpolation. A lead day at which nobody was eligible has no share, so it is left off the curve and the line breaks there; the exact numbers, including those, are in the data table below the chart.

Each point's 95% interval, exactly as the API served it, is drawn as a shaded band behind the line, straight between served points and never smoothed, widened or narrowed, and as a whisker where a point stands alone. A point the API served no interval for has no band, and the band breaks wherever the line does. Each arm has its own marker shape as well as its own colour, the legend shows both, and no two of the first 32 arms share both.

Arms are named for how their alerts are counted. For a silence-bounded (prefix only) arm, an alert counts at the first cutoff, and at a later cutoff only if the entity has at least one anchor since the previous cutoff; a first alert at a later cutoff with no anchor in that interval is set aside, and the page shows how many per arm (such a positive can still succeed through a later counted alert). An observation-time arm can alert during silence, so none of its alerts is set aside. Below the chart the page shows each arm's median lead time with its interval, the paired comparison of the model against each arm on each declared lead-time metric, and, when the protocol declares lead-time metrics, the verdict.

Running one. Members and above can run a backtest (POST /backtests). You choose:

• the protocol;
• what to replay: a model by id, a deployment slot (only slots with an active model are offered; the backtest replays the model that is active when you start, and the report records which bundle that was) or a promotion bundle;
• the engine: on the dataset's test split (the default) or on the full entity book out of time;
• optionally, the cutoffs, one RFC 3339 instant per line, two to twelve of them, strictly ascending and inside the engine's window (blank means six evenly spaced cutoffs chosen by the server and recorded in the report);
• optionally, a seed (the default is 0).

The server checks the order of the cutoffs, their window and, for the full-book engine, that outcomes have had time to mature. When it refuses the cutoffs it says why in its own words; the page shows that sentence with the request id and keeps your choices so you can correct them. As with evaluations, asking for an identical backtest again returns the one that already exists and the console says so (including "Relaunched" when the earlier run had failed and was started again), and a backtest the API records as succeeded without sending its report is shown as that.

Roles

Reading protocols, evaluations and backtests needs the viewer role. Declaring a protocol, running an evaluation or a backtest, and registering a comparator need the member role or above. A viewer sees the same lists and reports with a note in place of the form, which says who can start one.