Baseline evaluations: the bar before any model

A baseline evaluation measures what the platform's internal quality safeguards achieve on your data before you train anything. Pick a dataset build and a pre-registered protocol, call POST /evaluations:baseline, and Tessera grades the protocol's internal reference checks and your own uploaded comparator scores on the build's test split. The result is the bar a trained model must later clear under the same protocol. The same safeguards are how the platform checks a sequence model before it can be promoted: gate G1 compares every gating label with an internal reference model (see Promotion gates).

A baseline evaluation never needs a training run, a model version or a bundle, and never creates one. Training against the bar is a separate, explicit POST /runs.

What runs

Everything is graded by the same code a full model evaluation uses, on the same masked test targets:

Source	What it is
Internal reference checks	the platform's internal quality safeguards over the dataset's own feature view, where the protocol declares them
cmp_…	your registered comparators (for example scores exported from a tool you use today)

The protocol decides which checks and comparators run and which metrics, margins and banding policy apply. A baseline request never restates them.

• Primary metrics (AUC for binary labels, macro-F1 for multiclass labels, R² for regression labels) cover every masked test target.
• Decision metrics (lift and capture at k%, precision and recall at the list size, alert precision at the alert budget) cover one row per entity under the protocol's banding policy.
• Every value carries a 95% entity-level bootstrap confidence interval.

Order matters: protocol, then dataset, then baseline

1. Define the label and register your comparators (below).
2. Declare the protocol (POST /protocols, see Protocols and backtests) with its labels, metrics, internal reference checks, comparators, margins and banding policy. Protocols are immutable and fingerprinted.
3. Build the dataset (POST /datasets). The protocol must be declared strictly before the build's manifest created_at. A protocol declared at or after the build is refused with protocol_after_data, so the bar can never be chosen after seeing the data.
4. Run the baseline evaluation (POST /evaluations:baseline) and read the result (GET /evaluations/{report_id}).

The request names only the dataset, the protocol and, optionally, a seed and a subset of the protocol's comparators:

{
  "dataset_id": "ds_01j9z3k2m4n5p6q7r8s9t0v1w2",
  "protocol_id": "prt_01j9z3k0a1b2c3d4e5f6g7h8j9",
  "seed": 0
}

The response is 202 with the evaluation's index row: basis: "baseline_eval", status: "running" and no model_id. Naming a comparator the protocol does not declare is invalid_argument, and so is restating any criterion or selecting an empty comparator subset on a protocol without internal reference checks (the bar would have no criteria).

Repeating the identical request returns the existing evaluation while it is running or once it has succeeded. A repeat of a failed or timed-out evaluation relaunches it under the same report_id. The report id is derived from the project, the dataset fingerprint, the protocol id and fingerprint, the comparator file fingerprints, the tokenizer version, the seed and the baseline configuration.

While the job runs, GET /evaluations/{report_id} reads running. Once it finishes it reads succeeded with the report attached, or failed with a reason. A row still running after 6 hours reads as failed / timeout.

Reading the report

The report follows https://spec.tessera.dev/schemas/baseline_eval_report.json. It has no model_id, no checkpoint, no promotion gates and no model rows. There is no trained model to gate.

• baselines: one entry per internal reference check. Each entry has, per label, baseline_metric ({name, value, ci95}) with n_train_targets and n_test_targets, or a typed skip reason such as insufficient_label_support.
• decision_metrics: each check's decision metrics under the banding policy.
• comparators: each comparator's coverage, point-in-time check and metrics. A comparator row that is newer than the target it would score fails the comparator closed, listing the violating rows. Each listed row names its entity by a pseudonym (hash: followed by the entity's keyed hash), never by the id in your file.
• evaluation_basis: basis: "baseline_eval", the protocol id and fingerprint, the dataset build, and the target_set_fingerprint of the exact test targets graded.
• verdict: overall: "bar_only". Each criterion (label × metric × check or comparator) records the reference value and confidence interval, the declared margin and the test a model must pass (paired_delta_ci_lower_gt_margin), with model: "absent". A criterion that produced no value carries a reason instead of a number.

The verdict states no win and no loss. Without a model there is nothing to pair. The bar is not a single number: when a model is evaluated under the same protocol, it passes a criterion only if the lower bound of the 95% CI of its paired difference over that reference, on the same targets, exceeds the margin.

Side by side with your trained model

After training, evaluate the model under the same protocol (POST /evaluations with the model_id and protocol_id). Both reports carry the same protocol_id and a byte-equal evaluation_basis.target_set_fingerprint. Both were graded on exactly the same test targets, and GET /evaluations?protocol_id=… lists the two side by side.

Worked example: an e-commerce churn bar

An online store wants to know whether a model can beat the churn scores its email tool already exports. The ids and numbers below are illustrative.

1. The label. Define churned_30d over the store's customers (Labels, POST /labels):

{
  "name": "churned_30d",
  "entity_type": "customer",
  "task": "binary",
  "source": {
    "kind": "label_table",
    "source_id": "src_01j9z2w1a2b3c4d5e6f7g8h9j0",
    "entity_id_column": "customer_id",
    "value_column": "churned",
    "observed_at_column": "observed_at"
  },
  "maturation_days": 30
}

2. The comparator. Create an upload session without a source_id (POST /uploads), PUT the email tool's export to it and do not complete it. The file needs a customer id, the instant each score was computed and the score:

customer_id,scored_at,churn_score
c-1001,2026-03-01T00:00:00Z,0.82
c-1002,2026-03-01T00:00:00Z,0.11

Then register it (POST /comparators):

{
  "name": "email-tool churn score 2026-03",
  "upload_id": "upl_01j9z3k4p5q6r7s8t9v0w1x2y3",
  "column_mapping": {
    "entity_id": "customer_id",
    "as_of": "scored_at",
    "score": "churn_score"
  }
}

Validation runs asynchronously: grade the comparator only once GET /comparators/{comparator_id} reads status: "ready". Scores must be point-in-time: for each test target at time t, grading uses the row with the greatest as_of <= t. A target that only a later row (as_of > t) could score fails the comparator closed, listing the violating rows by entity pseudonym; the report never repeats an id from your file.

3. The protocol (POST /protocols):

{
  "name": "churn bar before any model",
  "provenance": "production",
  "labels": ["lbl_01j9z2x4c5v6b7n8m9q0w1e2r3"],
  "metrics": ["auc", "lift@10pct"],
  "comparator_ids": ["cmp_01j9z3k8w2x3y4z5a6b7c8d9e0"],
  "margins": { "auc": 0.01 },
  "banding_policy": {
    "bands": [
      { "band": "high", "min_score": 0.7 },
      { "band": "medium", "min_score": 0.4 },
      { "band": "low", "min_score": 0 }
    ],
    "list_size": 2000,
    "alert_budget": 500
  }
}

4. The dataset build, after the protocol (Datasets, POST /datasets):

{
  "entity_type": "customer",
  "labels": ["lbl_01j9z2x4c5v6b7n8m9q0w1e2r3"],
  "window": {
    "max_events_per_entity": 1024,
    "time_range": { "start": "2025-09-01T00:00:00Z", "end": "2026-03-01T00:00:00Z" }
  },
  "split": { "method": "entity_hash", "train": 0.8, "val": 0.1, "test": 0.1, "seed": 7 }
}

5. The baseline evaluation. Post the request shown above with the build's dataset_id and the protocol's protocol_id. The 202 body is the index row (ProofEvaluation):

{
  "report_id": "rpt_01j9z3m7c8d9e0f1g2h3j4k5m6",
  "project_id": "proj_01j9z1a2b3c4d5e6f7g8h9j0k1",
  "environment": "live",
  "basis": "baseline_eval",
  "protocol_id": "prt_01j9z3k0a1b2c3d4e5f6g7h8j9",
  "dataset_id": "ds_01j9z3k2m4n5p6q7r8s9t0v1w2",
  "status": "running",
  "created_at": "2026-03-02T09:00:00.000000Z",
  "updated_at": "2026-03-02T09:00:00.000000Z"
}

6. The result. GET /evaluations/rpt_01j9z3m7c8d9e0f1g2h3j4k5m6 returns the row and, once it has succeeded, the stored report (ProofEvaluationDetail). The row:

{
  "report_id": "rpt_01j9z3m7c8d9e0f1g2h3j4k5m6",
  "project_id": "proj_01j9z1a2b3c4d5e6f7g8h9j0k1",
  "environment": "live",
  "basis": "baseline_eval",
  "protocol_id": "prt_01j9z3k0a1b2c3d4e5f6g7h8j9",
  "dataset_id": "ds_01j9z3k2m4n5p6q7r8s9t0v1w2",
  "status": "succeeded",
  "fingerprint": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
  "created_at": "2026-03-02T09:00:00.000000Z",
  "updated_at": "2026-03-02T09:21:40.000000Z"
}

This protocol declares one comparator and no internal reference checks. Here is report.verdict from the attached baseline_eval_report.json, in its schema shape (illustrative values), cut to the criterion for auc (the full list is every label × metric × comparator):

{
  "protocol_id": "prt_01j9z3k0a1b2c3d4e5f6g7h8j9",
  "protocol_fingerprint": "eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee",
  "provenance": "production",
  "overall": "bar_only",
  "criteria": [
    {
      "label": "lbl_01j9z2x4c5v6b7n8m9q0w1e2r3",
      "metric": "auc",
      "arm": "cmp_01j9z3k8w2x3y4z5a6b7c8d9e0",
      "arm_value": 0.702,
      "arm_ci95": [0.681, 0.722],
      "margin": 0.01,
      "rule": "paired_delta_ci_lower_gt_margin",
      "model": "absent"
    }
  ],
  "recompute": {
    "fingerprints": {
      "banding_policy": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
      "cmp_01j9z3k8w2x3y4z5a6b7c8d9e0": "cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc",
      "dataset": "dddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddddd",
      "protocol": "eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee",
      "target_set": "ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff"
    },
    "seeds": { "report": 0 },
    "api_calls": [
      {
        "method": "POST",
        "path": "/v1/evaluations:baseline",
        "body": {
          "dataset_id": "ds_01j9z3k2m4n5p6q7r8s9t0v1w2",
          "protocol_id": "prt_01j9z3k0a1b2c3d4e5f6g7h8j9",
          "comparator_ids": ["cmp_01j9z3k8w2x3y4z5a6b7c8d9e0"],
          "seed": 0
        }
      },
      { "method": "GET", "path": "/v1/evaluations/rpt_01j9z3m7c8d9e0f1g2h3j4k5m6" }
    ]
  }
}

Those are the numbers your trained model must beat, by more than the margin, on the same customers.

Cost and metering

A baseline evaluation runs as one job on the CPU batch pool (the cpu-standard class, 8 reserved vCPUs). It never uses a GPU. A succeeded run is metered once as build_cpu_seconds: wall-clock seconds × reserved vCPUs, with dims.basis = "baseline_eval". It is billed under the existing compute line. Failed or retried attempts are not metered.

Runtime estimate

The declared target is 45 minutes of wall time for the full set of internal reference checks plus one comparator on 100,000 entities, with up to 1,024 events per entity and up to 5 labels, on the 8-vCPU node. Each run gets an estimate from that target. Work inside that shape is estimated at 45 minutes. Larger work is scaled up by its largest ratio of evaluated rows (entities, plus the augmented rows of an observation-time build) or events, its labels beyond 5, and its checks plus comparators beyond the full set of internal reference checks plus one comparator. So 200,000 entities are estimated at 90 minutes.

The estimate is information, not a limit on your run:

• Before launch. A run whose estimate exceeds 6 hours, the deadline of the shared evaluation pool, is refused with 400 failed_precondition. This is the only refusal. details[0] has reason runtime_budget_exceeded and shows the estimate (estimated_seconds), the ceiling (ceiling_seconds), the shape it was computed from (entity_count, event_count, label_count, arm_count, comparator_count) and the reference shape. Evaluate fewer labels, internal checks or comparators, or a smaller build, to fit. Repeating a request for an evaluation that already exists still returns it.
• While it runs. The run is never stopped by its estimate. It runs under the pool's 6-hour deadline like every evaluation, records its report and is metered for the wall time it used. If the attempt that records it took longer than its estimate, it is flagged for the operators in the service log; its result is unaffected. A row whose job died without recording anything reads failed / timeout within about one 6-hour deadline window of its last update.