Skip to main content

Build the dataset

A dataset build assembles one training example per entity — the user's events in order, plus supervision targets with masks — under everything you just approved: the contract scopes attributes, the leakage gates audit them, and the label's point-in-time rules decide which positions may be supervised.

05-build-dataset.sh
#!/usr/bin/env bash
# Quickstart step 5 — build the leakage-gated training dataset.
set -euo pipefail

: "${TESSERA_API_BASE:?e.g. https://api.staging.tessera.zone/v1}"
: "${TESSERA_API_KEY:?your tsk_test_… key}"
: "${TESSERA_STATE_DIR:=.}"
AUTH=(-H "Authorization: Bearer $TESSERA_API_KEY")
LABEL_ID="$(cat "$TESSERA_STATE_DIR/label_id")"

# Small windows keep the tutorial models CPU-friendly: each training example
# is a user's last 8 events inside the sample's fixed time range.
BUILD="$(curl -sS -X POST "$TESSERA_API_BASE/datasets" "${AUTH[@]}" \
-H "Content-Type: application/json" \
-H "Idempotency-Key: quickstart-dataset" \
-d '{
"entity_type": "user",
"labels": ["'"$LABEL_ID"'"],
"window": {
"max_events_per_entity": 8,
"time_range": {"start": "2025-01-01T00:00:00Z", "end": "2025-07-01T00:00:00Z"}
},
"split": {"method": "entity_hash", "train": 0.7, "val": 0.1, "test": 0.2, "seed": 7}
}')"
DATASET_ID="$(printf '%s' "$BUILD" | python3 -c 'import json,sys; print(json.load(sys.stdin)["dataset_id"])')"

for _ in $(seq 1 600); do
DATASET="$(curl -sS "$TESSERA_API_BASE/datasets/$DATASET_ID" "${AUTH[@]}")"
STATUS="$(printf '%s' "$DATASET" | python3 -c 'import json,sys; print(json.load(sys.stdin)["status"])')"
case "$STATUS" in succeeded|failed|aborted_leakage) break ;; esac
sleep 2
done

# The manifest is the build's receipt: row counts per split, per-label
# supervision accounting, the (empty) violation report, and every pinned
# fingerprint.
printf '%s' "$DATASET" | python3 -c '
import json, sys
dataset = json.load(sys.stdin)
manifest = dataset.get("manifest") or {}
print(json.dumps({
"dataset_id": dataset["dataset_id"],
"status": dataset["status"],
"row_counts": manifest.get("row_counts"),
"violations": (dataset.get("violation_report") or {}).get("violations"),
"contract_version": manifest.get("contract_version"),
}, indent=2))
'
printf '%s' "$DATASET_ID" > "$TESSERA_STATE_DIR/dataset_id"

Download quickstart/05-build-dataset.sh — this exact file runs in CI on every change (hosts shown are for this environment).

About the request:

  • window — each example keeps the user's last 8 events inside the sample's fixed time range. Small windows keep this tutorial's models CPU-sized; production defaults keep 1,024.
  • split — deterministic by entity hash: a user is train or val or test, forever, across rebuilds. The build fails if any label lacks both a positive and a negative in every split (insufficient_label_support) — a broken join fails loudly, not silently.
  • The build is content-addressed: re-POST the same request against the same snapshot and you get the same dataset_id back, not a second build.

The manifest is the receipt: row counts per split, per-label supervision accounting (how many targets attached, how many masked and why), the empty violation report, and the pinned fingerprints of the contract, policy and snapshot. If a gate had tripped, the build would end aborted_leakage with the violating attributes named.

Next: train.