Skip to main content

Warehouse exports

Land a slot's predictions as queryable, governed tables in your own warehouse — BigQuery or Snowflake — or as Parquet in GCS or S3. One request to POST /slots/{slot}/warehouse-exports picks the column groups, the destination and how rows land in the table:

{
"destination": {
"kind": "snowflake",
"credential_id": "src_01j9v10lakewriter000000000",
"database": "ANALYTICS",
"schema": "TESSERA",
"table": "HOST_SIGNALS"
},
"include": ["scores", "signals", "segments"],
"mode": "merge",
"refresh_first": true
}

The answer is the export's warehouse record: its destination, the column groups it wrote, the write mode it applied and, for a scheduled export, schedule_id and scheduled_for. Poll GET /slots/{slot}/warehouse-exports/{export_id} until status is succeeded or failed; GET /slots/{slot}/warehouse-exports lists every export of the slot the same way, newest first.

Warehouse exports and the classic export record

The classic export operations keep their original record for existing integrations, and that record cannot state a Snowflake destination or a column group beyond embedding and scores. So POST /slots/{slot}/exports refuses such a request (400 invalid_argument naming this endpoint), GET /slots/{slot}/exports/{export_id} refuses such an export (400 failed_precondition), and GET /slots/{slot}/exports leaves it out. Every other export appears on both: the classic record simply omits mode and the schedule fields, which only the warehouse record carries.

Entity baseline and event slots

A slot whose active bundle is an entity baseline model exports its scores only, through POST /slots/{slot}/exports (a BigQuery mode is applied there but not echoed in the classic record). POST /slots/{slot}/warehouse-exports and export schedules accept sequence-model slots only: on an entity baseline or event slot they answer 400 failed_precondition, reason model_family_not_supported.

What the tables claim

Until a slot enables WP-M16/WP-V14, a quiet entity's refreshed rows are byte-identical across cutoffs — the customer-facing claim is "fresh as-of-cutoff tables on a cadence", with the absence-semantics dependency declared (docs + manifest), never hidden.

Concretely: every row is the entity's latest stored state when the export runs — with refresh_first: true, scored as of the chained refresh's cutoff. An entity with no new events keeps byte-identical value columns from one cutoff to the next, and its as_of column is NULL: the export is fresh as of the cutoff, not an observation-time re-score. Every manifest.json states the same claim in its freshness block.

Column groups​

include selects from a closed set (unknown groups are rejected with 400 invalid_argument). Omitting it exports embedding + scores — exactly the classic export layout. Columns appear in this order:

ColumnTypeGroup
entity_typestringalways
entity_id — or entity_hash with pseudonymize: true (never both)stringalways
embeddinglist<float32>embedding
score__{label}, score__{label}__{class}float, one per binary/regression head and per multiclass classscores
model_fingerprint, bundle_idstringalways
event_cursor_timestamp, event_cursor_event_idstringalways
computed_at — when the state row was writtenstring (RFC 3339)always
behavior_surprisefloat in [0, 1]signals
segment_idstring, NULL when unsegmentedsegments
propensity__{event_type} (one per shortlist event type)floatpropensities, signals
propensity_statusJSON text: the per-type calibration status map, NULL for rows scored by older bundlespropensities, signals
evidenceJSON text: the top contributing transitions and attributes behind behavior_surpriseevidence, signals
framestring: the surprise reference frameevidence, signals
as_ofUTC timestamp: the stored observation time, NULL until the slot enables observation-time re-scoringas_of

include: ["signals"] alone lands the full evidence-bearing row. Event types map to column names with . → _ (login.failed → propensity__login_failed). Per-row absence is always NULL, never an error, and every value is byte-for-byte what was stored when the entity was scored — JSON blocks are serialized exactly as stored (sorted keys, compact). Two rules keep the layout honest:

  • signals, propensities and evidence need a model whose bundle enables behavioral signals; otherwise the request fails with 400 failed_precondition naming runtime.signals.enabled.
  • If two shortlist event types map onto the same column name (for example login.failed and login_failed), the request fails with 400 invalid_argument naming both. The manifest records the propensity_columns mapping so you can join column names back to event types.

JSON columns are text: query them with PARSE_JSON("EVIDENCE"):transitions in Snowflake or JSON_QUERY(evidence, '$.transitions') in BigQuery.

Table write modes​

mode applies to warehouse destinations only (bigquery, snowflake; sending it with gcs or s3 is 400 invalid_argument):

ModeSemanticsBigQuerySnowflake
replace (default)the table holds exactly this snapshotWRITE_TRUNCATE load — the table takes the snapshot's schemastaging load, then DELETE + INSERT in one transaction — a failure leaves the previous rows
appendscore history keyed by (entity, computed_at)staging load, then one MERGE on identity + computed_atstaging load, then one MERGE on identity + computed_at
mergeone row per entity, updated in placestaging load, then one MERGE on the identity keystaging load, then one MERGE on the identity key

append adds a row only when an entity's state changed since a previous export (a new computed_at). A state that already landed is refreshed in place — a stored state never changes, so this only fills columns you added to include later. Re-running an export, or exporting entities whose state did not change, never duplicates history. merge upserts on entity_type + entity_id, or on entity_hash alone when the export is pseudonymized. Per-export staging tables are named {table}__tsr_{export_id}, expire within a day and are dropped when the export finishes — also when it fails. rows_exported and the export_rows metering count the rows an export delivered to the load; rows for states already in the table are refreshed rather than added.

For append and merge, column growth across runs is additive: new groups add nullable columns, and a column whose type differs from the existing table fails the export with a typed schema_conflict error naming the column — Tessera never coerces or drops your columns. Columns are never dropped either: when a later export selects fewer groups, merge leaves the columns it no longer writes with the values they last received, and new append rows leave them NULL. BigQuery replace is the exception by design: WRITE_TRUNCATE replaces the table's schema with the snapshot's.

These SQL-built writes (every Snowflake export, BigQuery append and merge) need column names that are plain identifiers, unique ignoring case. An export column that is not — for example a score__{label}__{class} column built from a free-form multiclass class name — is rejected with 400 failed_precondition naming the column.

BigQuery​

Rows are staged as Parquet in Tessera's export bucket and loaded as your project's export service account (shown in the permission_denied error details); load and query jobs run in your destination project. Grant the service account:

  • roles/bigquery.dataEditor on the destination dataset (create, load, MERGE, and drop the per-export staging table);
  • roles/bigquery.jobUser on the destination's project.

The grant is verified with a dry run before any export is accepted. If Tessera cannot act as your project's export service account yet (it is not provisioned), the export is refused with 400 failed_precondition, reason export_identity_not_provisioned, naming the service account; retry once it exists. Each project acts only as its own service account, so a grant to another project's account never lets your export through. append and merge exports embed only validated identifiers in SQL: project, dataset and table names must use letters, digits and _ (plus - in project ids). Tables they create have only NULLABLE columns. A REQUIRED column the export does not write — for example entity_id in a table first written with replace, now targeted by a pseudonymized merge — fails the export with schema_conflict before anything changes. Other BigQuery job failures fail it with destination_write_failed.

Snowflake​

credential_id is the id of a Snowflake source of the same project and environment (see Snowflake connector): exports act as that source's key-pair user — the same account, warehouse, role and vaulted private key — so there is no second credential to manage, and secret material never appears in requests, job records, webhooks or logs. Any other credential_id is 404 not_found; a disabled source is 400 failed_precondition until you enable it again.

Each export creates a temporary internal stage tessera_export_{export_id}, PUTs the Parquet parts, copies them into a temporary staging table with MATCH_BY_COLUMN_NAME = CASE_INSENSITIVE, applies the mode, and drops the staging table and the stage — also when the export fails. Database, schema and table are double-quoted (case-sensitive: give them exactly as stored). Tables Tessera creates use upper-case column names, so you can query them unquoted (SELECT entity_id, behavior_surprise FROM …).

Minimal grant set for the source's role:

GRANT USAGE ON WAREHOUSE EXPORT_WH TO ROLE TESSERA_WRITER;
GRANT USAGE ON DATABASE ANALYTICS TO ROLE TESSERA_WRITER;
GRANT USAGE ON SCHEMA ANALYTICS.TESSERA TO ROLE TESSERA_WRITER;
-- the per-export stage and temporary staging table, and the target table
-- on the first export:
GRANT CREATE STAGE, CREATE TABLE ON SCHEMA ANALYTICS.TESSERA TO ROLE TESSERA_WRITER;

A table the first export creates is owned by that role, which covers everything later exports do (INSERT, UPDATE, DELETE, and ALTER TABLE … ADD COLUMN when you add column groups). To export into a table you created yourself, transfer its ownership to the role, or grant SELECT, INSERT, UPDATE, DELETE on it and keep the column set fixed (adding columns needs ownership). A missing grant fails the export with permission_denied; the error details carry the Snowflake username so you know which user to grant.

Schedules​

An export schedule (sch_ id) binds one full export spec — destination, include, mode, pseudonymize, refresh_first and a cutoff lag — to a UTC cadence:

{
"cadence": {"kind": "daily", "at_utc": "05:00"},
"export": {
"destination": {
"kind": "bigquery",
"gcp_project": "acme-analytics",
"dataset": "tessera",
"table": "account_scores"
},
"include": ["scores", "signals"],
"mode": "append",
"refresh_first": true,
"cutoff_lag_seconds": 0
}
}

Each tick runs at most one chained refresh (at scheduled_for − cutoff_lag_seconds), exactly one export, one delivery, one export.* webhook, one warehouse_export.* webhook and one export_rows metering message; after downtime only the latest missed tick runs. The export's warehouse record carries schedule_id and scheduled_for, and its manifest records the chained refresh's {refresh_id, cutoff, model_fingerprint}.

The export spec is validated when you create or change the schedule, exactly as POST /slots/{slot}/warehouse-exports validates a request, including the destination grant probes. A tick that cannot run any more (its slot was removed) is skipped: it creates no export and sends no webhook, and the schedule's last_run stops advancing.

Manage schedules with GET, PATCH and DELETE on /slots/{slot}/export-schedules/{schedule_id}:

  • PATCH with {"status": "paused"} stops ticks; "active" resumes the schedule, and only its latest missed grid point runs.
  • PATCH with export replaces the whole export spec, which is validated again.
  • Each schedule's last_run names the newest export one of its ticks created, with that export's status.

In the console, open a slot and choose its Exports tab to list the slot's schedules, add one, pause or resume it, or delete it. Each change opens a confirmation that lists the values it sets before anything is saved. The tab shows the newest run the server recorded for each schedule and does not compute a next run time. On a slot with observation-time serving enabled, a cutoff later than now is refused, as with a direct export request.

Settings, then Schedules, lists every schedule of the project and environment in one read-only, paged table: source syncs, exports, re-score sweeps and alert rules, and any kind added later (shown by its own name). It is one read of GET /schedules. Each row shows what the schedule is for, its cadence, whether it is active or paused, the next run the server computed (a paused schedule has none; the console never works one out itself), the last run, and the last outcome of its own kind with the consumer's code and message. Filter by kind, by active or paused, or, with a link from a source or slot, by that target; the filters are kept in the page address. When a target filter matches nothing on a page the list keeps reading further pages, and says so if it stops. Each row links to the page where the schedule is paused, resumed or edited (the source, the slot's Exports tab, the slot's observation-time panel or its alert rules); an alert-rule schedule evaluates every rule of its slot and cannot be paused by itself, which the row says. Nothing is changed from this list.

The customer_map kind. A project whose customer map has customer groups has one more schedule in that list, of kind customer_map (one sch_ id for each project and environment). It is not an export schedule: it has no /slots/{slot}/export-schedules/{schedule_id} record, and you do not pause, resume or edit it from here. The map's own settings (frequency, time and time zone, pause, cooldown and pin) decide when it runs, and an Admin changes them with PUT /customer-map/settings or in the console's Customer map settings. So its row has no next run, because the settings decide the day, the local time and the pin; its hint reads "Updates follow the Customer map settings." and its link opens /customer-map/settings. Its last run and last outcome are those of the newest scheduled update that finished, or of a later scheduled run that failed before it could start one, so a tick the map skipped (for example while a map is pinned) is never shown as a success. Each scheduled update is a projection_2d export that publishes export.completed or export.failed and meters no export_rows; the write mode, the destination and the cutoff lag described above do not apply to it.

Governance​

  • Suppression first. Every exported row passes the delivery suppression filter before any destination write; suppressed entities are absent from the table and from the manifest row_count, and the manifest's suppression block counts what was withheld.
  • Read-only. Exports read stored state and never write model state; the only scoring is the standard batch refresh that refresh_first chains.
  • Provenance. manifest.json (beside the staged parts in Tessera's export bucket) records the bundle fingerprints, cutoff, columns, resolved groups, mode, the propensity column mapping, the schedule tick and the chained refresh fingerprint.
  • Metering. Each delivery meters export_rows with {slot_id, destination_kind} — snowflake included.
  • Webhooks. Each export ends, as before, with one export.completed or export.failed webhook whose data has the classic export record's fields (never the write mode or the schedule fields). It also ends with one warehouse_export.completed or warehouse_export.failed webhook whose data is the warehouse record (write mode and schedule fields included), with output summarized as the manifest URI, file_count and table; read the full file list from the GET. Webhooks are sent after the export finishes; in the rare case of a crash at that moment one can be missing, so reconcile from the GET when an export ends without one. These types reach only endpoints that list them by name, never "*".
  • Scopes. Creating exports and schedules needs the admin scope; reads need read.