Skip to main content

Connect Google Pub/Sub

Availability: Available

Google Pub/Sub: Create the source through the API as shown below. The console source catalog does not list this intake yet.

Tessera pulls a subscription you own — no container to run, no credentials to hand over. Create a kind: stream source carrying the subscription path and a declarative mapping config (the same bus-mapping format the planned Kafka bridge uses), grant Tessera's intake service account roles/pubsub.subscriber on that one subscription, and JSON messages start landing as canonical events.

Prerequisites​

  • An API key with the admin scope (to create the source).
  • A Pub/Sub topic in your GCP project + a dedicated subscription for Tessera (don't share the one your own consumers use: Tessera acks what it ingests).

Grant the intake (least privilege, no keys)​

Bind roles/pubsub.subscriber for Tessera's intake service account — the tessera-ingest-api@… address shown in the console's source setup panel — on the one subscription, not the project:

gcloud pubsub subscriptions add-iam-policy-binding tessera-intake-sub \
--member="serviceAccount:INTAKE_SERVICE_ACCOUNT" --role="roles/pubsub.subscriber"

Create the source​

The mapping block says how one message becomes one event — JSON Pointers into the message body. It is validated at write time and again at load: unknown fields or a version other than 1 are rejected outright.

{
"name": "match-results",
"kind": "stream",
"config": {
"subtype": "pubsub_subscription",
"subscription": "projects/acme-prod/subscriptions/tessera-intake-sub",
"mapping": {
"version": 1,
"encoding": "json",
"entity_type": "player",
"entity_id_pointer": "/player/id",
"event_type_source": { "static": "match.finished" },
"timestamp_pointer": "/finished_at",
"event_id_pointer": "/match_id",
"attributes": "passthrough_flat",
"attribute_drops": ["player_id"]
}
}
}

Delivery semantics​

  • Exactly-once landing. Pub/Sub redelivers at-least-once; when event_id_pointer is null (or resolves empty) the intake derives event_id from the subscription path + message id, so redeliveries dedup downstream. Messages are acked only after the ingest publish is acknowledged.
  • Unmappable messages are kept, not dropped. Non-JSON payloads, missing entity ids, entity ids that are not identifiers (true, 12.5, NaN, a whole number above 2^53 − 1 sent as a JSON number; see the rule) and bad timestamps park in the project's dead-letter view with reason: mapping_failed and the raw payload metadata. Fix the mapping config (PATCH /sources/{id}), then POST /events/deadletter:replay — replay re-runs the mapping with the current config.
  • Isolation. A subscription that becomes unreadable (grant revoked, subscription deleted) flips the source to status: errored and is skipped; other sources are unaffected. Fix the grant, then PATCH the source back to status: active. A temporary failure (a Pub/Sub quota or server error) never changes the status: the intake retries that source with a backoff of up to 5 minutes, and nothing is acked until it is ingested.
  • Metering. Ingested events count as events_ingested, exactly like events sent to POST /v1/events.

Test and verify​

Publish one message and read the entity's timeline:

gcloud pubsub topics publish match-results \
--message='{"player":{"id":"p1"},"match_id":"m1","finished_at":"2026-07-11T12:00:00Z"}'

GET /v1/entities/player/p1/events shows the mapped match.finished event within seconds.

Troubleshooting​

SymptomCauseFix
source flipped to status: erroredgrant missing or on the topic instead of the subscriptionbind roles/pubsub.subscriber on the subscription, then PATCH the source to active
dead-letter items with reason: mapping_failedpointer misses or unparseable timestampsfix config.mapping, then POST /events/deadletter:replay
duplicates on the timelinenone expectedredeliveries dedup via derived event_ids — check whether the producer publishes distinct payloads
backlog growing on the subscriptionquota exhausted (429s downstream)check GET /usage/quotas