Skip to main content
An event is a short message saying that something happened to a dataset in your Data Feeds subscription: a new version, a schema change, a dataset joining or leaving. Each subscription keeps its events in one numbered log with no gaps, so reading after the last event you applied never misses one. A webhook, an https address of yours, receives the same events the moment they happen, so you don’t have to keep checking. Treat the webhook as the signal and the log as the record.

​
The feed event

Every event Ploti sends, in the log and to a webhook, has the same four fields as Ploti’s other webhook events: id, type, timestamp and data. A feed event adds subscriptionId, the subscription it belongs to.
links are absolute API URLs. Call them with your API key: links.version returns the version with signed links to its files, for a dataset one of the workspace’s subscriptions delivers.

​
Event types

Every event’s data carries datasetId. feed.dataset.added names the version to load. When version is set, load that version’s snapshot, replacing any rows you kept for that dataset. When it is null, the dataset has no released version yet, and its first version arrives as feed.dataset.resync_required with reason first_version. feed.dataset.resync_required carries one reason. When several apply, the first of these wins:
  1. record_ids_changed
  2. first_version
  3. previous_unavailable, when the version does not change the schema
  4. key_columns_changed, when the schema change includes the key
  5. schema_changed
What each means:
  • first_version: the dataset’s first version.
  • schema_changed: the version makes a MAJOR schema change, or changes the schema while it needs a reload for another reason. A MINOR change on its own arrives as feed.dataset.updated.
  • key_columns_changed: as schema_changed, when the change includes the key columns. A record keeps its id when it is matched to the version before.
  • record_ids_changed: the version changes ids you hold, because Ploti’s staff replaced the dataset’s ids or Ploti’s matching agent found that a record you hold under one id continues one under another. The version’s files include an id map from each id you hold to the one it carries now. It can change a few ids or all of them. See when ids are replaced.
  • previous_unavailable: the snapshot of the version before it is past retention, or a version subscribers already had was released again, and there is no id map. After a version is released again, the dataset’s next version sent is a resync_required, even when a newer release arrives before the rebuild. Its reason is previous_unavailable unless that version also changes the schema, the key or record ids. Its records are matched to what Ploti kept for the version released again, so a record it matches keeps its id.
feed.dataset.removed reasons:
  • left_coverage: the subscription still matches the dataset, but it is outside your workspace’s coverage.
  • plan_changed: the dataset is still in your workspace’s coverage, but its plan no longer opens it.
  • subscription_changed: the dataset no longer matches the subscription’s places, topics or datasets. This includes a change you made, and Ploti filing the dataset under another topic or place.
  • no_longer_deliverable: the dataset is no longer delivered through Data Feeds.
When Ploti cannot read your workspace’s plan at that moment, it removes nothing for it and checks again later. feed.dataset.schema_changed carries schemaChange: that version’s own change, measured from the schema of the last version released before it. feed.dataset.stale is sent once per missed check. feed.dataset.delayed says the dataset’s next version was built but is held for review (see holds). Nothing more arrives for that dataset until the hold ends.
  • version is the version you hold, which stays current meanwhile.
  • since is when the held version was built.
  • It is sent once per held version, including to a subscription that adds the dataset while it waits.
Its reason is one of:
  • schema_review: the source changed its columns in a way that would break a consumer (a column removed or retyped, or the key changed), and Ploti is checking the change is real.
  • data_check: the version looks wrong for the source (far fewer rows than before, a key with duplicates or nulls, or record ids that would turn over wholesale), and Ploti is checking it against the source.
  • processing: Ploti could not finish matching the version’s records to the ones you hold.
Nothing needs doing on your side. The hold ends one of three ways:
  • Ploti releases the version. Its usual updated or resync_required follows, after schema_changed when the schema changed.
  • Ploti discards it. The dataset’s next version follows from the one you hold.
  • A newer version replaces it.
A delay can end without any event for a while, when the version was discarded and the source has not changed since. Feed events use the feed. prefix and are separate from the dataset.changed events a dataset watch sends, which carry a different version number and go to your other webhooks.

​
Order

Versions reach subscribers in order for each dataset, and events for one dataset arrive in order. Events for different datasets can interleave. A version held for review is not sent, and neither is anything after it, until Ploti’s team release or discard it. feed.dataset.delayed says when one is waiting. A newer version of the dataset replaces a held one, so a source that is fixed unblocks the feed. See holds. Version numbers can skip: a held version that was discarded or replaced, and a version rebuilt under a new number, leave gaps. Follow previousVersion, on updated events and in the versions API, not the numbers.
  • Apply an updated only when its previousVersion is the version you hold.
  • When it is any other version, reload that dataset from its current version: read current.version from GET /v1/feeds/datasets/{dataset}, then load that version’s snapshot from GET /v1/feeds/datasets/{dataset}/versions/{version}.

​
Reading the log

Returns events and nextCursor. Store nextCursor only after you have applied the page’s events, and pass it next time. It advances even when a types filter matches nothing on the page. limit is 100 unless given, at most 500. Only loading a snapshot or applying a changes file changes the version you hold. Two rules make the log safe to replay:
  • Skip an updated, resync_required or schema_changed for a version at or below the one you hold, including after you reload a dataset part way through the log.
  • A schema_changed names the same version as the event that follows it: an updated for a MINOR change, or a resync_required for a MAJOR change or a reload for another reason. Act on it by changing your table to match, but do not record its version as held.
A 409 (not_subscribed, outside_coverage or not_deliverable) or a 404 from links.version while you apply older events means the dataset has since left your subscriptions. Skip that dataset’s events up to its feed.dataset.removed. Events are kept for 90 days:
  • A cursor older than retention, whose following events have been pruned, returns 410 expired, including when every event has been pruned. Reload from the subscription’s manifest, which gives a fresh cursor.
  • Reading with no cursor starts at the oldest kept event. Once the first events have aged out, it returns 410 too: start from a manifest’s eventCursor.
  • A cursor past the subscription’s newest event is not one it gave, and returns 400 invalid_input.

​
Webhooks for a subscription

Create a webhook for a subscription’s events with an API key that has both the webhooks and feeds permissions. Every new key has both. The webhook belongs to the member whose key created it. It stops when that member leaves the workspace, and when the subscription is deleted, so create it with a key from an account that will stay, such as a shared service member.
events is optional. Left out, the webhook gets every feed event type, which is what most receivers want. A filter that leaves out feed.dataset.added or feed.dataset.schema_changed never tells you about new datasets or schema changes. Each POST’s body is the event object above, signed with Standard Webhooks using the secret returned once when the webhook is created.
  • Verify the signature, answer 2xx quickly and do the work afterwards.
  • Failed deliveries are retried six times, at growing intervals, over about a day and a half.
  • An answer of 413 is not retried, and an answer of 410 turns the webhook off.
  • Delivery is at least once, so skip an event whose id in the body you have already applied. The webhook-id header names the delivery, not the event.
GET /v1/webhooks, DELETE /v1/webhooks/{webhookId} and POST /v1/webhooks/{webhookId}/test (ploti webhooks list, delete and test) show or touch a subscription’s webhooks only for a key with the feeds permission. POST /v1/webhooks/{webhookId}/test sends one signed ping now: {"id", "type": "ping", "timestamp", "data": {}}. It is not a feed event, so do not apply it. Treat a webhook as the signal to read the log, not as the record. If your receiver was down, the log still has every event, and the event id tells you whether you have applied one already.