> ## Documentation Index > Fetch the complete documentation index at: https://ploti.ai/docs/llms.txt > Use this file to discover all available pages before exploring further. # Sync a local copy > Keep a folder on your machine or server current with a Data Feeds subscription, using one command, from cron or when a webhook arrives `ploti feeds sync` downloads every dataset your Data Feeds subscription delivers into a folder, then keeps that folder current. Data Feeds is Ploti's subscription for keeping a copy of its official county and city records in your own systems. Run sync again, by hand, from cron or when Ploti tells you something changed, and it downloads only the datasets that have a new version. When nothing changed, it changes nothing. ```bash theme={null} ploti feeds sync sub_8f3k2m --to ./ploti-data ``` A few words first, since the rest of the page uses them: * A **subscription** is your standing order with Ploti: the places you serve, such as Orange County, Florida, and optionally the topics you want, such as permits or zoning. Its id looks like `sub_8f3k2m`. Datasets Ploti adds to those places later join it on their own. * A **dataset** is one set of public records, such as a county's building permits. * A **version** is a numbered release of a dataset. Ploti makes a new one each time the county publishes new or changed records. * A **snapshot** is the whole dataset as of one version, delivered as [GeoParquet](https://geoparquet.org) files: Parquet tables with a geometry column, which DuckDB, GeoPandas, QGIS and most warehouses read directly. Sync keeps the latest snapshot of each dataset on disk. It does not apply changes record by record; for that, see [what sync does not do](#what-sync-does-not-do). ## Before you start Sync needs the **Data Feeds** permission, which both `ploti login` and every new API key hold. On your own machine, `ploti login` is enough. On a server, give the CLI an API key. **1. Create the key** In Ploti, a workspace owner or admin opens **Settings, API keys** and creates a key. Every new key can read the workspace's subscriptions. A key belongs to the member who created it and stops working when that member leaves the workspace, so create it from an account that will stay, such as a shared service account. **2. Give it to the CLI** ```bash theme={null} export PLOTI_API_KEY=pmcp_... ``` The CLI needs Node.js 20 or later. Run it with `npx @ploti/cli`, or install it with `npm install -g @ploti/cli` to get the `ploti` command. **3. Find your subscription's id** ```bash theme={null} ploti feeds subscriptions list ``` ## Run it The first run downloads everything. It prints a line per dataset as it finishes, then a summary: ```text theme={null} $ ploti feeds sync sub_8f3k2m --to ./ploti-data ploti: ds_orange_fl_building_permits is at version 42 (1 of 2) ploti: ds_orange_fl_zoning is at version 7 (2 of 2) updated ds_orange_fl_building_permits: new -> version 42 updated ds_orange_fl_zoning: new -> version 7 fetched ds_orange_fl_zoning attachment original sub_8f3k2m -> ./ploti-data: 2 updated (318.4 MB), 0 unchanged, 0 removed, 0 waiting, 1 attachments fetched, through event evt_000000001204. ``` Later runs download only what moved: ```text theme={null} $ ploti feeds sync sub_8f3k2m --to ./ploti-data ploti: ds_orange_fl_building_permits is at version 43 (1 of 1) updated ds_orange_fl_building_permits: version 42 -> version 43 sub_8f3k2m -> ./ploti-data: 1 updated (161.2 MB), 1 unchanged, 0 removed, 0 waiting, 0 attachments fetched, through event evt_000000001231. ``` Before it downloads anything, sync checks that the disk has room for the whole run, and stops with nothing changed when it does not. ## What ends up in the folder ```text theme={null} ploti-data/ .ploti-sync.json what sync holds: each dataset's version, files and attachments ds_orange_fl_building_permits/ snapshot/part-0000.parquet ... the version's snapshot, in parts of up to 256 MB schema.json its columns, and the columns that identify a record manifest.json the version's own description: its number, row counts, source attachments/.pdf ... the files the dataset's records cite ds_orange_fl_zoning/ ... ``` Each dataset gets a folder named by its dataset id, and that folder always holds the current version. Point your reader at `ploti-data/ds_orange_fl_building_permits/snapshot/*.parquet` and the path never has to change: ```sql theme={null} SELECT * FROM read_parquet('ploti-data/ds_orange_fl_building_permits/snapshot/*.parquet'); ``` Every row carries `_ploti_record_id`, an id that stays with the record from version to version, and `_ploti_version`, the version it came from. [Envelope columns](https://ploti.ai/docs/feeds/versions-and-changes.md#envelope-columns) lists every column Ploti adds. Files are named as the version's `manifest.json` names them, so a path on disk is the same path Ploti's API uses. ### Attachments Some datasets come with files: the archived copy of each source Ploti read the records from, and the document or map file it downloaded, such as a zoning ordinance's PDF. A dataset that keeps a file per record, such as a permit's scanned application, names it in the record's cell as `document:`. These are the dataset's [attachments](https://ploti.ai/docs/feeds/versions-and-changes.md#attachments). Sync downloads each one into the dataset's `attachments/` folder, named by its sha256 checksum, and checks the bytes against it. Two records citing the same document share one file, and a file already on disk is never downloaded again, even when the dataset moves to a new version. `.ploti-sync.json` lists each attachment under its dataset with the name Ploti gives it (`original`, `source:` or `document:`), its original filename and its path on disk, so you can join a record's `document:` cell to its file. An attachment that a new version no longer lists stays on disk, marked `"listed": false` in the state file, until you run sync with `--prune`. ## Options | Option | Effect | | - | - | | `--to ` | The folder to keep current. Required. Created when missing. | | `--dry-run` | Print what would change and how much it would download, and download nothing. | | `--prune` | Delete the folders of datasets that left the subscription, and attachments their dataset no longer lists. Without it, both stay on disk and are marked in the state file. | | `--raw` | Print the summary as JSON instead of text. | Each folder keeps one subscription. Sync refuses to write a second subscription into a folder that already holds another. ## Run it on a schedule Sync is safe to run as often as you like, since a run with nothing new downloads nothing. ### From cron Run it every hour, or every night: ```cron theme={null} 0 * * * * PLOTI_API_KEY=pmcp_... npx @ploti/cli feeds sync sub_8f3k2m --to /srv/ploti-data ``` ### When Ploti tells you A webhook on the subscription has Ploti POST each event to you as it happens, so your copy updates within moments of a new version instead of on the next cron run. ```bash theme={null} ploti webhooks create https://example.com/hooks/ploti --subscription-id sub_8f3k2m ``` The reply carries the signing secret, shown once. Store it as `PLOTI_WEBHOOK_SECRET` beside the handler. The handler verifies the POST, answers at once, and runs the sync in the background. The event itself only says that something changed: `ploti feeds sync` brings every dataset up to its current version, so a missed or repeated POST does no harm. ```python theme={null} # pip install flask standardwebhooks import os import subprocess import threading from flask import Flask, request from standardwebhooks.webhooks import Webhook app = Flask(__name__) verifier = Webhook(os.environ["PLOTI_WEBHOOK_SECRET"]) lock = threading.Lock() again = threading.Event() def sync(): # The sync's own lock keeps runs from overlapping. This also queues one # more run for a POST that arrives mid-run, so its version is not left # waiting for cron. if not lock.acquire(blocking=False): again.set() return try: while True: again.clear() subprocess.run(["ploti", "feeds", "sync", "sub_8f3k2m", "--to", "./ploti-data"], check=False) if not again.is_set(): break finally: lock.release() @app.post("/hooks/ploti") def hook(): try: verifier.verify(request.get_data(), dict(request.headers)) except Exception: return "", 401 threading.Thread(target=sync).start() return "", 204 ``` Ploti retries a failed delivery six times, at growing intervals, over about a day and a half. `ploti webhooks test ` sends a signed `ping` to check the handler. Keep the cron job as well, at a slower rate: if the handler is down when a version lands, the next scheduled run still catches up. Two runs on the same folder take turns: the second waits for the first, so a run started by an event never misses the version that event announced. ## How sync keeps your copy whole * **No half versions.** A new version downloads into `.ploti-sync/` inside the folder, gets checked, and then replaces the dataset's folder in one step. A reader, or a crash part way, never finds a folder holding half of one version and half of another. * **Checked bytes.** Every file is checked against the sha256 checksum Ploti gives for it, and downloaded again when it does not match. * **Links that run out.** Download links last an hour. When one runs out during a long run, sync asks Ploti for a fresh link to the same file of the same version and carries on. * **Datasets that leave.** A dataset that leaves the subscription, because you changed its places or your plan no longer covers it, keeps its folder and is marked `removed` in the state file. `--prune` deletes it. * **Datasets not ready yet.** A dataset whose first version is still being built, or whose files are being rebuilt, is listed as `waiting` with the reason. What is on disk stays until its next version is ready. ## What sync does not do * **It loads snapshots, not changes.** Each new version replaces the dataset's folder with the full snapshot. If you load records into a database of your own and want only what was added, changed and removed, apply each version's changes file instead: read the [events](https://ploti.ai/docs/feeds/events.md) after the `eventCursor` that `.ploti-sync.json` records, with `ploti feeds subscriptions events`, as [Run the loop yourself](#run-the-loop-yourself) shows. * **It writes files, not tables.** Sync writes to a folder. To load a database, point your loader at the folder after each run. * **It syncs the whole subscription.** To sync fewer datasets, narrow the subscription itself. ## From Python Ploti's Python SDK has the same command, and leaves the same folder and state file: ```python theme={null} import ploti summary = ploti.feeds.sync("sub_8f3k2m", to="ploti-data") print(summary["updated"], summary["attachments"]["fetched"]) ``` It reads the key from `PLOTI_API_KEY` too, and takes `dry_run=True` and `prune=True` for the options above. It returns the summary `--raw` prints, as a dict. ## Run the loop yourself To apply change files to your own tables, or to load somewhere the sync does not write, run the same loop with the REST API. Create the key and the subscription as above (see [Subscriptions](https://ploti.ai/docs/feeds/subscriptions.md)), then: ### Load the snapshots The manifest first brings the subscription's datasets up to date, then lists every one it still delivers that has a released version and that your workspace's coverage and plan still open, with links to that version's snapshot, and the event cursor it is current as of. ```bash theme={null} curl https://api.ploti.ai/v1/feeds/subscriptions/sub_8f3k2m/manifest \ -H "Authorization: Bearer $PLOTI_API_KEY" ``` ```json theme={null} { "subscriptionId": "sub_8f3k2m", "generatedAt": "2026-10-18T14:02:11.000Z", "eventCursor": "evt_000000000042", "expiresAt": "2026-10-18T15:02:11.000Z", "datasets": [ { "datasetId": "ds_orange_fl_building_permits", "title": "Building permits", "place": "fl-county-orange", "version": 42, "schemaVersion": "2.1", "keyed": true, "lastCheckedAt": "2026-10-18T06:00:03.000Z", "snapshot": [ { "url": "https://...part-0000.parquet", "rows": 184220, "bytes": 21933012, "sha256": "9b1f..." } ], "schema": { "url": "https://...schema.json", "rows": 0, "bytes": 4120, "sha256": "c03a..." }, "attachments": [] } ], "unavailable": [] } ``` Then: * Download each snapshot's parts (GeoParquet) and load them into a table keyed on `_ploti_record_id`. * Record each dataset's `version` as the version you hold. * Save `eventCursor`: it is the last event already reflected in these snapshots. Links expire at `expiresAt`, an hour after the manifest was made. For fresh links to the same versions, get the manifest again with `?asOf=`, or get each version from `GET /v1/feeds/datasets/{dataset}/versions/{version}`. `attachments` lists the files the dataset's records cite, as on [a version](https://ploti.ai/docs/feeds/versions-and-changes.md#attachments). Two kinds of dataset are listed under `unavailable` instead: * **Files being rebuilt,** with reason `rebuilding`. Ask for the manifest again in a few minutes to load it. Its events after `eventCursor` still apply. * **No released version yet,** with no `version` and its `status` as the reason: `building`, `held`, `failed` or `awaiting_release`. Its first version arrives as `feed.dataset.resync_required`. A `503` `plan_unavailable` means Ploti could not read your workspace's plan just now: try again in a minute. ### Apply events Read the events after your cursor, apply them in order, and save `nextCursor` once a page is applied. Only loading a snapshot or applying a changes file changes the version you hold. * Skip an `updated`, `resync_required` or `schema_changed` for a version at or below the one you hold. * Act on a `schema_changed` by changing your table to match, without recording its version as held. It names the same version as the `updated` (MINOR change) or `resync_required` (MAJOR change) that follows it. * Apply an `updated` only when its `previousVersion` is the version you hold. Version numbers can skip. When `previousVersion` is any other version, reload that dataset from its current version instead: read `current.version` from `GET /v1/feeds/datasets/{dataset}`, then load that version's snapshot from `GET /v1/feeds/datasets/{dataset}/versions/{version}`. You now hold that version, so skip the dataset's later events at or below it. ```bash theme={null} curl "https://api.ploti.ai/v1/feeds/subscriptions/sub_8f3k2m/events?cursor=evt_000000000042" \ -H "Authorization: Bearer $PLOTI_API_KEY" ``` | Event | What to do | | - | - | | `feed.dataset.added` | When `data.version` is set, load that version's snapshot from `data.links.version`, replacing any rows you kept for the dataset. When it is null, wait: the first version arrives as `resync_required`. | | `feed.dataset.updated` | When `data.previousVersion` is the version you hold, get the version from `data.links.version` and apply its changes files in order: upsert or delete by `_ploti_record_id`. You now hold that version. Otherwise reload the dataset from its current version. | | `feed.dataset.schema_changed` | Read `data.schemaChange` and change your table to match. The version you hold stays the same: that version's `updated` follows for a MINOR change, or its `resync_required` for a MAJOR one. | | `feed.dataset.resync_required` | Replace the dataset's rows with the version's snapshot. When `data.reason` is `record_ids_changed`, also rewrite the record ids your own systems store with the version's `files.idMap` ([record IDs](https://ploti.ai/docs/feeds/record-ids.md#when-ids-are-replaced)). You now hold that version. | | `feed.dataset.removed` | Stop expecting updates for that dataset. The rows you loaded stay yours. | | `feed.dataset.stale` | The source is a day past its due check; the data may be behind. | | `feed.dataset.delayed` | The next version is held for review; keep the version you hold until the next event. | Each version's links come back signed for an hour from `GET /v1/feeds/datasets/{dataset}/versions/{version}`, which `data.links.version` names. It signs links only for a dataset one of the workspace's subscriptions delivers. For any other, it answers `409` `not_subscribed`. A `409` (`not_subscribed`, `outside_coverage` or `not_deliverable`) or a `404` while you apply older events means the dataset has since left your subscriptions. Skip that dataset's events up to its `feed.dataset.removed`. A version past retention answers `410` `expired`, and so does a version Ploti released again and rebuilt under a new number. Reload that dataset from its current version, as above. Add a webhook for the subscription's events, as above, and use it as the signal to read the log. The log stays the source of truth. See [Events and webhooks](https://ploti.ai/docs/feeds/events.md). ```bash CLI theme={null} ploti feeds sync sub_8f3k2m --to ./ploti-data ploti feeds sync sub_8f3k2m --to ./ploti-data --dry-run ``` ```python Python theme={null} import ploti summary = ploti.feeds.sync("sub_8f3k2m", to="ploti-data") ``` ```bash REST theme={null} # Sync runs on your machine. Over REST, start from the manifest and load each snapshot. curl https://api.ploti.ai/v1/feeds/subscriptions/sub_8f3k2m/manifest \ -H "Authorization: Bearer $PLOTI_API_KEY" ``` ```text MCP theme={null} Data Feeds is not offered over MCP. Use the CLI, Python or REST with an API key that holds the Data Feeds permission. ``` This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.