> ## Documentation Index > Fetch the complete documentation index at: https://docs.closient.com/llms.txt > Use this file to discover all available pages before exploring further. # Data Delivery > Nightly delivery of your scan and page-visit aggregates to your own S3, R2 or Postgres. Closient Data Delivery writes your organization's scan and page-visit analytics into a destination you own, every night. It is provisioned per organization and requested from the [Data Delivery page](https://www.closient.com/data-delivery/) on the Closient site. This page is the v1 data contract: what is delivered, in what shape, and what you can rely on. ## What this is You choose the destination and supply the connection details. We deliver Parquet or CSV files into object storage, or rows into a Postgres schema. Nothing about the data or its shape changes with the destination. The data delivered is the same daily aggregates that power your Closient analytics dashboard. It is retained by Closient indefinitely, so the first delivery includes your full history back to your first scan. ## Delivery | Item | Contract | | ---------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Cadence | Once per day, starting at 04:00 UTC, covering the previous UTC calendar day. | | Unit of delivery | One partition per table per UTC date. A partition is always written whole. | | Re-delivery | A date partition may be rewritten for up to 3 days after first delivery as late enrichment lands. Treat `date` as the replace key: replace the partition, never append to it. | | Backfill | On activation we deliver every date from your first scan to yesterday. You can request a backfill of any date range at any time from the dashboard. | | Time zone | All dates and timestamps are UTC. `hour` breakdowns are UTC hours. | | Egress | Deliveries originate from a fixed IP address shown in your dashboard, so you can allowlist it. The address is not published anywhere public. | | Ordering | Tables within a day are delivered in the order listed under [Tables](#tables). The manifest is written last, so its presence means the day is complete. | ## Formats and layout ### Object storage (S3, R2, GCS) Default format is Parquet with Snappy compression. CSV (UTF-8, RFC 4180, header row, ISO 8601 dates) is available as an option per destination. ``` /closient/v1//dt=YYYY-MM-DD/.parquet /closient/v1/_manifest/dt=YYYY-MM-DD.json ``` Hive-style `dt=` partitioning is used so DuckDB, Spark, Athena, Snowflake external tables and BigQuery external tables read the tree directly. The `products` table is a full snapshot each night and lives under the same `dt=` partition so a day is self-contained. The manifest lists every object written for that date with row count, byte size, SHA-256 and the schema version: ```json theme={null} { "schema_version": "1", "organization_id": "org_2xk9…", "date": "2026-09-06", "generated_at": "2026-09-07T04:03:12Z", "files": [ {"table": "scans_daily", "path": "closient/v1/scans_daily/dt=2026-09-06/org_2xk9.parquet", "rows": 412, "bytes": 30188, "sha256": "…"} ] } ``` ### Postgres We write into one schema, `closient` by default, that you create and grant to the user you give us. Tables are created on first delivery with the primary keys listed under [Tables](#tables). Each day is applied as an upsert on the primary key inside one transaction per table. A `closient._deliveries` table records every run with the same fields as the manifest. Requirements: TLS with a certificate we can verify (`sslmode=verify-full`), network reachability from the egress IP shown in your dashboard, and a user with `CREATE` on the schema and `INSERT`, `UPDATE`, `SELECT` on its tables. We never touch objects outside that schema. ## Tables Column types are given as Parquet / Postgres. ### `products`: dimension, full snapshot nightly Primary key: `product_id` | Column | Type | Notes | | ------------------ | ---------------------------- | ----------------------------------------------------- | | `product_id` | string / text | Closient short id, stable for the life of the product | | `gtin` | string / text | GTIN-14, zero-padded | | `brand_id` | string / text | Closient short id | | `brand_name` | string / text | As shown in your catalog on the snapshot date | | `product_name` | string / text | As shown in your catalog | | `resolver_enabled` | boolean | Whether the product currently resolves | | `created_at` | timestamp(UTC) / timestamptz | | | `snapshot_date` | date | The `dt=` partition date | ### `scans_daily`: one row per product per day Primary key: `(date, product_id)` A scan is a resolver hit on one of your GS1 Digital Link URLs. Counts are final once the partition stops being rewritten (see [Delivery](#delivery)). | Column | Type | Notes | | -------------------------- | -------------- | -------------------------------------------------------------------------------------------------------------------------------------- | | `date` | date | UTC calendar day | | `product_id` | string / text | Joins to `products` | | `gtin` | string / text | Denormalized for convenience | | `total_scans` | int64 / bigint | All scans including bots | | `billable_scans` | int64 / bigint | Scans that count toward plan usage | | `bot_scans` | int64 / bigint | Scans classified as automated | | `ai_bot_scans` | int64 / bigint | Subset of `bot_scans` attributed to an AI provider | | `hosted_page_scans` | int64 / bigint | Scans that rendered a Closient hosted page | | `redirect_only_scans` | int64 / bigint | Scans that redirected to your destination | | `unique_visitors_estimate` | int64 / bigint | Distinct visitors, estimated with a HyperLogLog sketch, about 1.6% standard error. Not summable across days; sum `total_scans` instead | ### `scans_daily_breakdown`: long table, one row per product, day, dimension, key Primary key: `(date, product_id, dimension, key)` Rows exist only for keys with at least one scan. The sum of `scans` over all keys of one dimension equals `total_scans` for that product and day except where noted. | Column | Type | Notes | | ------------ | -------------- | ------------------------------ | | `date` | date | | | `product_id` | string / text | | | `gtin` | string / text | | | `dimension` | string / text | One of the values below | | `key` | string / text | Meaning depends on `dimension` | | `scans` | int64 / bigint | | | `dimension` | `key` values | | ------------- | ---------------------------------------------------------------------------------------------------------------- | | `country` | ISO 3166-1 alpha-2, or empty when unknown | | `region` | ISO 3166-2 subdivision code, for example `US-TX` | | `city` | City name as geolocated, for example `Austin` | | `device` | `Mobile`, `Tablet`, `Desktop`, or empty when unknown | | `context` | GS1 Digital Link context key, single character, as carried in the scanned URL | | `hour` | `0` to `23`, UTC hour of the scan | | `ai_provider` | AI crawler provider name. Sums to `ai_bot_scans`, not `total_scans` | | `ai_purpose` | AI crawler purpose classification. Sums to `ai_bot_scans` | | `lot` | Raw AI(10) lot value carried in the scanned URL. Only scans carrying a lot appear | | `link_type` | Raw `linkType` requested by the scanning client, for example `gs1:pip`. Only scans requesting a link type appear | ### `page_visits_daily`: one row per attribution tuple per day Primary key: `(date, gtin, page_type, source, country, is_bot)` A page visit is a render of a Closient hosted page in your organization. A scan that lands on a hosted page produces one scan and one page visit with `source = resolve`. Visits from search, AI assistants and links are page visits with no matching scan. | Column | Type | Notes | | ----------- | -------------- | ------------------------------------------------------------------------------------------------- | | `date` | date | | | `gtin` | string / text | GTIN-14 of the product page, empty for brand and custom pages | | `page_type` | string / text | `product_info`, `brand_page`, `recipe`, `instructions`, `recall_notice`, `custom` | | `source` | string / text | `resolve`, `ai_referral`, `organic_search`, `direct`, `referral`, `social`, `internal`, `unknown` | | `country` | string / text | ISO 3166-1 alpha-2, or empty when unknown | | `is_bot` | boolean | Whether the visits were classified as automated | | `visits` | int64 / bigint | | ### Available on request Hourly grain (`scans_hourly`, `page_visits_hourly`) with the same columns keyed on `bucket_start` instead of `date`. Hourly data is retained by Closient for a shorter window than daily, so backfill depth is limited. ## What is not delivered Raw scan events, IP addresses, user-agent strings, visitor identifiers, HyperLogLog sketches, and anything that describes an individual visitor. The aggregates above are the privacy-safe layer that Closient itself retains after raw events are purged. If you need per-scan events in real time, use [Scan Webhooks](/guides/webhooks), which are a separate feature. ## Schema versioning * Adding a column, a `dimension` value, or an enumeration value is not a breaking change and does not change the version. Read by column name, not position, and tolerate unknown enumeration values. * Renaming or removing a column, changing a type, or changing a primary key is a breaking change. It ships under a new prefix (`closient/v2/`, schema `closient_v2`) and both versions are delivered in parallel for at least 90 days. We announce breaking changes by email 30 days ahead. * Every manifest and every `_deliveries` row carries `schema_version`. ## Destinations and credentials | Destination | What you give us | What we hold | Notes | | ----------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------- | ------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | | Amazon S3 | Bucket, prefix, region, and an IAM role ARN that trusts Closient's AWS principal with an External ID we display to you | No secret of yours | Role needs `s3:PutObject` and `s3:AbortMultipartUpload` on the prefix and `s3:ListBucket` on the bucket. Preferred path | | Amazon S3 (static keys) | Access key pair scoped to the bucket | The key pair, encrypted at rest | Accepted, role preferred | | Cloudflare R2 | Bucket, account id, and an R2 API token with Object Read & Write on that bucket | The token, encrypted at rest | | | Postgres | Host, port, database, schema, user, password, and the server CA if private | The password, encrypted at rest | TLS required, egress IP must be allowed | | Google Cloud Storage, Snowflake, BigQuery, SFTP | On request | | Quoted individually | Credentials you give us are encrypted at rest, visible to no one at Closient in plain text, rotatable by you at any time from the dashboard, and deleted within 24 hours of deactivation. Object-storage destinations are write-only from our side. ## Monitoring and failure handling * Every run is recorded in the dashboard: date, tables, rows, bytes, duration, outcome. * A failed run is retried hourly up to 6 times. * An authentication or permission error (expired key, revoked role, rejected password) is emailed to your organization's technical and billing contacts immediately, because it never self-heals. * Any other failure is emailed after 2 consecutive failed nights, so a transient on your side does not page you and staleness stays under 48 hours. Errors are sent as we saw them, redacted of credentials. * A missed day is delivered automatically on the next successful run. Nothing is skipped silently. ## Activation and billing * Provisioned per organization. Activation completes within 10 business days of receiving valid destination details. * Billing starts on the day of your first successful delivery, not before. * Price: \$49 per month per destination. No setup fee. * Available on the Business plan.