Caching & ETags

The immutability contract, what you can safely pin forever, and what you cannot.

The Worker is a cache-miss handler, not a request handler. Reads dwarf writes by orders of magnitude and nearly all of them should be answered by the CDN without reaching our code at all.

That only works if the cache headers are honest, so here is exactly what they promise.

The four cache classes

Class	Paths	`Cache-Control`
Immutable	`/api/v1//{YYYY-MM-DD}.` older than 48h	`public, max-age=31536000, immutable`
Settling	`/api/v1//{YYYY-MM-DD}.` within 48h	`public, max-age=300, s-maxage=600, stale-while-revalidate=3600`
Rolling	`latest.`, `series/`, `/api/v1/all/latest.*`	`public, max-age=60, s-maxage=300, stale-while-revalidate=86400, stale-if-error=604800`
Meta	`meta.json`, `coverage.json`, `llms.txt`, `openapi.json`	`public, max-age=300, s-maxage=600, stale-while-revalidate=3600`

Every response also carries X-Snapdata-Cache-Class, so you can see which one applied from curl -I alone:

curl -sI https://snapdata.dev/api/v1/gold/in/2026-08-11.json | grep -i 'cache'

cache-control: public, max-age=31536000, immutable
x-snapdata-cache-class: immutable

What `immutable` promises

A dated URL older than 48 hours will return the same bytes forever.

That is the contract, and it is the reason this project exists. A backtest run in 2029 against /api/v1/gold/in/2026-08-11.json gets exactly what a backtest run today gets. Pin it, hash it, commit it to your own repo, it will not move under you.

immutable is a one-way door: once served, browsers and CDNs may hold those bytes for a year and no purge reaches a browser's disk cache. So we do not say it until the 48-hour settling window has closed.

What is not immutable

Anything inside its settling window. A file dated today is provisional with a five-minute TTL. Values may still be corrected as upstreams land, FBIL around 13:30 IST, the NSE bhavcopy around 18:00, LBMA PM around 22:30, EIA on US hours.

Aggregates settle later than their constituents. /api/v1/all/2026-08-11.json mixes an Indian close that has happened with a US close that has not, so it freezes only after the last constituent calendar's close plus 48 hours. See Dates & trading days.

Rolling selectors, ever. latest.* and series/* are never immutable, no matter how old the underlying data.

Corrections

If a value changes after its window closed, we increment revision, append to /api/v1/revisions.json, and purge that URL.

A purge reaches our CDN. It does not reach a copy you already fetched. If you hold long-lived caches of dated files, poll revisions.json, that is what it is for.

ETags and `304`

Every response carries a weak ETag and honours If-None-Match.

curl -sI https://snapdata.dev/api/v1/all/latest.json | grep -i etag
# etag: W/"3f9a1c8e4b2d7f60"

curl -s -o /dev/null -w '%{http_code}\n' \
  -H 'If-None-Match: W/"3f9a1c8e4b2d7f60"' \
  https://snapdata.dev/api/v1/all/latest.json
# 304

An agent polling latest.json every minute costs us (and you) essentially nothing after the first fetch. The validator is weak because our renderers are deterministic for a given payload but a CDN may recompress, and a strong validator would be a claim about byte equality that we cannot make.

If-None-Match accepts a list and *, and a 304 still carries the validators.

`stale-if-error` is the resilience story

Rolling endpoints carry stale-if-error=604800.

If NSE blocks us, or PPAC moves a URL, or our Worker falls over, the edge keeps serving last-known-good for a week rather than throwing 5xx at your bot. Your integration degrades to slightly-old data instead of breaking.

This is only defensible because /api/v1/meta.json tells the truth while it is happening. It reports per-source staleness honestly and will say degraded or down while latest.json is still cheerfully returning cached numbers.

Serving stale data is fine. Reporting healthy while doing it is not. If freshness matters to your use case, read meta.json, or check the X-Snapdata-Status header on it.

CORS

Access-Control-Allow-Origin: * on everything, including error responses. It is open data; there is no origin we would want to refuse. Call it from a browser.

Format by extension, not `Accept`

/api/v1/gold/in/latest.json
/api/v1/gold/in/latest.csv
/api/v1/gold/in/latest.txt

Each gets its own CDN cache key, there is no Vary handling to get wrong, and it works in a browser address bar and in bare curl. Sending Accept: text/csv to the .json URL gets you JSON, deliberately. Content negotiation is a correctness trap at CDN scale.

Purging

Cache-tag purge is a Cloudflare Enterprise feature, so on publish we enumerate the touched URLs explicitly (roughly 200) and purge them in batches of 30. Nothing here is designed around a feature we do not have.

Caching & ETags

On this page