Data
Building training datasets with provenance you can defend
Where the record came from, when, and under what terms. If you cannot answer that, the dataset is a liability.
AC
Andrew Cortés02 Jun 2026 · 9 min read

Training and evaluation datasets get assembled quickly and questioned slowly. Months later, when a customer or a regulator asks where a record came from, the answer is often a folder name and someone's memory.
Provenance is cheap to record at collection time and nearly impossible to reconstruct afterwards.
What we store with every record
The metadata that turns a scraped row into a defensible dataset.
- Source URL and the exact timestamp of collection.
- The terms in force at that moment, captured alongside the run.
- The extraction version that produced the row, so results are reproducible.
- A refresh policy: how stale a record can be before it is re-collected or dropped.
/ Data // Provenance // AI /