Data
Building training datasets with provenance you can defend
Where the record came from, when, and under what terms. If you cannot answer that, the dataset is a liability.
AC
Andrew Cortés02 Jun 2026 · 9 min read
Training and evaluation datasets get assembled quickly and questioned slowly. Months later, when a customer or a regulator asks where a record came from, the answer is often a folder name and someone's memory.
Provenance is cheap to record at collection time and nearly impossible to reconstruct afterwards.
What we store with every record
The metadata that turns a scraped row into a defensible dataset.
Source URL and the exact timestamp of collection.
The terms in force at that moment, captured alongside the run.
The extraction version that produced the row, so results are reproducible.
A refresh policy: how stale a record can be before it is re-collected or dropped.
/ Data // Provenance // AI /