Research philosophy
These principles decide how ARI3 is built, tested and reported. Each experiment in the notebook can be checked against them.
- 1
Version everything
Every model, dataset and experiment has a version and a hash. A frozen version is never changed.
- 2
Measure everything
A claim about ARI3 comes with the number behind it and the size of the sample it came from.
- 3
Never hide failures
Every pre-registered hypothesis is reported, including those that fail, and every gain is shown beside its cost.
- 4
Pre-register experiments
Questions and pass criteria are committed publicly before the data that answers them exists.
- 5
Separate hypothesis from hindsight
Analysis choices made after seeing data are listed as such, apart from the pre-registered plan.
- 6
Prefer reproducibility over novelty
Every result lists the code, data fingerprint and hashes behind it, and says plainly what an outsider cannot yet rerun.
- 7
Human judgment is data
The editor's labels are the ground truth ARI3 learns from and is tested against, and they are versioned like code.
- 8
Models should admit uncertainty
An ARI3 model says when it is not sure, and those items are meant for a person instead of an automatic count.