Experiment notebook / Reading notes

Compare experiments fairly

Prevent environment and input changes from obscuring the question.

Keep a baseline

Choose a reference run with a clear reason. Compare the same input split and objective rather than selecting whichever historical result looks most favourable.

Change one dimension

Separate code, configuration and dataset changes when possible. If several must change together, state that the comparison cannot isolate their individual effects.

Report variation

Repeat stochastic work when the decision depends on small differences. Record random seeds while acknowledging that a seed alone may not guarantee identical execution.

← Browse all field notesDocumentation