REPRODUCIBLE COMPUTE
Keep the question.
Keep the evidence.
An experiment does not end at a number. The useful part is the trail: what changed, what stayed fixed, and why the comparison matters.
Open the experiment notebook ↗Convergence, observed.
A controlled comparison of two sample estimators.
HYPOTHESIS & LIMITS
Hold the inputs fixed. Change one parameter.
This prepared example compares a reference curve with an adjusted curve. The range control uses the illustrative relationship 1 / √n. The adjusted curve uses 1 / √n; the reference uses 1.3 / √n. Both use the same axes. These illustrative relationships are not measured performance.
- Identify the input sample and its revision.
- Record the estimator and sample-size assumption.
- Preserve uncertainty alongside the interpretation.
RECONSTRUCTION RECORD
- Study
- convergence-reference
- Input
- prepared illustrative series
- Runtime
- browser / local example
- Parameter
- sample_size: 64
- Interpretation
- method illustration, not a benchmark
Prepared example · your adjustments stay in this page.
Make the next run
comparable.
A practical workspace model for computational experiments: input manifests, pinned environments, resource assumptions and results you can explain. Make the next run comparable to the last.
An explicit run manifest
Record code revision, parameters, input references and environment identity. A notebook title is not enough to reconstruct an experiment.
Comparable environments
Track dependency versions and relevant hardware characteristics. Separate a code change from a runtime change when comparing outcomes.
Results with provenance
Link each output to the run that produced it. Keep interpretation alongside raw results so another person can review the conclusion.
Resource-aware execution
State memory, time and storage expectations before launching work. Treat interruption and partial output as normal engineering cases.
From the methods shelf
01A useful experiment manifest
The minimum record needed to understand a computational run.
Compare experiments fairly
Prevent environment and input changes from obscuring the question.
Design for interrupted runs
Checkpoints and partial results that are safe to continue.