One change per run
Change one knob, rerun, compare the held out number. Changing three at once produces a better number you cannot explain, which is a result you cannot repeat.
Seed the run
Two runs of the same setup can differ. Fixing the seed and recording it is what makes a difference between runs real rather than noise.