The same question gets different answers: how do you control randomness without losing useful variety?

Separate wording changes from factual errors

This applies when facts, format or wording vary enough to obscure comparisons between prompt changes. Distinguish acceptable paraphrases from actual errors: alternative product wording may be fine, while changed amounts or citations need investigation. Put those criteria in the evaluation rather than using word-for-word identity as the quality standard for every task.

Check sampling and whether the input is fixed

Check whether sampling is enabled and whether the requested temperature and Top-p actually take effect. Some services expose no seed, and certain controls apply only to sampling. Also check whether retrieved material or tool results change between runs: different inputs can produce different outputs despite identical generation settings.

Compare one parameter on a fixed evidence snapshot

Fix an input set and evidence snapshot, retain repeated outputs under the current configuration, then change one control. In Transformers, compare do_sample off and on and save the effective GenerationConfig. Do not change temperature and several cutoffs together; creative variation should still meet readability and factual constraints.

Evaluate factual consistency separately from stylistic variety

Report whether fixed facts stay consistent separately from whether permitted wording variations satisfy the task. Preserve every raw repeated result rather than selecting the best one. Identical answers that consistently omit required fields or repeat a paragraph still fail the actual objective.

Deterministic decoding does not establish factual correctness, and lower temperature cannot supply missing evidence. Hardware computation and backend differences can affect reproducibility. This page does not promise that zero temperature produces absolute identity; record environment differences when comparing systems instead of blaming every variation on the prompt.