What is temperature? Separating sampling randomness from factual accuracy
Understand how temperature affects token probabilities through headline variation, including support limits and why low temperature does not guarantee truth.
Understand the concept
Temperature adjusts token-sampling distributions in some generation systems. Lower values generally emphasize high-probability candidates; higher values flatten the distribution and may increase variation. It does not update knowledge or check facts, and low values do not guarantee truth. Verify current model and interface support.
Changing probabilities does not supply factual evidence
Temperature can vary wording when generating headlines for one article. Correct dates still depend on evidence and processing; lowering temperature cannot automatically fix a wrong date. Check source numbers independently.
Hugging Face describes temperature’s effect on next-token probabilities within decoding configuration. Under a non-sampling strategy, sampling settings may not have the expected effect. Record the decoding setup rather than temperature alone.
Parameter availability differs across products
Current Google text-generation documentation says Gemini 4 and relevant 3.6-and-later models ignore custom temperature and other sampling values with deprecation warnings. Availability is not universal; verify support rather than copying old tuning steps.
Ranges and defaults can differ, so equal numbers across services need not mean equal behavior. Accepted fields may still be ignored; inspect documentation and warnings. No parameter comparison experiment was run here.
Use fixed inputs and repeated outputs for comparison
When tuning is supported, hold model and task fixed and inspect variation and errors. Writing can assess repetitive headlines; extraction can assess field stability. One interesting response does not establish an optimal setting for every task.
Low randomness does not guarantee exact repeatability because implementations can vary. Preserve inputs and versions and verify facts externally. Temperature concerns generation choices, not a complete acceptance test or a single measure of reliability.
Checked sources
These sources support the concepts and methods described here. Examples and test scope are labeled separately; check the version of the tools you use.