A model keeps repeating sentences: how do you distinguish decoding settings from input problems?
First determine whether repetition originates in the input, decoding settings or application code that appends previous output twice. Fix one failing input, inspect whether the raw response already repeats, and compare sampling and repetition controls one at a time.
Start troubleshooting here
First determine whether repetition originates in the input, decoding settings or application code that appends previous output twice. Fix one failing input, inspect whether the raw response already repeats, and compare sampling and repetition controls one at a time. Raising temperature may change the style without establishing that the underlying loop is fixed.
Rule out duplicated response chunks in the interface
Typical symptoms include repeated paragraphs, numbered lists restarting and repetition worsening as output grows. Preserve the response before front-end processing to rule out duplicate chunks after a streaming reconnect. If the raw response is normal but the interface repeats it, fix display or reassembly logic before changing model parameters.
Check history assembly and effective decoding settings
Check for duplicated examples in the prompt, a chat template applied twice, or a previous answer appended again as a new user instruction. After checking input, save both decoding settings and the model's generation_config. Explicit request values may override configuration files, so an interface's displayed defaults are insufficient evidence of the effective settings.
Adjust sampling and repetition controls individually
Keep a baseline with a shorter output budget, then change one decoding option at a time. Where supported in Transformers, compare greedy decoding with sampling or check the effective repetition_penalty. Preventing all repeated words can damage code and technical terminology; tune against the observed repeated passage instead of pushing several parameters to extremes.
Verify that legitimate repetition is still allowed
Retest the long input that repeated and add examples requiring valid repetition, such as table headings or code variables. Record whether repeated passages disappear, output ends as intended, and facts and formatting remain correct. Stopping the answer earlier while omitting the rest of the task is not a fix.
Sampling controls cannot guarantee repetition-free output for every model, and parameter names and ranges differ across services. This page offers a comparison process, not a universal parameter value. If behavior changes with the model version, retain both raw outputs to distinguish a model regression from message-assembly errors.
Checked sources
These sources support the concepts and methods described here. Examples and test scope are labeled separately; check the version of the tools you use.