Heuristic:Togethercomputer Together python Repetition Penalty Conflict
| Knowledge Sources | |
|---|---|
| Domains | Inference, Text_Generation |
| Last Updated | 2026-02-15 16:00 GMT |
Overview
Warning that `repetition_penalty` should not be used alongside `presence_penalty` or `frequency_penalty` as they produce conflicting text generation behavior.
Description
The Together SDK emits a runtime warning when users combine `repetition_penalty` with either `presence_penalty` or `frequency_penalty` in completion requests. These parameters control similar aspects of text generation (reducing repetition) through different mechanisms, and using them together can produce unpredictable or degraded output quality.
Usage
Apply this heuristic when configuring text completion or chat completion requests. If you need to reduce repetition, choose one approach:
- `repetition_penalty` (multiplicative penalty on repeated tokens)
- `presence_penalty` and/or `frequency_penalty` (additive penalties)
The Insight (Rule of Thumb)
- Action: Use either `repetition_penalty` OR `presence_penalty`/`frequency_penalty`, never both simultaneously.
- Value: No specific numeric threshold; the conflict is conceptual.
- Trade-off: `repetition_penalty` applies a multiplicative factor to all previously seen tokens. `presence_penalty` adds a constant penalty for any token that appeared before. `frequency_penalty` scales the penalty by how often a token appeared. Combining multiplicative and additive penalties leads to over-suppression of common tokens.
Reasoning
The three penalty parameters all target the same problem (reducing repetitive output) through different mathematical approaches. When combined, they can compound in unexpected ways: a token that appeared multiple times gets penalized by `repetition_penalty` (multiplicative), `presence_penalty` (fixed additive), AND `frequency_penalty` (count-scaled additive), leading to severe over-penalization and degraded output quality.
Code evidence from `src/together/types/completions.py:51-59`:
# Raise warning if repetition_penalty is used with presence_penalty or frequency_penalty
@model_validator(mode="after")
def verify_parameters(self) -> Self:
if self.repetition_penalty:
if self.presence_penalty or self.frequency_penalty:
warnings.warn(
"repetition_penalty is not advisable to be used alongside "
"presence_penalty or frequency_penalty"
)
return self