Principle:Pyro ppl Pyro Rational Speech Acts
| Knowledge Sources | |
|---|---|
| Domains | Computational Pragmatics, Natural Language Processing, Game Theory |
| Last Updated | 2026-02-09 09:00 GMT |
Overview
The Rational Speech Acts (RSA) framework models language understanding as recursive probabilistic inference between a speaker who chooses utterances to be informative and a listener who interprets utterances by reasoning about the speaker's intentions.
Description
Human language is profoundly pragmatic: the meaning of an utterance depends not just on its literal semantics but on the context, the speaker's intentions, and shared expectations between speaker and listener. The Rational Speech Acts (RSA) framework provides a formal model of this pragmatic reasoning.
The framework defines a hierarchy of increasingly sophisticated agents:
Literal listener (L0): Interprets utterances literally. Given an utterance u, L0 updates beliefs about the world state w using only the literal meaning:
P_L0(w | u) proportional to u is true of w * P(w)
Pragmatic speaker (S1): Chooses utterances to be informative to L0. The speaker selects utterances that maximize the probability that L0 will arrive at the correct interpretation:
P_S1(u | w) proportional to exp(alpha * log P_L0(w | u) - cost(u))
where alpha controls the speaker's degree of rationality (higher = more optimal) and cost(u) penalizes complex or long utterances.
Pragmatic listener (L1): Reasons about S1 to infer the likely world state:
P_L1(w | u) proportional to P_S1(u | w) * P(w)
This recursion can continue (S2 reasons about L1, etc.) but typically converges after 1-2 levels.
RSA has been applied to explain and predict human behavior in many pragmatic phenomena:
- Scalar implicature: "Some students passed" implies "not all" via pragmatic reasoning.
- Hyperbole: "I waited a million years" is understood as exaggeration.
- Generics: "Birds fly" is interpreted despite exceptions.
- Reference games: Choosing descriptions that uniquely identify referents.
In Pyro, RSA models are naturally expressed as nested probabilistic programs with recursive inference.
Usage
Use the RSA framework when:
- Modeling how humans interpret natural language pragmatically (beyond literal meaning).
- Building computational models of scalar implicature, metaphor, or hyperbole.
- Designing reference games or communication protocols between agents.
- Teaching probabilistic programming through an intuitive linguistic application.
- Studying Schelling coordination and focal points in game-theoretic settings.
Theoretical Basis
RSA hierarchy:
# Literal listener L0:
# P_L0(w | u) = [[meaning(u, w)]] * P(w) / Z
# where [[meaning(u, w)]] = 1 if u is true of w, 0 otherwise
# Pragmatic speaker S1:
# P_S1(u | w) = exp(alpha * log P_L0(w | u) - cost(u)) / Z
# alpha: rationality parameter
# cost(u): utterance cost (e.g., word count)
# Pragmatic listener L1:
# P_L1(w | u) = P_S1(u | w) * P(w) / Z
# Higher levels (optional):
# S2(u | w) proportional to exp(alpha * log P_L1(w | u) - cost(u))
# L2(w | u) proportional to P_S2(u | w) * P(w)
Scalar implicature example:
# World states: w in {none, some, all} (how many students passed)
# Utterances: u in {"none", "some", "all"}
# Literal meanings:
# "none": true of {none}
# "some": true of {some, all} (literal "some" is compatible with "all")
# "all": true of {all}
# L0("some"): P(some) = 0.5, P(all) = 0.5 (no preference)
# S1(w=some): prefers "some" (informative about "some" but not "all")
# S1(w=all): prefers "all" (more specific)
# L1("some"): P(some) > P(all) (pragmatic inference: "some but not all")
# This derives the scalar implicature without hard-coding it!
Implementation as nested inference in Pyro:
# L0: condition on literal meaning
def literal_listener(utterance):
world = sample("world", prior)
condition(meaning(utterance, world))
return world
# S1: sample utterance, weight by L0's accuracy
def speaker(world):
utterance = sample("utterance", utterance_prior)
L0_posterior = infer(literal_listener, utterance)
factor(alpha * L0_posterior.log_prob(world))
return utterance
# L1: infer world from speaker's choice
def pragmatic_listener(utterance):
world = sample("world", prior)
S1_dist = infer(speaker, world)
factor(S1_dist.log_prob(utterance))
return world
Schelling coordination games:
# Two agents must coordinate on a choice without communication
# Agent 1 chooses x, Agent 2 chooses y
# Both succeed if x == y
# RSA-like reasoning:
# Each agent reasons about what the other would choose
# Focal points: choices that are "salient" or "obvious"
# P(choice) proportional to salience(choice) * P(other chooses same)
# This creates a fixed-point problem:
# P_i(x) proportional to salience(x) * P_j(x)
# Solution: P(x) proportional to salience(x) (the most salient option)