Probability and delay of reinforcement as factors in discrete-trial choice.
Delay outweighs probability in choice, but only if the schedule keeps the learner responding.
01Research in Context
What this study did
Pigeons pecked keys in a two-key discrete-trial task.
One key gave food after a short delay but only on some trials.
The other key gave food after a longer delay but on more trials.
The experimenter changed the delay seconds and the odds of payoff until the bird chose each key about half the time.
What they found
Birds cared more about delay than about odds.
Doubling or quintupling the chance of food was worth several extra seconds of wait time.
Indifference points lined up in a tidy curve: delay ruled, probability followed.
How this fits with other research
Meyer et al. (1987) later asked college students to commit early or wait.
Humans showed the same pattern: low chance of future choice pushed them to lock in early, just like pigeons.
The 1985 bird data became the backbone for testing people.
Lancioni et al. (2011) added time between trials.
When birds could quit after a miss, longer gaps erased their sensitivity to probability.
The 1985 curve still held, but only if the session kept them playing.
Together the three papers show: delay is king, yet trial spacing can dethrone it.
Why it matters
Your client may "prefer" 80 % praise now over 100 % praise in ten seconds.
If you stretch the wait, even a sure thing can lose.
Keep delays short or bridge them with signals.
When you must use lean schedules, tighten trial spacing or add forced trials so the learner stays in the game.
Want CEUs on This Topic?
The ABA Clubhouse has 60+ free CEUs — live every Wednesday. Ethics, supervision & clinical topics.
Join Free →Cut response-to-reinforcer delay to three seconds or less before thinning the ratio.
02At a glance
03Original abstract
Pigeons chose between two alternatives that differed in the probability of reinforcement and the delay to reinforcement. A peck on the red key always produced a delay of 5 s and then a possible reinforcer. The probability of reinforcement for responding on this key varied from .05 to 1.0 in different conditions. A response on the green key produced a delay of adjustable duration and then a possible reinforcer, with the probability of reinforcement ranging from .25 to 1.0 in different conditions. The green-key delay was increased or decreased many times per session, depending on a subject's previous choices. The purpose of these adjustments was to estimate an indifference point, or a delay that resulted in a subject's choosing each alternative about equally often. In conditions where the probability of reinforcement was five times higher on the green key, the green-key delay averaged about 12 s at the indifference point. In conditions where the probability of reinforcement was twice as high on the green key, the green-key delay at the indifference point was about 8 s with high probabilities and about 6 s with low probabilities. An analysis based on these results and those from studies on delay of reinforcement suggests that pigeons' choices are relatively insensitive to variations in the probability of reinforcement between .2 and 1.0, but quite sensitive to variations in probability between .2 and 0.
Journal of the experimental analysis of behavior, 1985 · doi:10.1901/jeab.1985.43-341