Journal of Undergraduate Research
Volume 6, Issue 2 - October 2004

Maximization and Delay Sensitivity in a Self-Control Task With Adult Humans

Kelsey Sullivan

ABSTRACT

This study attempts to distinguish overall reinforcement maximization from delay sensitivity in human subjects by mimicking the experimental procedure of self-control tests of animal subjects. Four adult female participants were given choices between a smaller reinforcer ($.025) after a standard delay and a larger reinforcer ($.05) after an alternate delay, which varied across blocks of trials (components) in a 3-component multiple schedule of reinforcement. Choices were arranged such that maximization of reinforcer density was achieved by choosing the small reinforcer when the alternate delay was long, and the large reinforcer when the alternate delay was short. In a third component, the local reinforcer delays differed but the overall reinforcer density was the same no matter which choice was made. The mean overall reinforcer delays varied systematically across conditions. The results were mixed, but sensitivity to delay (preference for the smaller delay when maximization of reinforcer was held constant) was evident in two of the four participants. By using a procedure similar to that of other animal procedures on self-control tasks, this study showed delay sensitivity in human subjects.

INTRODUCTION

Logue (1988) defines self-control as “a choice of a larger, more delayed reinforcer over a smaller, less delayed reinforcer…” and impulsivity as “a choice of a smaller, less delayed reinforcer over a larger, more delayed reinforcer…” (p. 665). Within behavior analysis, research on self-control has been conducted with humans as well as other animals. Differences in outcomes have been observed and may be due to the fact that human research has typically involved choices with hypothetical outcomes (e.g., $50 now versus $500 one year from now), whereas nonhuman research involves choices with real outcomes (e.g., food). Typically, animals are "impulsive," preferring the smaller more immediate reinforcer, whereas humans typically show self-control (Logue, 1988).

The problem lies in trying to compare the data obtained from these two types of experiments: hypothetical reinforcers with humans vs. actual reinforcers with animals. The experiment reported here was conducted with humans, but many of the procedural features were more characteristic of animal research. First, real outcomes were used: points exchangeable for money. Second, participants were exposed to the task many times such that familiarity with the task was obtained. Finally, participants remained in a component until choice patterns were stable.

In most studies, self-control has been defined as choosing the larger delayed reinforcer, which is equivalent to maximizing overall reinforcement density (or reinforcer amount per unit time). The present experiment separated self-control, as defined by choices of the larger reinforcer each trial, from overall reinforcement maximization, defined by choices that produce the highest density of reinforcement across blocks of trials. The procedure was patterned after a similar study by Sonuga-Barke, Lea and Webley (1989) conducted with children (ages ranged from 4 to 12 years, all female). In that study, choices produced tokens, either 1 following a shorter delay or 2 following a longer delay. The results showed that the oldest children (the 12-year-old group) switched their preference to the smaller reinforcer when optimal, while the 6 and 9-year-old groups always picked the delays associated with the largest reinforcers, and only half of the 4-year-old group switched their preferences to the smaller reinforcer when optimal.

The present study used a 3-component multiple schedule. Each component entailed choices between a standard delay to a small reinforcer and an alternate delay to a larger reinforcer (twice the size of the small reinforcer density). The alternate delay value varied across components such that maximization of reinforcer was sometimes achieved by choosing the standard (small) reinforcer, and sometimes by choosing the large (alternate) reinforcer. In a third component, the delay and reinforcer amount associated with the alternate delay were exactly twice that of the standard; therefore, overall reinforcer density was the same no matter what choice pattern occurred. In this component, what is of most interest is delay sensitivity, that is, whether the participant prefers the shorter delay to x amount or the longer delay to 2x amount. By setting up the contingencies in this way, we can potentially separate overall reinforcement maximization (defined by patterns of choices across blocks of trials) from delay sensitivity (defined by delay to reinforcers on individual trials).

METHODS

Participants

A total of six college students were recruited. Four female participants (age range 20 to 22 years old) remained for the entire experiment. Each participant was paid a base rate of $3.00 per visit (two 30-min sessions). Any additional money earned during the sessions was paid as a bonus contingent on participation for the full duration of the experiment. The average pay over the course of the experiment was $6.39 per hour, with a range of $6.25 to $6.53.

Apparatus

The program (conducted on a computer) was arranged to have a similar appearance throughout the entire experiment, with three large square boxes in the center and two smaller rectangle boxes centered, one over the top of, and one underneath, the three larger square boxes. Of the large square boxes, the outer two were choice keys and the inner functioned as a trial initiation and/or exchange response key. The smaller boxes at the top and bottom displayed running totals of the amount of money earned.

Procedure

The choices were set up such that there was a standard (STD) delay pitted against an alternating (ALT) delay. Within each phase the STD delay stayed the same and was always associated with the yellow key and the smaller reinforcer at $.025. The ALT delay assumed three different delay values equal to, twice, or three times the STD delay (hereafter referred to as small, medium, and large delay, respectively), and the monetary reinforcer was always $.05. The associated colors were as follows: green, small; blue, medium; and purple, large. Thus, when the STD and ALT delays were equal, the participant could maximize reinforcer density by choosing the ALT delay; the opposite being true when the ALT delay was triple the STD delay. When the ALT delay was double the STD delay, the overall reinforcer density was equivalent no matter which one was chosen (2 x .025 = 1 x .05).

Phases were changed when choice patterns were stable across three sessions, determined by visual analysis. Table 1 shows the order of phases and the number of sessions conducted for each participant per phase.

Table 1 Order of Phases and Number of Sessions
Participant	Phase 1: STD 30 s	Phase 2: STD 60 s	Phase 3: STD 15 s	Replication 1: STD 30 s	Replication 2: STD 60 s
P1	18	10	10	8	N/A
P2	14	10	8	N/A	N/A
P3	8	6	6	6	6
P4	6	10	6	N/A	N/A

RESULTS

Figure 1 shows the average choice proportions of the last three sessions of each phase. The data for participant P1 show a clear preference for the small ALT delay when pitted against the STD delay. When the medium ALT delay was pitted against the STD delay, there was a preference for the ALT delay at STD 15 s and STD 30 s, but preference switched for the STD delay at STD 60 s. During the large ALT delay versus the STD delay, preference for the STD delay was very clear at STD 15 s and STD 60 s, but close to indifference at STD 30 s. The replicated phase showed a shift in preference toward the STD delay.

Figure 1. Average choice proportions and ranges of the last three sessions across all three experimental phases and replication phases (where indicated).

Participant P3’s data resembles P1’s data in preference for the small ALT delay when pitted against the STD delay. Preference for the medium ALT delay was also similar to P1 in STD 30 s, but differed in STD 15 s, where preference here was for the STD delay, and in STD 60 s, where preference was for the medium ALT delay. As compared to P1, there was an even stronger preference for the STD delay when pitted against the large ALT delay. Replication of the first two phases shows that there was strong preference for the small ALT delay rather than the STD delay in both the STD 30 s and STD 60 s. Consistent with P1’s data, replication of the STD 30 s and the additional replication here of the STD 60 s phase show that both the medium and large STD delay were preferred over the ALT delay.

Participant P2’s data was close to indifference at STD 15 s and STD 30 s, and just above indifference, preferring the small ALT delay at STD 60 s. During the medium delay preference was close to indifference in STD 15 s and STD 30 s, with preference slightly favoring the medium STD delay at STD 60 s. For the large ALT delay versus the STD delay, preference for the large ALT delay was shown at STD 60 s, and indifference at STD 30 s and STD 15 s. Additional control conditions conducted for this subject revealed a lack of sensitivity to the contingencies.

The data for participant P4 show strong preference for the ALT delay in STD 15 s, STD 30 s, and STD 60 s. Additional conditions revealed that this participant's choices were controlled by color rather than by reinforcement variables.

CONCLUSIONS

Choice patterns in two of the four subjects showed sensitivity to the contingencies. P1 and P3 generally selected the ALT delay in the small-delay component (9 of 9 cases), and the STD delay in the large-delay component (8 of 9 cases). Such patterns are predicted by reinforcement density maximization and are broadly consistent with the choices of the oldest (12-yr-old) children in the Sonuga-Barke et al. (1989) study. Also similar to that study, the present experiment included a condition in which overall reinforcement density did not favor either choice. With no gain in overall point/money earnings, would subjects prefer quicker access to a smaller reinforcer or later access to a larger reinforcer? The answer was mixed.

P3 slightly preferred the STD (small) option during the STD 15 s phase, suggesting sensitivity to delay. However, during the first two conditions, STD 30 s and STD 60 s preference was for the ALT delay, which can in turn be interpreted as insensitivity to delay or preference for the larger reinforcer. Lastly, during replication of STD 30 s and STD 60 s preference switched and was strongly for the STD delay, suggesting sensitivity to delay.

P1 strongly preferred the ALT delay during the STD 15 s phase, and like P3, during the STD 30 s phase, which, again, can be interpreted as insensitivity to delay or preference for the larger reinforcer. In the STD 60 s phase, unlike P3, P1 preferred the STD delay, suggesting sensitivity to delay. Only the STD 30 s phase was replicated for this participant, which resulted in a shift in preference for the STD delay. In sum, of the four medium-delay conditions for this subject, the ALT delay and STD delay were each preferred twice.

The other two subjects showed little sensitivity to the contingencies. P2 was roughly indifferent throughout the entire experiment. Subsequent conditions revealed that this participant was clearly insensitive to both delay and amount. P4 was extremely insensitive to delay, with nearly all of her choices favoring the ALT (larger reinforcer) delay. It appeared that she had a strong bias for the larger reinforcer, but when we switched the color associated with the large and small reward, her choice patterns switched accordingly, revealing a color bias.

In summary, by making this experiment procedurally similar to that of other animal procedures on self-control tasks, we obtained results in some conditions with humans more similar to those obtained with other animals (Jackson & Hackenberg, 1996): delay sensitivity. This study should be taken as just one step in the process of learning more about self-control and delay sensitivity in humans and other animals. It should encourage studies with other animals that are more ”human-like” as well as more experiments with humans that are more “animal-like.”

---

REFERENCES

1. Jackson, K., & Hackenberg, T. (1996). Token reinforcement, choice, and self-control in pigeons. Journal of the Experimental Analysis of Behavior, 66, 29-49.
2. Logue, A. W. (1988). Research on self-control: An integrating framework. Behavioral and Brain Sciences, 11, 665-709.
   
3. Sonuga-Barke, E., Lea, S., & Webley P. (1989). The development of adaptive choice in a self-control paradigm. Journal of the Experimental Analysis of Behavior, 51, 77-85.
   

---

--top--

Back to the Journal of Undergraduate Research