Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • View all journals
  • Explore content
  • About the journal
  • Publish with us
  • Sign up for alerts
  • Original Article
  • Published: 18 January 2012

Pramipexole-Induced Increased Probabilistic Discounting: Comparison Between a Rodent Model of Parkinson's Disease and Controls

  • Sandra L Rokosik 1 , 2 &
  • T Celeste Napier 2  

Neuropsychopharmacology volume  37 ,  pages 1397–1408 ( 2012 ) Cite this article

3416 Accesses

58 Citations

3 Altmetric

Metrics details

  • Parkinson's disease
  • Pharmacodynamics

The dopamine agonist pramipexole (PPX) can increase impulsiveness, and PPX therapy for neurological diseases (Parkinson's disease (PD) and restless leg syndrome) is associated with impulse control disorders (ICDs) in subpopulations of treated patients. A commonly reported ICD is pathological gambling of which risk taking is a prominent feature. Probability discounting is a measurable aspect of risk taking. We recently developed a probability discounting paradigm wherein intracranial self-stimulation (ICSS) serves as the positive reinforcer. Here we used this paradigm to determine the effects of PPX on discounting. We included assessments of a rodent model of PD, wherein 6-OHDA was injected into the dorsolateral striatum of both hemispheres, which produced persistent PD-like deficits in posture adjustment. Rats were trained to perform ICSS-mediated probability discounting, in which PD-like and control groups exhibited similar profiles. Rats were treated twice daily for 2 weeks with 2 mg/kg (±)PPX (ie, 1 mg/kg of the active form), a dose that improved lesion-induced motor deficits. In both groups, (±)PPX increased discounting; preference for the large reinforcer was enhanced 30–45% at the most uncertain probabilities. Tolerance did not develop with repeated treatments. Increased discounting subsided within 2 weeks of (±)PPX cessation, and re-exposure to (±)PPX reinstated heightened discounting. Such findings emulate the clinical scenario; therefore, ICSS for discounting assessments in rats exhibited high face validity. This model should prove useful in medication development where assessment of the propensity of a putative therapy to induce risk-taking behaviors is of interest.

Similar content being viewed by others

probability discounting task

Microdosing with psilocybin mushrooms: a double-blind placebo-controlled study

Federico Cavanna, Stephanie Muller, … Enzo Tagliazucchi

probability discounting task

Dopamine transients follow a striatal gradient of reward time horizons

Ali Mohebi, Wei Wei, … Joshua D. Berke

probability discounting task

Adults who microdose psychedelics report health related motivations and lower levels of anxiety and depression compared to non-microdosers

Joseph M. Rootman, Pamela Kryskow, … Zach Walsh

INTRODUCTION

Dopamine (DA) agonists pramipexole (PPX) and ropinirole are FDA approved for treatment of motor dysfunction in Parkinson's disease (PD) and restless leg syndrome (RLS). DA agonist therapy is associated with impulse control disorders (ICDs) in an estimated 14% of treated PD patients ( Voon and Fox, 2007 ; Weintraub et al, 2010 ) and 7–12% of treated patients with RLS ( Pourcher et al, 2010 ; Driver-Dunckley et al, 2007 ). These drugs are being used off label for other pathologies, including fibromyalgia and bipolar disorders wherein ICDs are also observed ( Holman, 2009 ; Strejilevich et al, 2011 ). Independent of the pathology for which the therapy is implemented, ICD onset is reported to relate to onset of DA agonist treatment, and symptoms typically subside with dose reduction or discontinuation ( Dodd et al, 2005 ; Driver-Dunckley et al, 2007 ; Mamikonyan et al, 2008 ; Quickfall and Suchowersky, 2007 ). In North America, ICDs associated with DA agonists commonly include problem/pathological gambling, compulsive sexual behavior, compulsive buying, and binge eating ( Weintraub et al, 2010 ). These behavioral disorders are reward or incentive based and repetitive in nature ( Evans et al, 2009 ), indicating that DA agonists can lead to dysregulation of general reward processes. Supporting this concept, acute PPX can enhance reward-mediated learning ( Pizzagalli et al, 2008 ; Santesso et al, 2009 ) and impulsivity in healthy human volunteers ( Riba et al, 2008 ; but see Hamidovic et al, 2008 ).

To better understand the link between DA agonists and ICDs, and to provide a means to screen new therapies without a propensity to induce aspects of impulsivity, a valid animal model is needed. Towards that end, we developed a novel probability discounting paradigm in laboratory rats ( Rokosik and Napier, 2011 ). This task measures how changes in probabilities alter decision making. For example, subjects are given a choice between a small reward that is always delivered and a large reward that is sometimes delivered. If the probability of obtaining a large reinforcer is high, the subject will prefer the large reinforcer; however, lower probabilities will drive preference for the small reward. If discounting increases, this reflects a reduced importance of the low probabilities, and the subject will exhibit preference for the large reward during both high and low probabilities for reward obtainment. Thus, probability discounting is a popular method to study risky decision making, one facet of impulsivity. Problem gamblers demonstrate increased risk taking in probability discounting paradigms ( Holt et al, 2003 ; Madden et al, 2009 ; Petry, 2011 ). To provide a potent, rapid, and reliable reward that allows for repeated tests of discounting, we employed intracranial self-stimulation (ICSS) as the positive reinforcer in rats ( Rokosik and Napier, 2011 ). The ability for repeated testing is a critical feature for assessments of chronic treatments. As yet, laboratory evaluations have not yet been conducted for chronic PPX administration, and this is needed to better emulate the therapy scenario used clinically. To fill this gap, the current study evaluated the effects of chronic (±)PPX treatment on probability discounting. To emulate the pathology for which PPX is most often used clinically, we included assessments in a 6-hydroxydopamine-hydrobromide (6-OHDA) model of PD. As DA agonists, including PPX, are front-line therapy for early-stage PD ( Bonuccelli et al, 2009 ), we sought to model the human brain at this stage, that is, when dopaminergic lesions are largely confined to the putamen ( Kish et al, 1988 ). The rodent dorsolateral striatum (DLS) is the homolog of the primate putamen, and lesions of DA inputs to the DLS via 6-OHDA injections are a common way to model early stages of PD in rats ( Deumens et al, 2002 ; Przedborski et al, 1995 ). Thus, we used this approach to provide a PD model in which to study the effects of ±PPX.

MATERIALS AND METHODS

Male Sprague-Dawley rats weighing 250–274 g upon arrival (Harlan, Indianapolis, IN) were housed in pairs under environmentally controlled conditions (0700 h/1900 h light/dark cycle, temperature maintained at 23–25 °C) with access to rat chow and water ad libitum . Rats were handled according to federal standards. Protocols were approved by Rush University IACUC.

Treatment Drugs

Pramipexole (synthesized as the racemic mixture; Daya Drug Discoveries, Hazelwood, MO) (±PPX) was dissolved in saline and given intraperitoneally (IP) as 0.25, 0.5, 1.0, 2, or 4 mg/ml/kg for assessments in stepping and 2 mg/kg for the discounting task. To induce dopaminergic lesions, 6-OHDA (Sigma-Aldrich, St Louis, MO) was dissolved in 0.2% ascorbic acid in a sterile saline solution (pH=5.0) and infused into the striatum at a dose of 7.5 μg per 2 μl per side (as the salt). Rats were given a 30-min pretreatment of 25 mg/kg (as the salt) of desipramine-HCl (DMI; Sigma-Aldrich) dissolved in sterile water to reduced uptake of the 6-OHDA into adrenergic neurons.

Surgical Procedures for 6-OHDA Injections and Electrode Implantation

To stereotaxically lesion the striatum and implant the stimulation electrode, rats were anesthetized with sodium pentobarbital (50 mg/kg/ml IP; Sigma-Aldrich), administered DMI, and the head placed in a stereotaxic frame (David Koft, Tujunga, CA) with the nose piece set at 3.3 mm below the horizontal. A 33-gauge, bilateral injector was lowered to the DLS (1.0 mm anterior to bregma, 3.4 mm lateral from midline, 4.7 mm ventral from skull). At 30 min after DMI, 6-OHDA was injected at a rate of 0.2 μl/min for 10 min. Sham controls were similarly injected with the ascorbic acid vehicle. The injectors were left in place for an additional min (to allow the solution to diffuse away from the tip) and the skull holes were filled with bone wax. A bipolar stimulating electrode (MS303/3-B/SPC; Plastics One, Roanoak, VA) was lowered to the lateral hypothalamus (LH; 2.6 mm posterior to bregma; 1.8 mm lateral; 8.4 mm ventral). Electrodes were secured to the skull with stainless steel screws and dental acrylic, and the incision was sutured. Rats were allowed at least 5 days of recovery from surgery before operant testing was initiated.

Behavioral Testing

Motor assessment: forelimb adjusting step test.

The 6-OHDA-induced motor deficits were verified using the forelimb adjusting step test, ( Olsson et al, 1995 ) conducted 1 day before surgery and at least once a week after surgery. To do so, the experimenter suspended the rat's rear legs and one forelimb while the rat supported itself on its unrestrained forelimb. The rat was ‘dragged’ on the unrestrained forelimb 0.9 m for 5 s in abduction and adduction directions for both forelimbs, and the number of adjusting steps was counted. Three stepping trials were taken per session, and the average score was determined.

An initial study was conducted to validate the rat model of PD employed here with regard to (1) brain DA deficits, and (2) motor dysfunction for a time frame that would coincide with duration of the probability discounting paradigm. The 6-OHDA-treated rats were killed 21 days ( n =6) or 60 days after lesion ( n =6); sham rats ( n =5) were killed 60 days after lesion. Forelimb stepping adjustments were measured every 3 days. Lesion extent was verified in ex vivo tissue harvested 21 or 60 days after 6-OHDA infusion using tyrosine hydroxylase immunohistochemistry (TH-IHC).

A separate group of lesioned rats (also implanted with stimulation electrodes) were used to conduct a (±)PPX dose vs stepping response evaluation. These rats were tested with the stepping task 1 day before surgery and every week after. At ∼ 40 days after the lesion, the following protocol was used: (±)PPX was administered to sham ( n =7) and 6-OHDA-treated rats ( n =5) in the morning and stepping adjustments were measured immediately before, and 1 and 6 h after treatment. In the evening, a second (±)PPX injection (of the same dose) was given and stepping was measured 17 h later. Treatments (vehicle, 0.25, 0.5, 1, 2, and 4 mg/kg, IP) were administered weekly in a pseudorandomized order.

ICSS Procedures and Apparatus

ICSS experiments were conducted in operant chambers (30.5 cm × 24.1 cm × 21.0 cm; Med-Associates, St Albans, VT) outfitted with a chamber light, and two retractable levers each under a stimulus light and enclosed in ventilated, sound-attenuated boxes. Electrical brain stimulation (EBS) was delivered by a programmable stimulator (PHM-152/2) via bipolar leads connected to commutators (Plastics One, Roanoak, VA) mounted above the chamber. Typically, two ICSS test sessions were conduced per day. The following describes the testing protocols for various phases in the probability discounting paradigm.

ICSS-Mediated Probability Discounting

A nine-phase paradigm was used to determine rats' baseline discounting and effects of (±)PPX, as previously described ( Rokosik and Napier, 2011 ). Table 1 shows the acquisition criteria for phases 1–6 that were required before initiating (±)PPX treatment (phases 7–9) in the current study. Briefly, the phases are described as follows. Phase 1, shaping . A single lever was extended and EBS (200 μs biphasic square wave pulses, applied at 100 Hz for 500 ms) was delivered. Only the initial current intensity (100 μA) was adjusted for each rat based on their performance to approach and ultimately press the lever. The final intensity level was used for the remaining phases. Phase 2, fixed ratio-1 (FR-1) reinforcement . To establish stable ICSS lever pressing, rats underwent a continuous FR-1 reinforcement schedule wherein one lever was extended for a 30-min session. Phase 3, rate-frequency function . Rats were pseudorandomly presented with 1 of 16 different current frequencies tested in 10 Hz increments, ranging from 10 to 160 Hz. Train duration and current intensity were held constant. For each frequency, rats had access to the lever for 2 min and the numbers of lever presses were recorded. Following each 2 min period, the lever retracted for 10 s. In each session, a lever pressing rate vs ICSS current frequency (termed the rate-frequency function ) was collected and the maximal ( E max ) and minimal (threshold) number of lever presses were determined using a nonlinear regression (GraphPad Prism, La Jolla, CA). When a rat met phase acquisition criteria (see Table 1 ), averages of three curves were used to determine ICSS frequencies that produced 90, 60, and 40% of E max (termed effective current (ECur); ECur 90 , ECur 60 , and ECur 40 , respectively; see Figure 1 ). Phase 4, discrete trials . Rats were trained to recognize the temporal nature of trials using each rat's own ECur 60 as the reinforcer. Each session comprised 200 trials. Trials occurred in 15 s intervals. Each session began with both levers retracted and the chamber light off; 2 s later, the chamber light was illuminated, followed 3 s later by the extension of one lever. The rat had 10 s to press the lever, if the response was not executed, the trial was aborted (termed an omitted trial ), the lever retracted, and the chamber light turned off. If a lever press was made, an EBS was delivered and the stimulus light over the lever was turned on. After 0.5 s, all lights were turned off and the lever retracted. The two levers were alternately extended among trials. Phase 5, choice test . The purpose of this phase was to determine for each rat, a small and large reinforcer that could be used in the probability discounting phase. Using the FR-1 discrete trials described in phase 4, rats were trained to select from different, lever-specific, reinforcement values. Each session consisted of three blocks. Each block consisted of 20 forced-choice trials followed by 20 free-choice trials. In forced-choice trials, one lever was extended at a time allowing the rat to learn the reinforcement value associated with that lever. In free-choice trials, both levers were extended, and the rat had to choose between the lever-specific reinforcement values. Initially, small and large reinforcers corresponded to the rat's ECur 90 and ECur 40 (obtained in phase 3). To complete this phase, rats had to demonstrate a ‘ free-choice ratio ’ (the number of selections for the large reinforcer divided by the total number of lever responses made × 100) of at least an average of 70% across the three blocks. Phase 6, probability discounting task . Each session consisted of nine blocks as used in phase 5, but here, one lever was designated ‘small/certain lever’ (SC) and the other was ‘large/risky’ lever (LR). A press on the SC lever always delivered the small reinforcer (ie, approximately ECur 40 ); a press on the LR lever delivered the large reinforcer (approximately ECur 90 ) with varying probabilities. The following three series of probability presentations were cycled during this, and subsequent phases: (1) 0.5, 0.3, 0.85, 0.6, 0.05, 0.7, 1.0, 0.4, and 0.15; (2) 0.15, 0.6, 0.4, 0.05, 0.7, 0.3, 0.85, 1.0, and 0.5; and (3) 0.7, 0.4, 1.0, 0.15, 0.5, 0.85, 0.05, 0.3, and 0.6. For each series, the LR lever was designated either to the left or right lever; therefore, each rat experienced six different probability formats. Data from free-choice trials of each probability (ie, block) were analyzed to determine a baseline free-choice ratio vs probability function. If in a block, there were ⩾ 50% omissions from the free-choice trials (ie, >10 of 20 trials tested), data from that block were excluded from subsequent analysis. This criterion was held for phases 6–9 (each of which employed the probability discounting task), and overall, <2% of the blocks were excluded. Phase 7, (±)PPX treatment . At 1 day following the last baseline test, (±)PPX treatment was initiated. The regimen was 2 mg/kg (±)PPX, IP, twice a day (in the morning and evening) for 13 days (termed, chronic treatment ). The ±PPX-induced changes in discounting were assessed 30 min and 6 h following the morning injection on the first and every third day of the chronic treatment. Phase 8, withdrawal . In a subset of rats, (±)PPX was withdrawn for 15 to 69 days after cessation of treatment. Phase 9, re-instatement . The (±)PPX treatment was reinitiated twice a day for 7 days. Probability discounting was assessed every third morning throughout phases 8 and 9.

figure 1

ICSS rate-frequency function. In phase 3, the relationship between ICSS lever pressing rate and stimulation frequency (an index of signal strength) was obtained for each rat. Illustrated is the final curve (ie, met stability criteria as described in Table 1 ) for an individual PD-like rat. From this curve, the ECur 90 (solid line), ECur 60 (dotted line), and ECur 40 (dashed line) were determined using a nonlinear regression (GraphPad Prism).

PowerPoint slide

Histology and TH-IHC

Rats were deeply anesthetized with chloral hydrate (400 mg/kg; Sigma, St Louis, MO). A 5 V DC current was applied to the stimulating electrode for 30 s to deposit iron and/or produce a very discrete lesion at the electrode tip. The iron deposits were visualized by a blue coloration produced via trychloroacetic acid (0.5%) and potassium ferricyanide (3%) added to a 4% paraformaldehyde solution used for transcardial perfusion after perfusing with ice-cold 0.9% NaCl. Brains were removed, postfixed in 4% paraformaldehyde, and stored in a 30% sucrose solution. Brains were sliced in 40 μm coronal sections. Striatal sections were immunoreacted with a primary, monoclonal mouse anti-TH antibody (ImmunoStar, 22941) diluted 1 : 10 000 and a biotinylated horse anti-mouse IgG (Vector Laboratories, BA2001) diluted 1 : 100. The signal was amplified by avidin and biotinylated horseradish peroxidase using the Elite ABC Vectastain Kit (Vector Labs, PK6100). Immunostaining was visualized with 3,3-Diaminobenzidine tetrachloride dehydrate (Sigma, D5637) solution activated with 0.3% H 2 O 2 .

Data Analysis

To compare PD-like and control rats, data from phases 1 to 6 were analyzed using Student's t -test. A linear correlation was conducted between (1) lever pressing rate and EBS frequency (Hz; phase 3) to verify that changes in EBS frequency altered ICSS, and (2) free-choice ratio and probability magnitude (phase 6) to determine if the two groups acquired the discounting task. To determine treatment-induced changes in free-choice ratio collected in phases 6–9, a two-way repeated measures (rm) ANOVA was conducted.

For phases 6 and 7, day and probability were factors. For phases 8 and 9, phase and probability were factors. A post hoc Newman–Keuls provided individual comparisons. Forelimb stepping was similarly analyzed with time and dose as factors. If a data point exceeded two SD from the group mean, it was considered an outlier and it was excluded from analysis. Significance was p <0.05 for group/treatment comparisons; data are reported as group means±SEM.

Intradorsolateral Striatal Injections of 6-OHDA Produced Persistent Motor Deficits That Were Reversed by Pramipexole

We conducted an initial study to validate that the DLS infusions of 6-OHDA resulted in a lesion that was sufficiently robust and persistent to produce stable and enduring reduction of TH in the DLS, and in deficits in forelimb stepping, similar to a previous report ( Chang et al, 1999 ). The DLS of 6-OHDA-treated rats showed profound reductions in TH staining that persisted for 60 days ( Figure 2 ). For the six rats killed at 21 days after lesion, the tissue sections that showed the largest lesion extent were between +1.2 mm and +0.7 mm anterior to bregma, and the lesion could be detected from +2.2 mm to −0.26 mm. Although all rats had similar presurgery baseline stepping, those treated with 6-OHDA displayed stepping deficits in both left and right forelimbs when tested in both the adduction and abduction direction. These deficits, which were similar for 21 and 60 days after lesion, were ∼ 40–50% of that obtained from sham rats (see Table 2 ). These data were analyzed using a planned contrast two-way rmANOVA. For all four parameters tested, there was a significant ( p <0.05) effect of treatment group and postsurgery time, and group by time interactions (Left abduction: group F 2, 48 =13.63, time F 1, 48 =193.95, interaction F 2, 48 =20.84; Right abduction: group F 2, 48 =14.21, time F 1, 48 =142.02, interaction F 2, 48 =15.78; Left adduction: group F 2, 48 =12.30, time F 1, 48 =146.04, interaction F 2, 48 =40.01; Right adduction: group F 2, 48 =3.30, time F 1, 48 =54.83, interaction F 2, 48 =18.37). This study verified that the 6-OHDA treatment protocol profoundly reduced dopaminergic innervation of the DLS and this lesion was sufficiently robust and enduring to produce stable deficits in motor function that persist for at least 60 days. Thus, this 6-OHDA treatment protocol was employed for the subsequent ICSS studies, and stepping adjustments of the left forepaw in the abduction direction were used as the representative motor index of the DLS lesion.

figure 2

Dorsolateral striatal lesions. (a) Representative photomicrographs of tyrosine hydroxylase-immunohistochemistry (TH-IHC) at the level of the DLS ( ∼ 1.0 mm AP from bregma) in one hemisphere. Compared with sham (vehicle-injected; left), 6-OHDA reduced staining in the DLS at 21 days (middle) and 60 days (right) after treatment. Scale bar=1 mm. (b) Bilateral illustration of the extent and location of 6-OHDA-induced lesions 21 days after injection. For the six rats killed at this time, the tissue sections that were targeted during surgery (1.0 mm anterior to bregma) were analyzed by two observers. Each independently outlined the TH-like staining for the section for each rat. The outermost borders delineated by lack of staining were determined. Illustrated are the outlines for the largest lesion area from both observers (neuroanatomical plates modified from Paxinos and Watson, 1998 ). The borders of the lesion after 60 days were less discrete (see (a), far right); but in general, the lesion size was similar to that seen at 21 days.

To verify that the deficits remained throughout the 85 days needed to complete the study, a separate group of rats that completed the ICSS-mediated discounting paradigm ( n =21) were also assessed for forelimb stepping each week after surgery. We determined that stepping remained at ∼ 17 steps/session for control rats and 4–5 steps/session for PD-like rats. Similar to the rats tested 60 days after lesion (discussed above), these motor deficits persisted throughout the study (ie, for 85 days, data not shown).

To evaluate the ability of (±)PPX to reverse 6-OHDA-induced motor deficits, rats that failed to meet acquisition phase criteria in the discounting paradigm ( n =5/16 PD-like; 7/17 shams; refer to Table 1 for criteria) were used. For these rats, the presurgery baseline average of adjusting steps/session were 14–15 and this level was not altered in control rats by either vehicle treatment or any dose of (±)PPX tested (data not shown). In contrast, PD-like rats showed a significant effect of (±)PPX dose (F 5, 20 =34.17, p <0.01) and post-treatment time (F 3, 60 =316, p <0.01) and an interaction (F 15, 60 =46.88, p <0.01). As shown in Figure 3 , at doses ranging from 0.5 to 4.0 mg/kg IP, (±)PPX improved stepping deficits in PD-like rats at 1 h after treatment; 1.0–4.0 mg/kg maintained stepping improvements for at least 6 h after treatment. Adjusting steps returned to pre-(±)PPX deficit levels 17 h after injection for all doses tested. The stepping deficit was not altered by vehicle or 0.25 mg/kg (±)PPX. The 2 mg/kg (±)PPX dose produced robust motor improvements that persisted for 6 h and yet was below maximal improvement seen. Furthermore, this dose was not sufficient to influence behavior at 17 h after injection ( Figure 3 ). Therefore, the treatment given in the late afternoon was mostly cleared from the animals before the morning injection, and (±)PPX likely did not accumulate during the repeated injections. These outcomes guided the dosing protocol selected for the probability discounting paradigm, that is, 2 mg/kg (±)PPX (ie, 1 mg/kg of the active form), administered twice a day. This decision was also guided by reports that (1) 1 mg/kg of (−)PPX alters reward-mediated behavior, that is, enhances the reinforcing effects of cocaine ( Caine et al, 1997 ) and (2) twice-daily injections of 1 mg/kg of (−)PPX in rats increases expression of forebrain D3 receptors ( Maj et al, 2000 ), which are involved in ICDs and addictions ( Heidbreder and Newman, 2010 ).

figure 3

Motor deficits produced by intradorsolateral striatal injections of 6-OHDA are reversed by pramipexole. Illustrated is adjusting stepping from the left forelimb in the abduction direction for PD-like rats. At ∼ 40 days after the lesion surgery, rats underwent a series of weekly step tests. Pre-(±)PPX deficits (Before) were obtained immediately before the (±)PPX injection. The ±PPX reversed these stepping deficits in a dose-dependent manner. The (±)PPX significantly increased the number of adjusting steps with 0.5, 1, 2, and 4 mg/kg at 1 h, whereas at 6 h this increase was only seen with 1, 2, and 4 mg/kg. The number of adjusting steps returned to pretreatment levels 17 h after injection. No change from before injection was seen after an injection with vehicle or 0.25 mg/kg of (±)PPX. The post-hoc Newman–Keuls: * vs before (±)PPX injection. Arrows indicate times of (±)PPX injection.

PD-Like and Control Rats Performed Similarly in the Probability Discounting Paradigm

The post-mortem histological evaluations verified that rats completing the ICSS-mediated paradigm had electrode tip placements located in the lateral hypothalamus ( Figure 4 ). To determine if PD-like rats differed from controls in any aspect of paradigm acquisition, performance in phases 1–6 was monitored and compared for the two groups (refer to Table 1 ). All rats quickly acquired stable ICSS lever pressing, and both groups lever pressed on an FR-1 at similar rates. Similarly, the ECur 90 , ECur 60 and ECur 40 obtained from each rat's ICSS rate vs current frequency curve did not differ between groups. The averaged rate-frequency functions for PD-like and control rats are graphically indicated in Figure 5 . Both groups exhibited significant linear regressions (PD-like, r 2 =0.94, p <0.01; and control rats, r 2 =0.91, p <0.01). The two groups learned and met phase criteria for the discrete trials and the choice tests in a similar time frame ( Table 1 ). All rats that entered phase 6 were able to learn the discounting task. Figure 6 illustrates that both groups acquired the probability discounting task in the first session of phase 6, as demonstrated by a reduction in selection of the LR lever as the probability for delivery of the large reinforcer decreased (PD-like rats: r 2 =0.73, p <0.01; control rats: r 2 =0.85, p <0.01). Although the range for individual rats to obtain stable baseline discounting was 3 to 6 days, as groups, both the PD-like and control rats met stability criteria in the first 3 test days. For these 3 days of discounting, control rats showed an effect of probability (F 8, 216 =47.89, p <0.01) but no day effect (F 2, 27 =0.32, p =0.73) or interaction (F 16, 216 =1.17, p =0.29). There were seven data points removed because of meeting statistical outlier criteria. Similarly, for PD-like rats, there was an effect of probability (F 8, 240 =62.6, p <0.01), without an effect of day (F 2, 30 =0.23, p =0.80) or interaction (F 16, 240 =1.3, p =0.20). There were eight data points removed because of meeting statistical outlier criteria. Thus, for both groups there was a direct relationship between reward probability and free-choice ratio that did not differ for the first 3 baseline test days. For this phase, 9 of 1134 total blocks had response omissions of ⩾ 50%, and were omitted from the free-choice ratio analyses.

figure 4

Electrode placement for intracranial self-stimulation (ICSS). Illustration of the stimulation electrode tip location within the lateral hypothalamus (LH) for 6-OHDA-treated (open circles, n =11) and sham (closed squares, n =10) rats that completed the probability discounting paradigm. Neuroanatomical plates were modified from Paxinos and Watson (1998 ) and numbers indicate the distance in mm from bregma. Note that the LH regions stimulated were similar for both groups of rats.

figure 5

ICSS rate-frequency functions: comparisons between 6-OHDA-treated and sham rats. The relationship between ICSS lever pressing rate and stimulation frequency was similar for 6-OHDA-treated ( n =11) and sham ( n =10) rats. Shown are the group means±SEM from stable curves generated by each rat. Plots are drawn as a third-order polynomial to visualize Emax and threshold.

figure 6

Acquisition of the probability discounting task. During phase 6, 6-OHDA-treated ( n =11) and sham ( n =10) rats acquired the probability discounting task during the first training session. Illustrated are the group means±SEM for the percent selection of the large/risky (LR) lever (ie, free-choice ratio) vs the probability that the large reinforcer was delivered for the first discounting session . The plot is drawn as a linear regression.

Pramipexole Increased Discounting in the Probability Discounting Task

To determine if (±)PPX altered probability discounting, rats were treated with 2 mg/kg (±)PPX twice a day (approximately 0800 h and 1700 h) for 13 days during phase 7. Discounting was measured 30 min and 6 h after the morning injection approximately every 3 days. These data were compared with pretreatment baseline sessions that were similarly conducted twice a day. Thus, to control for the possible effects of time of day for testing on outcomes, morning baseline sessions were compared with the tests taken 30 min after (±)PPX (also a morning test), and evening baseline sessions were compared with the tests taken 6 h after the morning (±)PPX injection (refer to Figure 7 ). For PD-like rats, comparisons of free-choice ratio for morning baseline to 30 min after the 1st and 25th (±)PPX injection revealed enhanced discounting ( Figure 7a ). There was a significant effect of test (ie, baseline, 1st, and 25th injection of ±PPX; F 2, 29 =14.45, p <0.01) and probability (F 8, 232 =31.07, p <0.01) and an interaction (F 16, 232 =5.85, p <0.01). Similarly, comparison of evening baseline with 6 h following the 1st and 25th (±)PPX injection revealed that ±PPX-induced heightened discounting was sustained ( Figure 7c ), with a significant effect of test (F 2, 30 =28.78, p <0.01), probability (F 8, 240 =65.42, p <0.01), and the interaction (F 16, 240 =6.49, p <0.01). A post-hoc Newman–Keuls comparison revealed that discounting was most pronounced following the 25th (±)PPX treatment for both 30 min and 6 h after injection ( Figure 7 ). Unexpectedly, some control rats exhibited a large number of trial omissions 30 min following the first treatment of (±)PPX. Observation of these rats in the operant boxes revealed that they were engaged in continuous stereotypic sniffing and licking of the floor metal bars, with some head bobbing. The behaviors abated 6 h after the (±)PPX injection. The rats became tolerant to the motor effects, for on the fourth day of treatment (and the second discounting test) they were fully engaged in the lever pressing task and discounting performance could be accurately evaluated. However, the acute motor confound precluded discounting assessments for the first, 30 min post-(±)PPX treatment in control rats. After the seventh injection (ie, the fourth day of (±)PPX treatment), control rats clearly demonstrated increased discounting, as the selection for the risky lever at the 0.05, 0.15, and 0.30 probabilities were greater than baseline by 31%, 25%, and 27%, respectively. Figure 7b illustrates the enhancement in discounting observed 30 min after the 25th injection for control rats. There was a significant effect of test (F 1, 18 =4.97, p =0.04) and probability (F 8, 144 =20.93, p <0.01) and an interaction (F 8, 144 =3.22, p <0.01). Comparison of evening baseline testing with the 1st and 25th injection for the 6 h period also showed a significant increase in risky behavior ( Figure 7d ), with a significant effect of test (F 2, 26 =22.34, p <0.01), probability (F 8, 208 =42.21, p <0.01), and an interaction (F 16, 208 =2.18, p <0.01). As illustrated in Figure 7d , a post-hoc Newman–Keuls comparison revealed that discounting was most pronounced following the 25th (±)PPX treatment. For this phase, 50 out of 2457 total blocks had response omissions of ⩾ 50%, and were omitted from the free-choice ratio analyses.

figure 7

Pramipexole increased probability discounting. In phase 7, 6-OHDA-treated ( n =11) and sham ( n =10) rats received 2 mg/kg (±)PPX IP twice a day for 13 days for a total of 26 injections. Discounting sessions were conducted 30 min and 6 h after the morning (AM) injection, on the first and every third day after initiating the treatment. Data from these two sessions were compared with the pretreatment baseline (BL0) for the respective time periods. The (±)PPX increased discounting in 6-OHDA-treated rats tested after the first (±)PPX treatment and the 25th (±)PPX treatment at both (a) 30 min and (c) 6 h after injection. Similar increases in discounting were seen in sham rats (b) 30 min and (d) 6 h after (±)PPX treatment. Shown is the percent selection of the large/risky (LR) lever (ie, free-choice ratio) vs the probability that the large reinforcer was delivered. The post-hoc Newman–Keuls: * vs BL; # vs first injection.

To help interpret (±)PPX-induced changes in probability discounting, we evaluated the effects of the agonist on various behaviors that are critical for the discounting task. First, demonstrating that rats maintained their ability to discriminate among the reinforcement values (ie, no reward vs small reward vs large reward), we determined at various times during the (±)PPX treatment that responding in the Choice Test (ie, the phase 5 protocol) was preserved (ie, selection for the larger reinforcer was approximately ⩾ 70%) for both PD-like ( n =9) and control ( n =8) rats. Second, we determined the ability of (±)PPX to alter the reward values. Following the 13 days of (±)PPX in phase 7, a subset of rats (controls, n =5 and PD-like, n =3) continued to receive 2 mg/kg (±)PPX twice a day for 3 additional days and the lever pressing rate vs ICSS current frequency (ie, the phase 3 protocol) was assessed. The ECur 90 was similar between baseline (as determined in phase 3) and chronic (±)PPX for both groups (controls paired t -test (4) =0.89, p =0.43; PD-like t -test (2) =2.5, p =0.13). However, (±)PPX increased the rate of lever pressing at the lowest ICSS frequencies with a decrease in apparent threshold for both groups (data not shown), and for the PD-like group there was an associated reduction in ECur 40 (paired t -test (2) =7.35, p =0.02). This shift went from 100 Hz at baseline to 52 Hz after the 32nd (±)PPX treatment. Such a change was not seen in control rats (paired t -test (4) =1.2, p =0.3). As a collective, these evaluations indicated that even though the value of the small reward may have been enhanced by (±)PPX, the rats continued to recognize ECur 40 as less than the ECur 90 so as to correctly execute the Choice Test and linked Discounting Test throughout the chronic (±)PPX treatment protocol.

Discontinuation of Pramipexole Decreased Probability Discounting

Following discontinuation of (±)PPX treatment, rats were continually assessed for discounting in phase 8. No overt behavioral indices of withdrawal were observed (eg, body weight, grooming), and hereafter the term ‘withdrawal’ is used to indicate the absence of drug treatment, not a behavioral index. At 3 days following the last injection, both control and PD-like rats maintained an increase in preference for the LR lever; however, reductions in this LR lever preference were evident 15 days after treatment cessation. Within this time period, some rats began to show a decrease in general performance and an increase in omissions during the discounting task (ie, >10 omitted trials out the 20 total); therefore, these rats were removed from the study. Of the rats that maintained performance, eight were PD-like and three were controls. For the PD-like rats, after 15 days of (±)PPX withdrawal, selection for the LR lever decreased as compared with 30 min after the 25th (±)PPX injection ( Figure 8a ). There was a significant effect of phase (ie, withdrawal vs 25th (±)PPX injection; F 1, 14 =7.29, p =0.02), probability (F 8, 112 =16.96, p <0.01), and an interaction (F 8, 112 =2.24, p =0.03). Indeed, discounting during this withdrawal time was nearly indistinguishable from baseline behavior; at the three lowest probabilities (ie, 0.05, 0.15, and 0.3), rats respectively selected the LR lever 52%, 55%, and 59% of the time during baseline and 42%, 55%, and 65% during withdrawal from chronic (±)PPX. As illustrated in the inset of Figure 8a , the three control rats demonstrated similar reduction in discounting as observed in the PD-like rats. That is, after 15 days of withdrawal, control rats selected the LR lever 44%, 29%, and 43% of the time at the three lowest probabilities (ie, 0.05, 0.15, and 0.3, respectively), which was similar to baseline values of 39%, 50%, and 56%, respectively. During this (±)PPX withdrawal period, 4 out of 198 total blocks had response omissions of ⩾ 50%, and were omitted from the free-choice ratio analyses.

figure 8

Withdrawal from pramipexole decreased probabilistic discounting whereas reinitiation of pramipexole reinstated the increase in discounting. Shown is the percent selection for the large/risky (LR) lever (ie, free-choice ratio) vs the probability that the large reinforcer was delivered. (a) Phase 8; (±)PPX treatments were terminated. Illustrated are data from 6-OHDA-treated rats ( n =8). Discounting measured on days 12 and 15 of withdrawal were averaged for each rat, and group data were compared with discounting obtained 30 min after the 25th (±)PPX injection. Inset illustrates data from sham rats ( n =3); smooth line indicates 25th injection of (±)PPX and dotted line indicates withdrawal phase. (b) Phase 9; ±PPX treatment was reinitiated in a subset of withdrawn 6-OHDA-treated rats ( n =6). Rats received 2 mg/kg (±)PPX IP twice a day for 7 days for a total of 14 injections. Discounting measured on the last 2 withdrawal days was averaged for each rat, and group data were compared with discounting data collected after the 13th (±)PPX injection during reinitiation. The post-hoc Newman–Keuls: * vs withdrawal.

Reinitiation of Pramipexole Reinstated Increased Discounting

A subset of drug-withdrawn rats ( n =6; all PD-like) maintained successful performance of the discounting task and thus were continually tested up to 69 days after treatment. Throughout this time period, discounting remained near baseline levels ( Figure 8b ). Subsequently, the twice-daily 2 mg/kg (±)PPX treatment was reinitiated. The increase in discounting was reinstated by the seventh day of treatment (ie, after the 13th injection; Figure 8b ), with a significant effect of phase (ie, withdrawal vs reinstatement; F 1, 10 =6.38, p <0.03), probability (F 8, 80 =10.06, p <0.01), and an interaction (F 8, 80 =5.86, p <0.01). The increase in discounting seen with reinstatement of (±)PPX was very similar to that obtained during the initial (±)PPX treatment. Indeed, at the three lowest probabilities (ie, 0.05, 0.15, and 0.3) during the initial (±)PPX treatment, 30 min after the 25th injection, rats respectively selected the LR lever 77%, 72%, and 90% of the time, which is comparative with 79%, 90%, and 84% (respectively) taken 30 min after the 13th reinstatement injection. During the reinstatement assessments, 1 out of 54 total blocks had response omissions of ⩾ 50%, and these were omitted from the free-choice ratio analyses.

Probability discounting is a popular method to study risky decision making, and problem gamblers demonstrate increased discounting in these paradigms ( Holt et al, 2003 ; Madden et al, 2009 ; Petry, 2011 ). The current study utilized our new rat model of probability discounting that employs ICSS as the positive reinforcer ( Rokosik and Napier, 2011 ) to reveal that (±)PPX increased discounting. We also revealed that tolerance did not develop with repeated treatments, and responding was comparable between PD-like and control rats. Additionally, we verified that increases in discounting returned to baseline levels within 2 weeks of (±)PPX treatment cessation, and re-exposure to (±)PPX reinstated heightened discounting. These outcomes are in line with clinical reports wherein ICD onset is related to onset of DA agonist treatment, and symptoms typically subside with dose reduction or discontinuation ( Dodd et al, 2005 ; Driver-Dunckley et al, 2007 ; Mamikonyan et al, 2008 ; Quickfall and Suchowersky, 2007 ). Thus, using ICSS for risk assessments in rats exhibits high face validity to the human experience with PPX.

ICSS provides an immediate and robust reward that does not suffer from satiety/tolerance, or cause any withdrawal-like symptoms. Using ICSS, as opposed to food reinforcement, proved to be exceptionally advantageous for evaluating the effects of chronic (±)PPX treatment on probability discounting. First, the ICSS-mediated discounting task was acquired by rats in the first test session, and stable baseline discounting was achieved in 3 days of testing. This contrasts food reinforcement discounting where typically 10 test sessions are needed for acquisition and 25–35 days are required to reach stable discounting behavior ( St Onge et al, 2010 ; Ghods-Sharifi et al, 2009 ). Second, ICSS allows for testing several probabilities in a randomized order, a feature that is not successfully implemented with food-reinforced discounting ( St Onge et al, 2010 ). Randomization encourages rats to continue selecting the LR lever even at very low probabilities (in contrast to what is obtained with protocols using predictable, descending probabilities; Rokosik and Napier, 2011 ). Thus, we were able to detect both increases and decreases in selection of the LR lever at the lowest probabilites, where the most robust discounting often occurs. Finally, in food reinforcement studies, animals typically are food deprived to motivate them to perform the operant tasks. Food restriction alters the behavioral effects of PPX ( Collins et al, 2008 ), which could confound outcomes of discounting tests with the agonist. To summarize, ICSS afforded a means to unambiguously assess discounting during chronic drug administration, following subsequent, long-term cessation of treatment, and drug reinstatement, all in the same test subjects.

The current study demonstrated the ability of a rodent model of PD to perform a probability discounting task. Although PD-like rats were robustly and persistently impaired in the forelimb adjusting step test, they readily performed the lever-pressing tasks and they did not show any behavioral deficiencies in the acquisition or execution of the discounting paradigm. Moreover, the PD-like rats displayed similar profiles as controls with regard to the reinforcing properties of ICSS currents (as assessed in the lever pressing rate vs current frequency profiles) and basal discounting. These observations indicate that DA deafferentation of the DLS does not alter the capacity, or motivation, to perform ICSS-mediated probability discounting.

Acute PPX treatment in healthy humans can increase measures of impulsiveness ( Riba et al, 2008 ) as well as disrupt reward-related learning ( Pizzagalli et al, 2008 ; Santesso et al, 2009 , but see also Hamidovic et al, 2008 ). As a therapeutic agent, PPX can promote problem gambling independent of the pathology for which the drug is prescribed (eg, PD ( Seedat et al, 2000 ; Weintraub et al, 2010 ), RLS ( Quickfall and Suchowersky, 2007 ; Tippmann-Peikert et al, 2007 ), fibromyalgia ( Holman, 2009 ), and bipolar depression ( Strejilevich et al, 2011 )). It is unclear if these pathological conditions render individuals more susceptible to the impulsivity-related effects of PPX. Given that PPX is highly prescribed during the early stages of PD and reports suggest these patients have a relatively high incidence of PPX-induced ICDs ( Weintraub et al, 2010 ), we included a model of PD in the current study. However, we demonstrated here that a brain state that models aspects of early stages of PD did not render rats more sensitive to the (±)PPX-induced effects. It should be noted that this lack of differentiation between PD-like rats and controls may reflect the relatively high dose of (±)PPX studied, and lower doses of the agonist may be able to discriminate the two groups. Our findings that (±)PPX increased discounting in control rats are in line with food-reinforcement studies using food-restricted intact laboratory rats, wherein (−)PPX increases preference for a gambling-like schedule of reinforcement (ie, variable ratio; Johnson et al, 2011 ). These converging preclinical findings support a link between PPX treatment and alterations in decision making with regard to discounting.

In humans tested in probabilistic choice tests, PPX can disrupt learning from negative outcomes (ie, when a reward is expected but not delivered; Cools et al, 2006 ; Bodi et al, 2009 ). In probability discounting, when the probability of delivery of the large reinforcer is very low (eg, 0.05, 0.15, and 0.3), the likelihood of not receiving a reward is at the highest. Negative outcomes during these low probabilities likely lessen the appeal of lever pressing for the large reinforcer and shift preference to the SC lever. This profile was seen in the current study for tests during baseline and withdrawal. In contrast, (±)PPX enhanced responding on the risky lever during low probability. This outcome is consistent with the agonist reducing the negative consequences of a non-rewarded response. A similar outcome might be predicted if (±)PPX reduced the value of ICSS reward; however, the ECur 90 (current level used for the large reward of the rats) was not altered by chronic (±)PPX and the ECur 40 (the small reward) was slightly elevated. Although we have recently determined with a condition place preference paradigm that (±)PPX can support reward-mediated associated learning ( Riddle et al, 2010 ), outcomes from the current operant task suggest that (±)PPX may increase discounting by reducing the perceived negativity of un rewarded operant responses rather than enhancing the value of the reward associated with the risky lever. This interpretation is supported by clinical studies with functional magnetic resonance imaging (fMRI) that investigated the influence of PPX on reward prediction errors during a gambling task. A positive reward prediction error occurs if an unpredicted reward is encountered and negative reward prediction error occurs if a predicted reward is omitted. In one study, PD patients treated with PPX showed a correlation between increases in risk taking and impairments in the deactivation of the fMRI signal in the orbitofrontal cortex during trials with a negative prediction error ( van Eimeren et al, 2009 ). This suggests that the subjects were impaired from learning in trials in which losing occurred. In another study, RLS patients treated chronically with DA agonists, including PPX, demonstrated increases in fMRI signaling in the ventral striatal during trials in which expected rewards were omitted ( Abler et al, 2009 ). It is noteworthy that the PPX-induced effects were observed in all RLS patients tested, similar to the ability of (±)PPX to enhance discounting in all rats tested in the current study. Nevertheless, none of the RLS patients developed an ICD ( Abler et al, 2009 ). This outcome underscores the fact that enhancement in discounting or risky behaviors is not equivalent to developing an ICD per se but likely represents a particular aspect of these complicated disorders.

Which receptors mediate the behavioral effects of PPX is unclear. PPX is a direct-acting DA agonist with a preference for the D3R subtype of DA receptors. For example, in in vivo rat studies using presumed D2R- and D3R-selective behavioral assays (ie, hypothermia and yawning, respectively), PPX is ∼ 30-fold selective for D3R over D2R ( Collins et al, 2007 ), and 1.0 mg/kg (−)PPX is sufficient to activate both D2R and D3R ( Collins et al, 2005 , 2007 , 2009 ). Thus, it is likely that both subtypes were engaged by 2 mg/kg dose of (±)PPX used in the current study. Indeed, both D2 and D3R have been implicated in reward-mediated behaviors ( Heidbreder et al, 2005 ; Self, 1998 ) and impulsivity ( St Onge and Floresco, 2009 ; van Gaalen et al, 2009 ; Buckholtz et al, 2010 ). Additional probability discounting studies including those with lower doses of PPX as well as receptor-subtype selective antagonists would aid in elucidating the particular receptor(s) involved in PPX-induced enhancement in discounting.

The (±)PPX shifted discounting in PD-like rats with a single injection; however, repeated treatments were required to reach maximal discounting. These findings indicate that acute occupation of relevant DA receptors is sufficient to enhance discounting; however, the adaptations in this system that were imposed with chronic administration may promote the effect. Chronic PPX treatments can lead to desensitization of DA neuronal D2/D3 autoreceptors ( Chernoloz et al, 2009 ) and an increase in expression of D3R in dopaminoceptive regions ( Maj et al, 2000 ). Whatever the mechanism, the neuroadaptations were reversible in the current study, for when (±)PPX treatment was discontinued for 2 weeks, discounting decreased near baseline levels. These findings concur with clinical reports showing that DA agonist-induced ICDs in humans can be eliminated with drug discontinuation ( Macphee et al, 2009 ; Mamikonyan et al, 2008 ; Quickfall and Suchowersky, 2007 ; Dodd et al, 2005 ; Driver-Dunckley et al, 2007 ).

In summary, converging evidence suggests that PPX can influence the processing of rewards and drive decision making towards higher discounting and more risky choices. The animal model of (±)PPX-induced discounting presented here provides a valuable new means to elucidate the pharmacological and neurobiological underpinnings of this aspect of impulsivity. This model should prove useful in the development of novel therapeutics devoid of enhancing discounting as well as a means to screen current and future compounds for their potential to promote risky behaviors.

Abler B, Hahlbrock R, Unrath A, Gron G, Kassubek J (2009). At-risk for pathological gambling: imaging neural reward processing under chronic dopamine agonists. Brain 132 : 2396–2402.

Article   Google Scholar  

Bodi N, Keri S, Nagy H, Moustafa A, Myers CE, Daw N et al (2009). Reward-learning and the novelty-seeking personality: a between- and within-subjects study of the effects of dopamine agonists on young Parkinson's patients. Brain 132 : 2385–2395.

Bonuccelli U, Del DP, Rascol O (2009). Role of dopamine receptor agonists in the treatment of early Parkinson's disease. Parkinsonism Relat Disord 15 (Suppl 4): S44–S53.

Buckholtz JW, Treadway MT, Cowan RL, Woodward ND, Li R, Ansari MS et al (2010). Dopaminergic network differences in human impulsivity. Science 329 : 532.

Article   CAS   Google Scholar  

Caine SB, Koob GF, Parsons LH, Everitt BJ, Schwartz JC, Sokoloff P (1997). D3 receptor test in vitro predicts decreased cocaine self- administration in rats. Neuroreport 8 : 2373–2377.

Chang JW, Wachtel SR, Young D, Kang UJ (1999). Biochemical and anatomical characterization of forepaw adjusting steps in rat models of Parkinson's disease: studies on medial forebrain bundle and striatal lesions. Neuroscience 88 : 617–628.

Chernoloz O, El MM, Blier P (2009). Sustained administration of pramipexole modifies the spontaneous firing of dopamine, norepinephrine, and serotonin neurons in the rat brain. Neuropsychopharmacology 34 : 651–661.

Collins GT, Calinski DM, Newman AH, Grundt P, Woods JH (2008). Food restriction alters N′-propyl-4,5,6,7-tetrahydrobenzothiazole-2,6-diamine dihydrochloride (pramipexole)-induced yawning, hypothermia, and locomotor activity in rats: evidence for sensitization of dopamine D2 receptor-mediated effects. J Pharmacol Exp Ther 325 : 691–697.

Collins GT, Newman AH, Grundt P, Rice KC, Husbands SM, Chauvignac C et al (2007). Yawning and hypothermia in rats: effects of dopamine D3 and D2 agonists and antagonists. Psychopharmacology (Berl) 193 : 159–170.

Collins GT, Truccone A, Haji-Abdi F, Newman AH, Grundt P, Rice KC et al (2009). Proerectile effects of dopamine D2-like agonists are mediated by the D3 receptor in rats and mice. J Pharmacol Exp Ther 329 : 210–217.

Collins GT, Witkin JM, Newman AH, Svensson KA, Grundt P, Cao J et al (2005). Dopamine agonist-induced yawning in rats: a dopamine D3 receptor-mediated behavior. J Pharmacol Exp Ther 314 : 310–319.

Cools R, Altamirano L, D'Esposito M (2006). Reversal learning in Parkinson's disease depends on medication status and outcome valence. Neuropsychologia 44 : 1663–1673.

Deumens R, Blokland A, Prickaerts J (2002). Modeling Parkinson's disease in rats: an evaluation of 6-OHDA lesions of the nigrostriatal pathway. Exp Neurol 175 : 303–317.

Dodd ML, Klos KJ, Bower JH, Geda YE, Josephs KA, Ahlskog JE (2005). Pathological gambling caused by drugs used to treat Parkinson disease. Arch Neurol 62 : 1377–1381.

Driver-Dunckley ED, Noble BN, Hentz JG, Evidente VG, Caviness JN, Parish J et al (2007). Gambling and increased sexual desire with dopaminergic medications in restless legs syndrome. Clin Neuropharmacol 30 : 249–255.

Evans AH, Strafella AP, Weintraub D, Stacy M (2009). Impulsive and compulsive behaviors in Parkinson's disease. Mov Disord 24 : 1561–1570.

Ghods-Sharifi S, St Onge JR, Floresco SB (2009). Fundamental contribution by the basolateral amygdala to different forms of decision making. J Neurosci 29 : 5251–5259.

Hamidovic A, Kang UJ, de WH (2008). Effects of low to moderate acute doses of pramipexole on impulsivity and cognition in healthy volunteers. J Clin Psychopharmacol 28 : 45–51.

Heidbreder CA, Gardner EL, Xi ZX, Thanos PK, Mugnaini M, Hagan JJ et al (2005). The role of central dopamine D3 receptors in drug addiction: a review of pharmacological evidence. Brain Res Brain Res Rev 49 : 77–105.

Heidbreder CA, Newman AH (2010). Current perspectives on selective dopamine D(3) receptor antagonists as pharmacotherapeutics for addictions and related disorders. Ann NY Acad Sci 1187 : 4–34.

Holman AJ (2009). Impulse control disorder behaviors associated with pramipexole used to treat fibromyalgia. J Gambl Stud 25 : 425–431.

Holt DD, Green L, Myerson J (2003). Is discounting impulsive? Evidence from temporal and probability discounting in gambling and non-gambling college students. Behav Processes 64 : 355–367.

Johnson PS, Madden GJ, Brewer AT, Pinkston JW, Fowler SC (2011). Effects of acute pramipexole on preference for gambling-like schedules of reinforcement in rats. Psychopharmacology (Berl) 231 : 11–18.

Kish SJ, Shannak K, Hornykiewicz O (1988). Uneven pattern of dopamine loss in the striatum of patients with idiopathic Parkinson's disease. Pathophysiologic and clinical implications. N Engl J Med 318 : 876–880.

Macphee GJ, Copeland C, Stewart D, Grosset K, Grosset DG (2009). Clinical follow up of pathological gambling in Parkinson's disease in the West Scotland study. Mov Disord 24 : 2430–2431.

PubMed   Google Scholar  

Madden GJ, Petry NM, Johnson PS (2009). Pathological gamblers discount probabilistic rewards less steeply than matched controls. Exp Clin Psychopharmacol 17 : 283–290.

Maj J, Rogoi Z, Margas W, Kata M, Dziedzicka-Wasylewska M (2000). The effect of repeated treatment with pramipexole on the central dopamine D3 system. J Neural Transm 107 : 1369–1379.

Mamikonyan E, Siderowf AD, Duda JE, Potenza MN, Horn S, Stern MB et al (2008). Long-term follow-up of impulse control disorders in Parkinson's disease. Mov Disord 23 : 75–80.

Olsson M, Nikkhah G, Bentlage C, Bjorklund A (1995). Forelimb akinesia in the rat Parkinson model: differential effects of dopamine agonists and nigral transplants as assessed by a new stepping test. J Neurosci 15 : 3863–3875.

Paxinos G, Watson C (1998). The Rat Brain in Stereotaxic Coordinates . Academic Press: New York.

Google Scholar  

Petry NM (2011). Discounting of probabilistic rewards is associated with gambling abstinence in treatment-seeking pathological gamblers. J Abnorm Psychol ; e-pub ahead of print 15 August 2011.

Pizzagalli DA, Evins AE, Schetter EC, Frank MJ, Pajtas PE, Santesso DL et al (2008). Single dose of a dopamine agonist impairs reinforcement learning in humans: behavioral evidence from a laboratory-based measure of reward responsiveness. Psychopharmacology (Berl) 196 : 221–232.

Pourcher E, Remillard S, Cohen H (2010). Compulsive habits in restless legs syndrome patients under dopaminergic treatment. J Neurol Sci 290 : 52–56.

Przedborski S, Levivier M, Jiang H, Ferreira M, Jackson-Lewis V, Donaldson D et al (1995). Dose-dependent lesions of the dopaminergic nigrostriatal pathway induced by intrastriatal injection of 6-hydroxydopamine. Neuroscience 67 : 631–647.

Quickfall J, Suchowersky O (2007). Pathological gambling associated with dopamine agonist use in restless legs syndrome. Parkinsonism Relat Disord 13 : 535–536.

Riba J, Kramer UM, Heldmann M, Richter S, Munte TF (2008). Dopamine agonist increases risk taking but blunts reward-related brain activity. PLoS ONE 3 : e2479.

Riddle JL, Rokosik SL, Napier TC (2010). Pramipexole induces a conditioned place preference in parkinsonian-like and control rats. Program No 813 21, 2010. Neuroscience Meeting Planner, San Diego, CA: Society for Neuroscience, 2010 Online .

Rokosik SL, Napier TC (2011). Intracranial self-stimulation as a positive reinforcer to study impulsivity in a probability discounting paradigm. J Neurosci Methods 198 : 260–269.

Santesso DL, Evins AE, Frank MJ, Schetter EC, Bogdan R, Pizzagalli DA (2009). Single dose of a dopamine agonist impairs reinforcement learning in humans: evidence from event-related potentials and computational modeling of striatal-cortical function. Hum Brain Mapp 30 : 1963–1976.

Seedat S, Kesler S, Niehaus DJ, Stein DJ (2000). Pathological gambling behaviour: emergence secondary to treatment of Parkinson's disease with dopaminergic agents. Depress Anxiety 11 : 185–186.

Self DW (1998). Neural substrates of drug craving and relapse in drug addiction. Ann Med 30 : 379–389.

St Onge JR, Chiu YC, Floresco SB (2010). Differential effects of dopaminergic manipulations on risky choice. Psychopharmacology (Berl) 211 : 209–221.

St Onge JR, Floresco SB (2009). Dopaminergic modulation of risk-based decision making. Neuropsychopharmacology 34 : 681–697.

Strejilevich SA, Martino DJ, Igoa A, Manes F (2011). Pathological gambling in a bipolar patient treated with pramipexole. J Neuropsychiatry Clin Neurosci 23 : E2–E3.

Tippmann-Peikert M, Park JG, Boeve BF, Shepard JW, Silber MH (2007). Pathologic gambling in patients with restless legs syndrome treated with dopaminergic agonists. Neurology 68 : 301–303.

van Eimeren T, Ballanger B, Pellecchia G, Miyasaki JM, Lang AE, Strafella AP (2009). Dopamine agonists diminish value sensitivity of the orbitofrontal cortex: a trigger for pathological gambling in Parkinson's disease? Neuropsychopharmacology 34 : 2758–2766.

van Gaalen MM, Unger L, Jongen-Relo AL, Schoemaker H, Gross G (2009). Amphetamine decreases behavioral inhibition by stimulation of dopamine D2, but not D3, receptors. Behav Pharmacol 20 : 484–491.

Voon V, Fox SH (2007). Medication-related impulse control and repetitive behaviors in Parkinson disease. Arch Neurol 64 : 1089–1096.

Weintraub D, Koester J, Potenza MN, Siderowf AD, Stacy M, Voon V et al (2010). Impulse control disorders in Parkinson disease: a cross-sectional study of 3090 patients. Arch Neurol 67 : 589–595.

Download references

Acknowledgements

We thank Jennifer L Riddle for her excellent technical assistance. This work was supported by the Michael J Fox Foundation, the Parkinson's Disease Foundation, Parkinson's Research Center at Rush University, and the Daniel F and Ada L Rice Foundation.

Author information

Authors and affiliations.

Neuroscience Program, Loyola University Medical Center, Maywood, IL, USA

Sandra L Rokosik

Department of Pharmacology, Center for Compulsive Behaviors and Addiction, Rush University Medical Center, Chicago, IL, USA

Sandra L Rokosik & T Celeste Napier

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to T Celeste Napier .

Ethics declarations

Competing interests.

The authors declare no conflict of interest.

PowerPoint slides

Powerpoint slide for fig. 1, powerpoint slide for fig. 2, powerpoint slide for fig. 3, powerpoint slide for fig. 4, powerpoint slide for fig. 5, powerpoint slide for fig. 6, powerpoint slide for fig. 7, powerpoint slide for fig. 8, rights and permissions.

Reprints and permissions

About this article

Cite this article.

Rokosik, S., Napier, T. Pramipexole-Induced Increased Probabilistic Discounting: Comparison Between a Rodent Model of Parkinson's Disease and Controls. Neuropsychopharmacol 37 , 1397–1408 (2012). https://doi.org/10.1038/npp.2011.325

Download citation

Received : 04 October 2011

Revised : 04 December 2011

Accepted : 05 December 2011

Published : 18 January 2012

Issue Date : May 2012

DOI : https://doi.org/10.1038/npp.2011.325

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

  • pramipexole
  • probability discounting

This article is cited by

Noradrenergic contributions to cue-driven risk-taking and impulsivity.

  • Chloe S. Chernoff
  • Tristan J. Hynes
  • Catharine A. Winstanley

Psychopharmacology (2021)

Motor impulsivity and delay intolerance are elicited in a dose-dependent manner with a dopaminergic agonist in parkinsonian rats

  • Haritz Jiménez-Urbieta
  • María C. Rodríguez-Oroz

Psychopharmacology (2020)

Increased motor impulsivity in a rat gambling task during chronic ropinirole treatment: potentiation by win-paired audiovisual cues

  • Melanie Tremblay
  • Michael M. Barrus

Psychopharmacology (2019)

The neurobiology of impulse control disorders in Parkinson’s disease: from neurotransmitters to neural networks

  • Chris Vriend

Cell and Tissue Research (2018)

Impulse control disorders in Parkinson’s disease

  • Ana Marques
  • Franck Durif
  • Pierre-Olivier Fernagut

Journal of Neural Transmission (2018)

Quick links

  • Explore articles by subject
  • Guide to authors
  • Editorial policies

probability discounting task

Moderate Stability among Delay, Probability, and Effort Discounting in Humans

  • ORIGINAL ARTICLE
  • Open access
  • Published: 15 February 2023
  • Volume 73 , pages 149–162, ( 2023 )

Cite this article

You have full access to this open access article

  • Gisel G. Escobar   ORCID: orcid.org/0000-0001-6822-8341 1 , 2 ,
  • Silvia Morales-Chainé   ORCID: orcid.org/0000-0001-9269-7877 1 ,
  • Jeremy M. Haynes   ORCID: orcid.org/0000-0002-0811-3849 3 ,
  • Carlos Santoyo 1 &
  • Suzanne H. Mitchell   ORCID: orcid.org/0000-0002-0225-7200 4  

1857 Accesses

2 Citations

Explore all metrics

The stability of delay discounting across time has been well-established. However, limited research has examined the stability of probability discounting, and no studies of the stability of effort discounting are available. The present study assessed the steady-state characteristics of delay, probability, and effort discounting tasks across time with hypothetical rewards in humans, as well as whether response characteristics suggested a common discounting equation. Participants completed delay, probability, and effort discounting tasks on three occasions. We found moderate relative stability of delay and probability tasks, and similar evidence for absolute stability across time for all tasks. The interclass correlations coefficient showed some correspondence across time points and tasks, and higher levels of between subject variability, especially for the effort discounting task, suggesting trait level variables has a stronger influence on performance than state level variables. Performance on the delay and probability tasks were moderately correlated and similar mathematical functions fit choice patterns on both tasks (hyperbolic), suggesting that delay and probability discounting processes shared some common elements. Lower correlations and different function fits suggested that effort discounting involves more unique features.

Similar content being viewed by others

probability discounting task

On Four Types of Devaluation of Outcomes Due to Their Costs: Delay, Probability, Effort, and Social Discounting

Wojciech Białaszek, Paweł Ostaszewski, … Joel Myerson

probability discounting task

Devaluation of Outcomes Due to Their Cost: Extending Discounting Models Beyond Delay

On the functional form of temporal discounting: an optimized adaptive test.

Daniel R. Cavagnaro, Gabriel J. Aranovich, … Jay I. Myung

Avoid common mistakes on your manuscript.

Delay discounting is the process by which an outcome loses value as the delay to its receipt increases and is widely used to describe intertemporal choices in human and nonhuman animals (Odum, 2011 ; Rachlin et al., 1991 ). Measures of delay discounting in humans typically involve assessing preferences between hypothetical outcomes that vary in amount and delay. Steep delay discounting reflects a relative preference for smaller, sooner rewards (i.e., more impatience), and has been called a trans-disease process because of its association with many significant health problems (see Amlung et al., 2017 ; Bickel et al., 2012 ).

Delay can be considered a cost associated with receiving a delayed outcome (e.g., by having to wait for the delayed outcome), but delay is not the only cost associated with receiving an outcome. For example, probability and effort have been examined with tasks that have used a similar structure to those assessing delay discounting. Thus, probability discounting refers to the process by which an outcome loses value as the odds against its receipt increases (Rachlin et al., 1991 ), and steep probability discounting reflects a relative preference for smaller, certain rewards (i.e., less risk-taking, more risk aversion). Effort discounting refers to the process by which an outcome loses value as the effort required to earn it increases (Białaszek et al., 2019 ; Mitchell, 2004 ), and steep effort discounting reflects a relative preference for smaller, easier rewards (i.e., more effort aversion). There is considerably less known about effort discounting than about delay and probability discounting. This may be partly attributable to, as Pinkston and Libman ( 2017 ) noted, research manipulating effort requirements must consider that effort has several dimensions, including intensity and duration, and potential differences in its aversive effects (i.e., effort is not always aversive per se).

Research comparing delay and probability discounting in the same participants has suggested both can be described using the same equation, suggesting to many that a single-process is operating for both (e.g., Green & Myerson, 2013 ; Johnson et al., 2020 ; Rachlin et al., 1986 ; but also see Killeen, 2022 ). One way to compare discounting tasks has been to examine correlations between the degree of discounting on these tasks. However, this strategy has led to widely differing correlations between delay and probability discounting being reported. Two common study differences that may contribute to the lack of consensus include different study populations and different measures of the degree of discounting. Comparing studies by Mitchell ( 1999 ) and Białaszek et al. ( 2019 ) provides a clear example of these mismatches. Mitchell ( 1999 ) reported statistically significant, moderately positive correlations between delay and probability discounting gradients in regular smokers and never smokers. In contrast, Białaszek et al. ( 2019 ) did not find significant correlations between the two discounting tasks using the area under the discounting curve (AUC) values in healthy participants. However, both Mitchell ( 1999 ) and Białaszek et al. ( 2019 ) reported significant, moderately positive correlations between effort and probability discounting. Białaszek et al. ( 2019 ) also reported moderately positive significant correlations between delay and effort discounting, whereas Mitchell ( 1999 ) did not.

As noted above, one reason for the discordant results may be a difference in the measures used to examine the degree of discounting: gradients versus AUC. When using the gradient, studies have often just fitted a single function to the data from all types of discounting, without asking whether the same mathematical model fits the data equally well. In this study, we assessed three commonly used functions to determine which provides the best description of choice for a delay, probability, and effort discounting tasks. First, the hyperbolic model has been used extensively to describe delay and probability discounting (Mazur, 1987 ; Rachlin et al., 1991 ) as well as in a few studies with effort discounting (Mitchell, 1999 , 2004 ).

where V represents the subjective value of an outcome, A represents the amount of the outcome, X can be the delay or odds against or effort to receiving the outcome ( cost ), and b represents a free parameter indexing the degree of discounting. Second, the Rachlin ( 2006 ) hyperboloid model has shown an adequate fit to the choice data obtained from humans on discounting tasks (e.g., Franck et al., 2015 ; Young, 2017 ):

where the parameters are the same as in Equation 1 , and s represents a second free parameter indexing the scaling of delay/probability/effort. Both the hyperbolic and hyperboloid functions assume a convex shape of the discounting curve, which tends to underestimate reward value for lower delay/probability/effort levels.

Third, in effort discounting, power functions have also been used to describe discounting (Białaszek et al., 2017 ):

where the parameters are the same as in Equation 2 . However, this function produces a concave fit distinguishing it from the two previous models. The concave function tends to overestimate reward value for higher delay/probability/effort cost levels. To our knowledge, only the study by Białaszek et al. ( 2017 ) reported that the power function was the best-fitting model for effort discounting, compared to the hyperboloid models and the hyperbolic function. However, the majority of effort discounting studies have not examined the power function. We selected this model as the third candidate to examine in our study because additional research is needed to assess whether the power function provides the best description of data generated from effort discounting tasks, and the adequacy of its fits for delay and probability discounting.

One important feature of delay discounting is its stability over time (test–retest reliability), assessed by having a participant complete the discounting task in the same context on at least two occasions (see Odum et al., 2020 ). A stable state is “one in which the behavior in question does not change its characteristics over a period of time” (Sidman, 1960 , p. 234). This definition does not differentiate between two types of stability. First, relative stability, which refers to whether discounting changes in a similar way across people (i.e., correlations across time). Second, absolute stability, which refers to whether rates of discounting alter across time points (i.e., means across time). With respect to relative stability, delay discounting shows moderate-strong stability in humans with two ( r ≥ .70; Matusiewicz et al., 2013 ; Ohmura et al., 2006 ; Smits et al., 2013 ) and three time points (e.g., r ≥ .57; Kirby, 2009 ; Xu et al., 2013 ). However, data from Matusiewicz et al. and Smits et al. suggested that the type of outcomes (i.e., hypothetical, potentially real or experiential) might affect the relative stability of discounting measures. Other studies have examined the stability of probability discounting and also reported moderate-strong levels of stability r ≥ .54–.76 after a 1-week period (Matusiewicz et al., 2013 ), 3-month period (Ohmura et al., 2006 ), and 4-month period (Peters & Büchel, 2009 ). Fewer studies have reported on absolute stability, and to our knowledge, no study has examined the absolute or relative stability of effort discounting.

Studies that have examined the relative stability of the discounting measures have most commonly examined correlations between AUC across time points (Anokhin et al., 2015 ; Ohmura et al., 2006 ). Martínez-Loredo et al. ( 2017 ) used the intraclass correlations coefficient (ICC) to assess the absolute agreement or internal consistency of the observations across time. The ICC had served as a measure of reliability. The absolute stability is usually explored using paired t -test (if 2 time points) or with repeated measures ANOVA (if > 2 time points). Unfortunately, there are drawbacks to using these frequentist analysis procedures, including their treatment of missing data and focus on rejecting a null hypothesis of no differences. A Bayesian approach, in contrast, provides us with an index of the strength of evidence for the null and alternative hypotheses based on prior evidence and the current observed data (Young, 2019 ). Thus, to assess whether the degree of discounting is similar between time points for the three discounting tasks we adopted a Bayesian approach.

In summary, our study had two main aims. Aim 1 was to assess the steady-state/stable characteristics of choice patterns in delay, probability, and effort discounting tasks in humans. To do this, we first assessed discounting using an adjusting amount procedure (Du et al., 2002 ) on each of three time points. Because of uncertainty about the best index of discounting (see Aim 2), we then calculated the AUC for each task on each time point. Based on prior research (e.g., Kirby, 2009 ; Ohmura et al., 2006 ), we expected that delay and probability discounting would be stable, in relative and absolute terms, across all time points. In the absence of prior evidence, we had no predictions for effort discounting. Further, we explored the extent to which variability in delay, probability, and effort discounting could be attributed to between- and within-subject differences using a similar approach to prior studies in using rats (Haynes et al., 2021 ). Aim 2 was to determine whether individuals behaved similarly on the difference discounting tasks using an AUC analysis and, in particular, whether the same mathematical functions could describe delay, probability, and effort discounting equally well.

Participants

Twenty-three undergraduate Mexican students (8 male, 15 female) were recruited from a university in Mexico City as a convenience sample. All were between 18 and 22 years old ( M = 20.48; SD = 1.08) and an average of 1.64 m tall ( SD = 0.06). Only volunteers with a zero or low probability of substance use problems were accepted, as assessed using the World Health Organization-ASSIST v3.0 (Henry-Edwards et al., 2003 ; Linaje & Lucio, 2013 ). This was expected to reduce sample heterogeneity because there is compelling evidence that participants with substance use or abuse show steeper delay discounting than the controls (Amlung et al., 2017 ; Bickel et al., 2012 ). We also required that participants did not report any psychiatric diagnosis nor use any psychiatric medication. Those were the only inclusion and exclusion criteria. Participants were paid MXN$100 (US$4.99) for their participation using Amazon gift cards or recharge cell phone minutes, provided at the end of the second experimental session. They also received course extra credit for participation. All participants provided informed consent prior to participating.

Sessions were conducted online and participants used their own computers with either Windows10® or macOS® operating systems. AnyDesk®, an open-access application, was used to establish the remote connection between each participant and the laboratory research computer (macOS® Catalina version 10.15.6) so that participants could respond with their own keyboard. All tasks were programmed in Python through an open-access application, OpenSesame® version 3.3.5 (Mathôt et al., 2012 ).

Recruitment Survey

We used the Alcohol, Smoking and Substance Involvement Screening Test (World Health Organization-ASSIST v3.0; Henry-Edwards et al., 2003 ). Individuals report lifetime and 3-month use of a variety of substances and ASSIST evaluates whether they have a low, moderate, or high risk of substance use problems based on their pattern of use. We used the adapted version of ASSIST by Linaje and Lucio ( 2013 ) for Mexican young people.

A longitudinal within-subjects design was used. Due to COVID-19 pandemic restrictions, all interactions between the research team and participants occurred remotely using Zoom®, and all sessions were conducted individually. Recommendations provided by the American Psychological Association were used to maintain confidentiality while conducting online sessions (American Psychological Association, 2020 ).

Participants completed four online sessions on separate days: a screening and informed consent interview, followed by three experimental sessions. During the screening and informed consent interview, participants completed a screening questionnaire that recorded demographics characteristics and substance use history (World Health Organization-ASSIST v3.0). If participants met inclusion criteria (age and zero/low substance use risk), they were provided more details about the study requirements and completed the informed consent process. This interview required approximately 25 min. In the three experimental sessions, the participants performed three computer tasks assessing discounting (described below).

At the beginning of the first experimental session (time point 1), participants completed a calibration task with the help of the researcher overseeing the session, so that physical effort cost levels in the effort discounting task could be individualized. Our calibration task aimed to individualize the effort requirements in an analogous way to procedures adopted in other studies (e.g., Mitchell, 1999 , 2004 ; Sofis et al., 2017 ), and allowed us to avoid the assumption that all people respond to specific effort costs in the same way. The physical effort required was a specific number of steps. To identify this number, participants were asked to identify a flat, obstruction-free, 3–6 m surface on which they could walk. The researcher showed participants a prerecorded video in which a person demonstrated walking rhythm and body position to walk during the calibration task. After that, participants watched the video again and were asked to count the number of steps to verify that they understood the instructions. Then, participants put a webcam in a position to enable the researcher to observe the participant’s performance and ensure they conducted the steps correctly in each of the 6-min of the task. The researcher used a chronometer and participants were notified about when to start and finish each minute. This stage was used to shape the walking and to offer the same instructions to participants. Later, participants were instructed to walk similarly and count the number of steps as follows (all instructions were provided in Spanish but English translations are provided):

You will walk for a total of 6 min. You have to walk as fast as possible, no running, no jumping. For each minute, you must count the number of steps you have taken. I will tell you when to begin and end with the instructions "Go ahead" and "Stop." At the end of each minute, when you have stopped, you must report the number of steps taken. There will be a new count for each minute. The plan is to calculate the mean number of steps you are able to do in a minute.

The calibration test was not performed for the second and the third experimental sessions. Rather, the same mean number of steps was used for all three time points to remove this as a source of within-subject choice variability between sessions.

After completing the calibration task on the first experimental session, and at the beginning of the second and third sessions, participants made a second remote connection to the researcher’s computer using AnyDesk®. This software allowed participants to view and complete discounting tasks, but the researcher controlled all the procedures and participant responses were downloaded to the laboratory computer; participants did not have access to the software configurations. During the discounting tasks, the researcher’s webcam was off to reduce distraction, but the participant’s webcam and microphone were on so that the researcher could verify responses were occurring from the participant.

On all three experimental sessions, the participants read the following instructions to familiarize/refamiliarize them with the discounting tasks:

You will respond to a series of options to earn rewards. There are no correct or incorrect choices. There is no time limit to respond. You will not actually receive the rewards during the task nor at the end of the session, but we ask you to respond as if you were going to win them. The gains are not cumulative across the alternatives. Also, each choice you make is independent of the other choices. Choose the option that you prefer and not the one that someone else would choose. Respond according to your preferences today. Avoid responding based on the past or future. The options will be displayed on the screen. To make your choices: use the Z keyboard to select the options on the left side. Use the M keyboard to select the option on the right side.

Then, participants completed six forced-choice trials, where alternatives were presented in the same way as in the subsequent discounting tasks but participants were told which choices to make. Afterwards, participants were prompted to ask the researcher if they had any questions. Then, the first discounting task began. There was no break between tasks.

For the three discounting tasks, an adjusting amount procedure was used to obtain indifference points (IPs) across the different cost levels (Du et al., 2002 ). Each discounting task included 30 choice trials: six choices at each of five delays, six choices at each of five probability levels and six choices at each of five effort cost levels. Thus, each discounting task yielded five IPs (one for each cost level). The order of task presentation (i.e., delay, probability, and effort) was random, but all 30 choice trials within each discounting task were presented before the next task began. Within each discounting task, the order in which the five delays/probabilities/effort levels were presented was random, but all six choice trials for each level were presented before the next level began. On each choice trial, the participants considered two alternatives: (1) a smaller amount of money available immediately/for sure/with a low effort exertion requirement; or (2) a larger amount of money available after a delay/with some probability/with a high effort exertion. The location of the two alternatives was randomly assigned to the right and left of the computer screen from trial to trial. On the first-choice trial of each level, the larger amount of money was always MXN$3,000 (US$147 at that time), and the smaller amount was half that (MXN$1,500).

The following description uses the delay discounting task as an example to illustrate the procedure for all tasks. On the first-choice trial in the delay discounting task, a participant might be asked to choose between $3,000 after a delay (e.g., 2 months) or $1,500 received immediately. For the subsequent five trials, the amount of the immediate reward was adjusted based on participant choices following the algorithms provided by Du et al. ( 2002 ), whereas the delay to $3,000 was maintained at 2 months. This procedure was repeated until six choices were made for each delay level (i.e., each delay varied within a block of six trials). The amount of the immediate reward on the sixth and final choice at a specific delay level was coded as the IP. IPs represent the amount of the smaller, sooner reward that is considered subjectively equal to the amount of the delayed alternative. Immediate amounts were rounded to the nearest whole number to avoid a harder processing of the amounts (e.g., Kallai & Tzelgov, 2014 ).

Specific Task Instructions

For the delay discounting task, participants read the following instructions “In this task you will have to choose between immediate or delayed rewards, for example, would you rather earn $10 NOW or $20 AFTER a delay (1 day)?” We used different delays and amounts of money for the instructions to avoid a bias before the trials. The researcher asked to participants if they had any questions and pointed out the relevant portions of the instructions when answering questions. The five delay levels used were 2 weeks, 2 months, 6 months, 1 year, and 3 years.

Before beginning the probability discounting task, participants read instructions clarifying the concepts about certainty and chance of receiving a reward (Secretaría de Educación Pública, 2011 ), and had an opportunity to ask questions about these concepts. Then, the participants read the following instructions, “In this task you will have to choose between certain or risky rewards, for example, would you rather earn $10 for sure or $20 with 20% of chance?” The five probability levels were 90, 75, 50, 25, and 10% (0.11, 0.33, 1, 3, and 9 odds against receiving the reward).

For the effort discounting task, participants read the following instructions “Imagine walking at the same speed as you did the step test that you performed in the first session. In that test, your average of number of steps was [ insert average number of steps ] in 1 min. In this task you will choose between rewards for walking fewer steps or walking more steps, for example, would you rather earn $10 after walking 78 steps or $20 after walking 120 steps?” The fewer steps alternative always used the mean number of steps in 1 min taken by the specific participant, whereas the five more steps effort levels corresponded to the number of steps the participant would take in 10, 20, 60, 90, and 120 min at that walking speed. Thus, although the number of steps was individualized, the underlying durations of walking was the same for each participant and was used as the cost level for analyses of IPs and AUC calculations. The specific increments of minutes selected for the study was based on several health advertisements in Mexico used to encourage walking and reduce sedentarism. No walking duration was specified in the instructions to the participants. A nonzero number of steps was used for the fewer steps alternative to distinguish it from the immediate reward alternative in the delay discounting task (e.g., Mitchell, 1999 ).

Session Timing

Data collection was conducted between August and December 2020, when colleges closed due to the COVID-19 pandemic and students worked from home. Sessions took place from between 8 am –4 pm (UTC-6) for each participant. Participants were asked to return for the second experimental session 2 weeks after the first session, and for the third session, 2 weeks after the second session (e.g., Xu et al., 2013 ). The participants were contacted by text message to confirm and remind them of session appointments. Most of the participants met the interval appointment. Four participants completed the second and the third time points a couple of days later than the expected interval (2 or 4 days).

Data Analysis

The model fits, ICC, and all graphical analyses were conducted using R ® (R Core Team, 2022 ) and RStudio ® as the development environment (RStudio Team, 2020 ). The packages/functions used in the analysis are noted where relevant (Online Resource 1). Microsoft Excel® (version 16.16.27) was used to facilitate the calculation for AUC-values (Online Resource 2). We used JASP® (version 0.16.3), an open-source software, to perform the Bayesian analyses (e.g., Vincent, 2015 ).

Relative and Absolute Stability

Aim 1 of the present study was to assess the stability of choice patterns across the three time points for delay, probability, and effort discounting. To do this, we selected an aggregate measure of discounting that is neutral with respect to mathematical function: AUC (Myerson et al., 2001 ). The AUC is derived by summing the area of each trapezoid formed by two adjacent IPs and their corresponding levels of delay, probability (odds against), or effort: x 2 – x 1 [( y 1 + y 2 ) / 2]. The values x 2 and x 1 are the normalized levels of cost, and y 1 and y 2 are the normalized IPs at those levels. We also calculated the ordinal AUC (AUC ord ), because this improved normality and homoskedasticity (Borges et al., 2016 ), by replacing the numerical values of each cost level in the discounting task with integers from 1 through 5 (e.g., the first delay, 2 weeks, is recoded as “1,” the second delay, 2 months, as “2,”). The AUC ord ranges from 0 to 1, with lower AUC ord values indicating greater impatience , more risk aversion, or more effort aversion. Because the results using both traditional and ordinal AUC were nominally different, only analysis involving AUC ord are reported.

The relative stability was assessed in two ways. First, we calculated Bayesian Pearson product–moment correlations using the AUC ord values to examine the test–retest reliability between each time point for each task (Online Resource 3). Values provided in Taylor ( 1990 ) were used to interpret the strength of the correlation coefficient: r ≤ .35 are weak, ranges between .36 and .67 are moderate, ranges between .68 and .9 are strong, and ≥ .9 are very strong correlations. Values provided in Doorn et al. ( 2021 ) were used to interpret the Bayes Factor (BF 10 ) values; an index to quantify the weight of evidence for the competing null ( H 0 ) and alternative ( H 1 ) hypotheses. In these correlational analyses, the H 0 is that there is no relationship between the AUC ord for a given pair of time points, whereas the H 1 is that there is a relationship. BF 10 value ranges between 0.3 and 3.0 indicate weak evidence for either H 0 or H 1 , ranges between 3 and 10 indicate moderate evidence for the H 1 , and BF 10 > 10 indicates strong evidence for the H 1 .

Second, we standardized the AUC ord values by converting them to z -scores and used a two-way mixed-effects model with absolute agreement to calculate the ICC for theses standardized AUC ord values across time for each task (e.g., Martínez-Loredo et al., 2017 ). The ICC is suitable for testing test–retest reliability when there are more than two repeated measures over time (Koo & Li, 2016 ), allowing us to conducted a separate ICC for each task using all three time points for each participant. The ICCs were calculated with the irr package (Gamer et al., 2010 ). Values less than 0.50 indicate poor reliability over the three time points, values between 0.50 and 0.75 indicate moderate reliability, values between 0.75 and 0.90 indicate good reliability, and values greater than 0.90 indicate excellent reliability. The ICC has also been used to study the variability attributed to state- and trait-like differences in rats (e.g., Haynes et al., 2021 ), and we used these same ICC values to explore the extent of variability in delay, probability, and effort discounting that could be attributed to between- and within-subject differences. In this context, ICC > .5 indicate that the AUC ord differs more between-subjects than within-subjects, i.e., is relatively more consistent across time points. This is interpreted as reflecting trait-like differences. ICC < .5 indicate that AUC ord differs more within-subjects, i.e., is relatively less consistent across time points within-subjects. This is interpreted as reflecting state-like differences. Although ICCs are considered measures of trait and state variability (Merz & Roesch, 2011 ), they should be interpreted cautiously because ICCs do not allow us to identify the specific sources of between-subject variability or within-subject variability.

As supporting analyses, we conducted three separate Bayesian repeated measures ANOVAs with the IPs to explore the absolute stability across time points for each discounting task. That is, for each task, we performed a separate repeated measures ANOVA with the cost levels as the within-subject factor and the three time points as the between-subject factor. All ANOVAs incorporated random intercepts and slopes (Online Resource 4) and examined the weight of evidence for whether there was no effect of cost level or time point on the IPs ( H 0 ) or one or both of these variables affected the IPs ( H 1 ).

Evaluation of a Common Process for Discounting

Aim 2 was to determine the extent to which choice behavior on the three discounting tasks was similar and could be described by the same mathematical equation, potentially implying a common process. To address this aim we conducted two analyses. First, to Bayesian correlations between the AUC ord values for each task at each time point were used to explore whether participants responded in a similar way among the discounting tasks, i.e., someone who discounted delayed rewards to a large degree also discounted effort-requiring rewards to a large degree, etc. (Online Resource 3). We used the same criteria to evaluate the evidence for the H 0 and H 1 hypotheses as used in the relative stability analyses described earlier. Strong positive or strong negative correlations were viewed as consistent with a common process (Johnson et al., 2020 ). On the other hand, weak or moderate positive or negative correlations between the AUC ord values for each pair of tasks were viewed as consistent with the operation of different choice processes.

Second, we examined whether the same mathematical function could describe delay, probability, and effort discounting over time. For each task and each time point, we used the nlmrt nonlinear regression package (Nash, 2016 ) to fit the three discounting models (Eqs. 1 , 2 , and 3 ) to the median IPs. We used the Second-order Akaike Information Criteria (AICc) for model comparisons because accounts for the best-fitting model for small sample sizes. We used the AICc differences (Δ i AICc) for fit comparisons and ranking of candidate models (Burnham & Anderson, 2004 ):

where AIC i is the AICc for the i th model and AICc min in the minimum of the AICc among all the models. Models differing from the AICc min model by ≤ 2 have substantial support, those for which 4 ≤ Δ i ≤ 7 have less support, and models having Δ i > 10 have essentially no support. Thus, the best model has Δ 𝑖 ≡ Δ 𝑚𝑖𝑛 ≡ 0. These guidelines have similar counterparts in the Bayesian literature (Raftery, 1995 ).

The relative stability was assessed first by examining the Bayesian test–retest correlations of AUC ord values on pairs of the time points for each task. Figure 1 displays the scatterplots of pairs of time points for each of the three tasks. All correlations were positive. The magnitude of correlations between AUC ord values for delay discounting were moderate (range in r = .52–.59), as well as for probability discounting (range in r = .38–.61), whereas moderate and strong correlations were observed for effort discounting (range in r = .43–.73). Bayesian statistics indicated that there was moderate-to-strong evidence for the alternative hypothesis ( H 1 ) that there was a relationship between pairs of time points in delay discounting (BF 10 ≥ 5.33–14.96). Evidence for a relationship was ranged from weak-to-strong for probability discounting (BF 10 ≥ 1.18–24.12) and effort discounting (BF 10 ≥ 1.82–393.88). Time point pairs for which the evidence was weakest or strongest varied. That is, it was not the case that correlations were higher and relationships were strongest for consecutive time points, with the time point 1 versus time point 3 showing the least stability. This observation was supported by the ICC analyses, which considered all three time points for each discounting task. These analyses indicated moderate stability of choice patterns for each task. The ICCs for delay, probability, and effort discounting were 0.56 (95% CI [0.31, 0.76]), 0.53 (95% CI [0.27, 0.73]), and 0.62 (95% CI [0.38, 0.79]), respectively. By multiplying the ICCs by 100%, the percentages can be used to examine the extent of variability attributed to trait- and state-like differences in AUC ord . The 56%, 53%, and 62% values for the delay, probability, and effort discounting tasks indicates that more of the variability in AUC ord is attributable to between-subject (trait-like) variability than within-subject (state-like) variability. The ICCs for delay and probability discounting are fairly similar, with a slightly highest ICC for effort discounting. This difference is attributable to the larger variability in the choice patterns between-subjects for the effort discounting task. Figure 2 displays the AUC ord values for the three discounting tasks at each time point.

figure 1

Test–Retest Correlations with 95% Confidence Intervals. Note. Circles represent the AUC ord values for individuals. The left column shows time points 1 and 2, the middle column shows time points 2 and 3, and the right column shows time points 1 and 3. The top row depicts the test–retest data for delay discounting, the middle depicts the probability discounting time points, and the bottom row depicts the effort discounting data. Each test–retest graph includes its BF 10 value and the Pearson correlation coefficient

figure 2

AUC ord for Delay, Probability, and Effort Discounting across Time Points. Note. Box plots of AUC ord values for the participants who completed the discounting tasks across the three time points. The bottom and top of each box represent the 25 th and 75 th percentiles, the horizontal line within each box represents the group median. The vertical lines extending from the boxes represent the minimum and maximum values that are not outliers. The circle out of the whiskers is an outlier (delay discounting at Time 1). The symbols inside each box represent the individual AUC ord.

Evaluation of a Common Discounting Process

We conducted two analyses to determine the extent to which all discounting tasks shared a common process. First, we calculated the correlations between AUC ord between pairs of tasks at each time point (see Online Resource 3). Overall, there was strong evidence for the H 1 , which stated that there was a relationship between delay and probability discounting, at time point 2 (BF 10 = 14.13, 95% CI [0.19, 0.78]) and time point 3 (BF 10 = 37.98, 95% CI [0.27, 0.81]). Both correlations were moderate and positive r = .58 and r = .63, respectively. These results indicate that individuals who showed high AUC-delay values also tended to show high AUC-probability values. However, support for the alternative hypothesis was weak for all other comparisons (0.3 < BF 10 < 3).

We also examined whether the same mathematical function could describe delay, probability, and effort discounting across time. Overall, Table 1 shows that the hyperbolic (Eq. 1 ) and the hyperboloid (Eq. 2 ) functions had the lowest AICc values for the median IPs for all three time points, indicating superior fits to the data. There were no instances of the power function providing a comparable fit (Δ i AICc > 7). In other words, a convex mathematical form was clearly better able to describe the IPs obtained from the delay and probability tasks. The data were more mixed for the effort discounting IPs. The hyperboloid and power functions provided comparable fits for IPs at time points 1 and 2, whereas the hyperbolic and hyperboloid were indistinguishable for time point 3 IPs. However it is work noting that there was limited support for the hyperbolic at time points 1 and 2 (Δ i AICc ≤ 7), and the power function narrowly missed that cut off for time point 3 ((Δ i AICc = 7.38). In summary, there was not a unique model which best described IPs for all time points for each task. Figure 3 shows the three equations fitted to the median IPs for each task on all three time points. By visual inspection, the three tasks show a convex form of the discounting patterns.

figure 3

Model-Fitting to the Median Indifference Points across Time Points for Each Task. Note. Hyperbolic (first column), hyperboloid (middle column), and power function (last column) model fits to the median IPs among tasks and across time points. The top row shows the fits for the delay discounting task, the middle row shows those for the probability discounting task, and the bottom row, the effort discounting task

The Aim 1 of the present study was to assess the steady-state characteristics of choice patterns across three time points for delay, probability, and effort discounting tasks with hypothetical rewards in humans. Overall, our results replicated the previous findings of positive and moderate relative stability in delay discounting (e.g., Kirby, 2009 ; Martínez-Loredo et al., 2017 ), as well as in probability discounting (e.g., Matusiewicz et al., 2013 ; Ohmura et al., 2006 ). Our conclusions about stability are supported by the ICC scores, which indicated there was more variability between-subjects compared to within-subject differences across time points. The evidence for the absolute stability (see Online Resource 4) indicated a negligible role for time point on IPs for all three discounting tasks, and that the levels of delay, probability, and effort were the primary determinants of the IPs. This result for delay discounting was partially consistent with prior findings (e.g., Kirby, 2009 ; Xu et al., 2013 ). The use of the Bayesian approach to determine that there is only weak evidence to support an interaction between the IPs and the three time points in delay discounting extends the results from Xu et al. ( 2013 ) in young adults, who explored the absolute stability across three time points with the frequentist approach. This consistency is important because it should reduce the concerns about using different statistical approaches. It should also increase confidence in using Bayesian analyses, which provide information about the degree of support to both the null and the alternative hypothesis, rather than the dichotomous decisions process based on the p -values in frequentist analyses. To our knowledge, no previous study has explored the absolute and relative stability of effort discounting in humans. Thus, our study extends results indicating that the delay and probability are fairly stable to effort discounting, at least over the 4-week period of the current study. We suggest that future studies should explore longer periods.

As part of the evaluation process for our examination of stability we used the ICC scores. The data suggested that variability in AUC ord values for each of the discounting tasks was associated with trait-level rather than state-level differences. Unfortunately, this analysis does not permit us to identify the sources of between and within-subject variability. There are several potential factors that may contribute to the general incidence of variability across all tasks. Perhaps the most salient is that our study was conducted during the COVID-19 pandemic during the early phases when lockdowns and self-isolation occurred in Mexico City. Romanowich and Chen ( 2021 ) found low test–retest reliability in delay discounting immediately after the environmental disruption by COVID-19, suggesting that major environmental disruptions might negatively affect the stability of discounting measures. This may have occurred because the COVID-19 pandemic may have altered the value of rewards when individuals were socially isolated and less certain about the future (Romanowich & Chen, 2021 ). Thus, our stability estimates may be lower than would have been obtained during non-COVID times.

Our Aim 2 was to evaluate evidence that a common process underlies the three discounting tasks. The evidence examined included the correlations between AUC ord values for each task at each of the three time points, and similarities in the mathematical functions that best described the IPs. On balance, the positive correlations were larger between AUC ord values for delay and probability discounting than for any comparisons with effort discounting, with strong support for this conclusion from the Bayesian analyses. Thus, there appears to be some commonality in the processes underlying choices in the delay and probability tasks but not with the effort discounting task, despite the small positive correlations between its AUC ord values with those of the other tasks (Online Resource 3). These general conclusions were supported by our finding that the hyperbolic and hyperboloid functions showed the best-fitting for more instances to the delay and probability discounting tasks across time points, whereas the best fitting equations for IPs from the effort discounting task included the power function. The result for the delay and probability tasks is partially consistent with prior evidence (e.g., Ohmura et al., 2006 ). However, Ohmura et al. used R 2 values to compare the mathematical functions and drew conclusions based on which had the higher scores. Use of R 2 rather than AIC, AICc (used in our study) or the Bayesian Information Criterion (BIC) is an active area of discussion among researchers. For example, Johnson and Bickel ( 2008 ) warned about using R 2 values to compare delay discounting models as they suggest there is overfitting for models with two or more free parameters and correlations between R 2 and discounting parameters. It is to be hoped that future discussions will achieve a consensus about methods to identify the best-fitting model.

We found that the hyperboloid function fit the effort discounting task IPs well for two time points. This is inconsistent with prior evidence indicating the superiority of the power function (e.g., Białaszek et al., 2017 ) and our expectation that the shape of the effort discounting function would be more concave (i.e., power function) rather than convex. However, the visual inspection of IPs data and Δ i AICc values indicated that the effort discounting choice patterns were convex, similarly to those for delay and probability discounting. However, the shape of the effort discounting function seems to vary across studies. For example, Mitchell ( 1999 ) found a shallow effort discounting curve and a good fit to the hyperbolic model. Białaszek et al. ( 2017 ) also found a similar shallow pattern, but the power function fitted their data better rather than any tested convex models. In both studies, effort was defined as exertion of force using a hand dynamometer (i.e., Maximum Voluntary Contraction). In contrast, when effort is defined as number of activities or the number of responses during a specific period, the shape of the curve seems more convex, as in our study, and in a study reported in Ostaszewski et al. ( 2013 ). We suggest that future research should explore whether the mathematical form of effort discounting data depends on the definition of effort requirements (i.e., force vs. number of responses or duration of responding).

Despite the clear and intriguing differences in choices from the discounting tasks revealed in this study, we also identify some study limitations. First, our study used hypothetical cost levels and reward outcomes for all tasks. There is some data suggesting the use of hypotheticals does not result in different qualitative results (Lawyer et al., 2011 ; Madden et al., 2003 ). However, this is not the case for all studies. For example, Hinvest and Anderson ( 2010 ) reported that the use of real outcomes was associated with significantly decreased impulsive choices in delay discounting compared to hypothetical outcomes. Also, Matusiewicz et al. ( 2013 ) reported inconsistent results about the stability of delay discounting with hypothetical and potentially real outcomes. Possibly consistent with this hypothetical–real concern is that the ICC score was higher for the effort discounting task, reflecting more variability between-subjects. Although speculative, this may be attributable to the pre-exposure to the effort requirements during the calibration task (Eisenberger, 1992 ), which could have altered participants’ ability to imagine the hypothetical effort requirements. The larger between-subject variability in ICC may reflect the individual differences in performance on the calibration task, which we conducted to ensure the number of steps in specific time periods were equated for each participant. The analysis revealed that there was relatively weak evidence for relationship between the AUC ord for the effort discounting task at any time point and the mean number of steps taken during that task (Online Resource 5). Future studies should compare performance on effort discounting tasks when participants are exposed to a calibration task that allows them to experience the effort requirements compared with wholly hypothetical or real requirements and outcomes.

A second limitation related to the use of a nonzero effort requirement for the small reward, compared to the zero delay (immediate smaller reward) and zero odd-against/probability =1 (smaller reward for sure) in the other tasks. Prior research has indicated that nonzero delays in delay discounting tasks do not alter the discounting function (e.g., Green et al., 2005 ; Mitchell & Wilson, 2012 ), but it is unclear whether this is the case with probability discounting. Thus, although we do not rule out the possibility that the current manipulation could influence effort discounting differently from the other two tasks, we could not identify any data that would indicate the relative relationships across time points or between tasks would be systematically disrupted. However, future research to evaluate the effects of nonzero values would be useful.

A final limitation associated with the effort discounting task was that we created the different levels of effort by multiplying the steps completed in 1 min, assessed the calibration task, by specific numbers of minutes (10, 20, 60, 90, and 120 min). In the natural environment, walking pace might be expected to vary limiting our ability to translate to the number of steps to match the duration of walking. Individuals’ recognition of this change in walking pace might have contributed to the high levels of between subject variability observed in this task. Again, we suggest future research exploring the effects of real versus hypothetical requirements would be useful, especially in the realm of physical effort discounting.

In conclusion, although we recognize that the study has some limitations, the data indicate that an individual’s effort discounting is stable and reliable over approximately a month, similarly to delay and probability discounting. Further, the choices made in the three discounting tasks are only modestly similar, which supports the conclusion that the choice process in effort discounting is dissimilar to that of delay and probability discounting, and that discounting processes are a function of cost type (Białaszek et al., 2019 ).

Data Availability

The datasets generated during and/or analyzed during the current study, as well as the code in R® for conducting the analyses, are hosted on Open Science Framework: https://osf.io/3bkjq/?view_only=ee27660653b244e2be28c498a99a7918 and available from the corresponding author.

American Psychological Association (2020). Guidance on psychological tele-assessment during the COVID-19 crisis . Retrieved April 15, 2020, from https://www.apaservices.org/practice/reimbursement/health-codes/testing/tele-assessment-covid-19

Amlung, A., Vedelago, L., Acker, J., Balodis, I., & MacKillop, J. (2017). Steep delay discounting and addictive behavior: A meta-analysis of continuous associations. Addiction, 112 (1), 51–62. https://doi.org/10.1111/add.13535

Article   PubMed   Google Scholar  

Anokhin, A. P., Golosheykin, S., & Mulligan, R. C. (2015). Long-term test-retest reliability of delayed reward discounting in adolescents. Behavioural Processes, 111 , 55–59. https://doi.org/10.1016/j.beproc.2014.11.008

Białaszek, W., Marcowski, P., & Ostaszewski, P. (2017). Physical and cognitive effort discounting across different reward magnitudes: Tests of discounting models. PLoS One, 12 (7), 1–25. https://doi.org/10.1371/journal.pone.0182353

Article   Google Scholar  

Białaszek, W., Ostaszewski, P., Green, L., & Myerson, J. (2019). On four types of devaluation of outcomes due to their costs: Delay, probability, effort, and social discounting. The Psychological Record, 69 , 415–424. https://doi.org/10.1007/s40732-019-00340-x

Article   PubMed   PubMed Central   Google Scholar  

Bickel, W. K., Jarmolowicz, D. P., Mueller, E. T., Koffarnus, M. N., & Gatchalian, K. M. (2012). Excessive discounting of delayed reinforcers as a trans-disease process contributing to addiction and other disease-related vulnerabilities: Emerging evidence. Pharmacology & Therapeutics, 134 (3), 287–297. https://doi.org/10.1016/j.pharmthera.2012.02.004

Borges, A. M., Kuang, J., Milhorn, H., & Yi, R. (2016). An alternative approach to calculating area-under-the-curve (AUC) in delay discounting research. Journal of the Experimental Analysis of Behavior, 106 , 145–155. https://doi.org/10.1002/jeab.219

Burnham, K. P., & Anderson, D. R. (2004). Multimodel inference: Understanding AIC and BIC in model selection. Sociological Methods & Research, 33 (2), 261–304. https://doi.org/10.1177/0049124104268644

Doorn, J., van den Bergh, D., Böhm, U., Dablander, F., Derks, K., Draws, T., Etz, A., Evans, N. J., Gronau, Q. F., Haaf, J. M., Hinne, M., Kucharský, Ś., Ly, A., Marsman, M., Matzke, D., Gupta, A. R. K. N., Sarafoglou, A., Stefan, A., Voekel, J. G., & Wagenmakers, E.-J. (2021). The JASP guidelines for conducting and reporting a Bayesian analysis. Psychonomic Bulletin & Review, 28 , 813–826. https://doi.org/10.3758/s13423-020-01798-5

Du, W., Green, L., & Myerson, J. (2002). Cross-cultural comparisons of discounting delayed and probabilistic rewards. The Psychological Record, 52 , 479–492. https://doi.org/10.1007/BF03395199

Eisenberger, R. (1992). Learned industriousness. Psychological Review, 2 , 248–267. https://doi.org/10.1037/0033-295X.99.2.248

Franck, C. T., Koffarnus, M. N., House, L. L., & Bickel, W. K. (2015). Accurate characterization of delay discounting: A multiple model approach using approximate Bayesian model selection and a unified discounting measure. Journal of the Experimental Analysis of Behavior, 103 (1), 218–233. https://doi.org/10.1002/jeab.128

Gamer, M., Lemon, J., Fellows, I., & Sing, P. (2010). Irr: Various coefficients of interrater reliability and agreement (version 0.83) [software]. Retrieved March 09, 2022, from http://CRAN.R-project.org/package=irr

Green, L., & Myerson, J. (2013). How many impulsivities? A discounting perspective. Journal of the Experimental Analysis of Behavior, 99 (1), 3–13. https://doi.org/10.1002/jeab.1

Green, L., Myerson, J., & Macaux, E. W. (2005). Temporal discounting when the choice is between two delayed rewards. Journal of Experimental Psychology: Learning, Memory, and Cognition, 31 (5), 1121–1133. https://doi.org/10.1037/0278-7393.31.5.1121

Haynes, M. J., Galizio, A., Frye, C. C. J., Towse, C. C., Morrissey, K. N., Serang, S., & Odum, A. L. (2021). Discounting of food and water in rats shows trait- and state-like characteristics. Journal of the Experimental Analysis of Behavior, 115 (2), 495–509. https://doi.org/10.1002/jeab.677

Henry-Edwards, S., Humeniuk, R., Ali, R., Poznyak, V., & Monteiro, M. (2003). The alcohol, smoking and substance involvement screening test (ASSIST): Guidelines for use in primary care (draft version 1.1 for field testing). World health Organization. Retrieved October 21, 2020, from http://www.who.int/substance_abuse/activities/en/Draft_The_ASSIST_Guidelines.pdf

Hinvest, N. S., & Anderson, I. M. (2010). The effects of real versus hypothetical reward on delay and probability discounting. Quarterly Journal of Experimental Psychology, 63 (6), 1072–1084. https://doi.org/10.1080/17470210903276350

Johnson, K. L., Bixter, M. T., & Luhmann, C. C. (2020). Delay discounting and risky choice: Meta-analytic evidence regarding single-process theories. Judgment & Decision Making, 15 (3), 381–400.

Johnson, M. W., & Bickel, W. K. (2008). An algorithm for identifying nonsystematic delay-discounting data. Experimental & Clinical Psychopharmacology, 16 , 264–274. https://doi.org/10.1037/1064-1297.16.3.264

Kallai, A. Y., & Tzelgov, J. (2014). Decimals are not processed automatically, not even as being smaller than one. Journal of Experimental Psychology: Learning, Memory, & Cognition, 40 (4), 962–975. https://doi.org/10.1037/a0035782

Killeen, P. R. (2022). Variations on a theme by Rachlin: Probability discounting. Journal of the Experimental Analysis of Behavior, 119 (1), 140–155Advance online publication. https://doi.org/10.1002/jeab.817

Kirby, K. N. (2009). One-year temporal stability of delay-discount rates. Psychonomic Bulletin & Review, 16 (3), 457–462. https://doi.org/10.3758/PBR.16.3.457

Koo, T. K., & Li, M. Y. (2016). A guideline of selecting and reporting intraclass correlation coefficients for reliability research. Journal of Chiropractic Medicine, 15 , 155–163. https://doi.org/10.1016/j.jcm.2016.02.012

Lawyer, S. R., Schoepflin, F., Green, R., & Jenks, C. (2011). Discounting of hypothetical and potentially real outcomes in nicotine-dependent and nondependent samples. Experimental & Clinical Psychopharmacology, 19 (4), 263–274. https://doi.org/10.1037/a0024141

Linaje, M., & Lucio, E. (2013). Validez y confiabilidad del ASSIST en una muestra de estudiantes mexicanos. Revista Española de Drogodependencia, 38 (1), 37–51.

Google Scholar  

Madden, G. J., Begotka, A. M., Raiff, B. R., & Kastern, L. L. (2003). Delay discounting of real and hypothetical rewards. Experimental & Clinical Psychopharmacology, 11 (2), 139–145. https://doi.org/10.1037/1064-1297.11.2.139

Martínez-Loredo, V., Fernández-Hermida, J. E., Carballo, J. L., & Fernández-Artamendi, S. (2017). Long-term reliability and stability of behavioral measures among adolescents: The delay discounting and Stroop tasks. Journal of Adolescence, 58 , 33–39. https://doi.org/10.1016/j.adolescence.2017.05.003

Mathôt, S., Schreij, D., & Theeuwes, J. (2012). OpenSesame: An open-source, graphical experiment builder for the social sciences. Behavior Research Methods, 44 (2), 314–324. https://doi.org/10.3758/s13428-011-0168-7

Matusiewicz, A. K., Carter, A. E., & Landes, R. D. (2013). Statistical equivalence and test-retest reliability of delay and probability discounting using real and hypothetical rewards. Behavioural Processes, 100 , 116–122. https://doi.org/10.1016/j.beproc.2013.07.019

Mazur, J. E. (1987). An adjusting procedure for studying delayed reinforcement. In M. L. Commons, J. E. Mazur, J. A. Nevin, & H. Rachlin (Eds.), Quantitative analysis of behavior: Vol. 5: The effect of delay and intervening events on reinforcement value (pp. 55–73) . Lawrence Erlbaum Associates.

Merz, E. L., & Roesch, S. C. (2011). Modeling trait and state variation using multilevel factor analysis with PANAS daily diary data. Journal of Research in Personality, 45 (1), 2–9. https://doi.org/10.1016/j.jrp.2010.11.003

Mitchell, S. H. (1999). Measures of impulsivity in cigarette smokers and nonsmokers. Psychopharmacology, 146 (4), 455–464. https://doi.org/10.1007/PL00005491

Mitchell, S. H. (2004). Effects of short-term nicotine deprivation on decision-making: Delay, uncertainty and effort discounting. Nicotine & Tobacco Research, 6 (5), 819–828. https://doi.org/10.1080/14622200412331296002

Mitchell, S. H., & Wilson, V. B. (2012). Differences in delay discounting between smokers and nonsmokers remain when both rewards are delayed. Psychopharmacology, 219 , 549–562. https://doi.org/10.1007/s00213-011-2521-z

Myerson, J., Green, L., & Warusawitharana, M. (2001). Area under the curve as a measure of discounting. Journal of the Experimental Analysis of Behavior, 76 (2), 235–243. https://doi.org/10.1901/jeab.2001.76-235

Nash, J. C. (2016). Nlmrt: Functions for nonlinear least squares solutions. R package version 1 . Retrieved February 19, 2021, from https://cran.r-project.org/web/packages/nlmrt/nlmrt.pdf

Odum, A. L. (2011). Delay discounting: I'm a k , you're a k . Journal of the Experimental Analysis of Behavior, 96 (3), 427–439. https://doi.org/10.1901/jeab.2011.96-423

Odum, A. L., Becker, R. J., Haynes, J. M., Galizio, A., Frye, C. C. J., Downey, H., Friedel, J. E., & Perez, D. M. (2020). Delay discounting of different outcomes: Review and theory. Journal of the Experimental Analysis of Behavior, 113 (3), 657–679. https://doi.org/10.1002/jeab.589

Ohmura, Y., Takahashi, T., Kitamura, N., & Wehr, P. (2006). Three-month stability of delay and probability discounting measures. Experimental & Clinical Psychopharmacology, 14 , 318–328. https://doi.org/10.1037/1064-1297.14.3.318

Ostaszewski, P., Bąbel, P., & Swebodzin ́ski, B. (2013). Physical and cognitive effort discounting of hypothetical monetary rewards. Japanese Psychological Research, 55 (4), 329–337. https://doi.org/10.1111/jpr.12019

Peters, J., & Büchel, C. (2009). Overlapping and distinct neural systems code for subjective value during intertemporal and risky decision making. Journal of Neuroscience, 29 (50), 15727–15734. https://doi.org/10.1523/JNEUROSCI.3489-09.2009

Pinkston, J. W., & Libman, B. M. (2017). Aversive functions of response effort: Fact or artifact? Journal of the Experimental Analysis of Behavior, 108 (1), 73–96. https://doi.org/10.1002/jeab.264

R Core Team. (2022). R: A language and environment for statistical computing. R Foundation for Statistical Computing. Retrieved February 19, 2021, from https://www.R-project.org/

Rachlin, H. (2006). Notes of discounting. Journal of the Experimental Analysis of Behavior, 85 (3), 425–435. https://doi.org/10.1901/jeab.2006.85-05

Rachlin, H., Logue, A. W., Gibbon, J., & Frankel, M. (1986). Cognition and behavior in studies of choice. Psychological Review, 93 (1), 33–45. https://doi.org/10.1037/0033-295X.93.1.33

Rachlin, H., Raineri, A., & Cross, D. (1991). Subjective probability and delay. Journal of the Experimental Analysis of Behavior, 55 (2), 233–244. https://doi.org/10.1901/jeab.1991.55-233

Raftery, A. E. (1995). Bayesian model selection in social research (with discussion). Sociological Methodology, 25 , 111–195.

Romanowich, P., & Chen, Q. (2021). The effects of COVID-19 on test-retest reliability in a behavioral measure for impulsivity. International Journal of Social Research Methodology, 893–903 . https://doi.org/10.1080/13645579.2021.1935821

RStudio Team. (2020). RStudio: Integrated development for R . RStudio. Retrieved February 19, 2021, from http://www.rstudio.com/

Secretaría de Educación Pública. (2011). Aprendizaje y enseñanza de las matemáticas escolares . Casos y prespectivas.

Sidman, M. (1960). Tactics of scientific research: Evaluating experimental data in psychology . Authors Cooperative.

Smits, R. R., Stein, J. S., Johnson, P. S., Odum, A. L., & Madden, G. J. (2013). Test-retest reliability and construct validity of the experiential discounting task. Experimental & Clinical Psychopharmacology, 21 (2), 155–163. https://doi.org/10.1037/a0031725

Sofis, M. J., Carrillo, A., & Jarmolowicz, D. P. (2017). Maintained physical activity induced changes in delay discounting. Behavior Modification, 41 (4), 499–528. https://doi.org/10.1177/0145445516685047

Taylor, R. (1990). Interpretation of the correlation coefficient: A basic review. Journal of Diagnostic Medical Sonography, 1 (6), 35–39. https://doi.org/10.1177/875647939000600106

Vincent, B. T. (2015). Hierarchical Bayesian estimation and hypothesis testing for delay discounting tasks. Behavioral Research Methods, 48 (4), 1608–1620. https://doi.org/10.3758/s13428-015-0672-2

Xu, S., Korczykowski, M., Zhu, S., & Rao, H. (2013). Assessment of risk-taking and impulsive behaviors: A comparison between three tasks. Social Behavior & Personality, 41 (3), 477–486. https://doi.org/10.2224/sbp.2013.41.3.477

Young, M. E. (2017). Discounting: A practical guide to multilevel analysis of indifference data. Journal of the Experimental Analysis of Behavior, 108 (1), 97–112. https://doi.org/10.1002/jeab.265

Young, M. E. (2019). Bayesian data analysis as a tool for behavior analysts. Journal of the Experimental Analysis of Behavior, 111 (2), 225–238. https://doi.org/10.1002/jeab.512

Download references

Acknowledgments

The authors are grateful to Carlos Rosillo, Elizabeth Escobedo, and Noel Escobedo for their assistance in recruiting participants.

This research was supported by a Graduate Research Grant from the National Council of Science and Technology (CONACyT) awarded to GGE (CVU: 660452, Grant: 20656), by the PAPIIT project IN305120 from UNAM to SMC, and it was partially supported by funding to SHM from the National Institutes of Health (R21 MH121073). This research was part of the Ph.D. dissertation for GGE. Portions of this manuscript were presented at the annual meeting of the Mexican Society for Behavior Analysis in October 2022.

Author information

Authors and affiliations.

Faculty of Psychology, National Autonomous University of Mexico, Universidad Avenue 3004, Coyoacán, 04510, Mexico City, Mexico

Gisel G. Escobar, Silvia Morales-Chainé & Carlos Santoyo

Faculty of Higher Studies Iztacala, National Autonomous University of Mexico, State of Mexico, Mexico

Gisel G. Escobar

Utah State University, Logan, UT, USA

Jeremy M. Haynes

Oregon Health & Science University, Portland, OR, USA

Suzanne H. Mitchell

You can also search for this author in PubMed   Google Scholar

Contributions

GGE created the discounting tasks and designed the study with help from SMC, SHM, and CS. Besides, GGE collected the data, completed the data analyses, and created the figures and tables with help from SHM. The R code for conducting most of analyses was written by GGE, whereas the code for conducting the ICC and AICc analysis was written by JMH, and he assisted in interpreting the results generated. GGE wrote the first draft of the manuscript, whereas SHM and JMH provided edits as native English speakers. All authors read and contributed to the introduction, results, and discussion sections.

Corresponding authors

Correspondence to Gisel G. Escobar or Suzanne H. Mitchell .

Ethics declarations

The current manuscript has not been previously published and has not been and will not be submitted elsewhere during the review process. The current manuscript has been approved by all co-authors.

Ethical Approval

The methodology for this study was approved by the Graduate Ethics committee in Psychology of the National Autonomous University of Mexico.

Consent to Participate

All individuals who participated in this study provided informed consent through individual remote sessions.

Consent for Publication

All individuals who participated in this study provided consent for publication of their results with full anonymity, through individual remote sessions.

Conflict of Interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Escobar, G.G., Morales-Chainé, S., Haynes, J.M. et al. Moderate Stability among Delay, Probability, and Effort Discounting in Humans. Psychol Rec 73 , 149–162 (2023). https://doi.org/10.1007/s40732-023-00537-1

Download citation

Accepted : 30 January 2023

Published : 15 February 2023

Issue Date : June 2023

DOI : https://doi.org/10.1007/s40732-023-00537-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

  • Delay discounting
  • Effort discounting
  • Probability discounting
  • Mathematical functions
  • Find a journal
  • Publish with us
  • Track your research
  • Reference Manager
  • Simple TEXT file

People also looked at

Original research article, loss of sensitivity to rewards by dopamine neurons may underlie age-related increased probability discounting.

probability discounting task

  • 1 Department of Psychology, University of Washington, Seattle, WA, United States
  • 2 Laboratory of Behavioral Neuroscience, Neurocognitive Aging Section, National Institute on Aging, National Institutes of Health, Baltimore, MD, United States
  • 3 Program in Neuroscience, University of Washington, Seattle, WA, United States

Normative aging is known to affect how decisions are made in risky situations. Although important individual variability exists, on average, aging is accompanied by greater risk aversion. Here the behavioral and neural mechanisms of greater risk aversion were examined in young and old rats trained on an instrumental probability discounting task. Consistent with the literature, old rats showed greater discounting of reward value when the probability of obtaining rewards dropped below 100%. Behaviorally, reward magnitude discrimination was the same between young and old rats, and yet these same rats exhibited reduced sensitivity to positive, but not negative, choice outcomes. The latter behavioral result was congruent with additional findings that the aged ventral tegmental neurons (including dopamine cells) were less responsive to rewards when compared to the same cell types recorded from young animals. In sum, it appears that reduced responses of dopamine neurons to rewards contribute to aging-related changes in risky decisions.

Introduction

According to the world health organization (WHO), in almost every country, the proportion of people aged over 60 years is growing faster than any other age group as a result of both longer life expectancy and declining fertility rates. An accompaniment of aging is, changes in behavior, that is governed by underlying changes in brain physiology. Understanding how the brain normally ages and affects behavior will allow us to adapt to the new population landscape. Normal cognitive aging does not usually result in great losses in the number of neurons but is often accompanied by changes in decision making and memory functions even in the absence of neurodegenerative disease ( Rapp and Gallagher, 1996 ; Morrison and Hof, 1997 ; Rapp et al., 2002 ; Gallagher et al., 2003 ; Rosenzweig and Barnes, 2003 ; Fletcher and Rapp, 2012 ; Orsini et al., 2019 ). Thus, an important avenue of current research is to understand how behavior changes with age, and the mechanisms mediating normative brain aging. While studies exploiting the power of in vivo neuroimaging in humans have yielded important insights, the approaches available in animal models are necessary for a comprehensive cell biological account of the neural basis of cognitive aging. The latter is especially relevant since similar behavioral and cognitive patterns of change are observed in old rodents and primates ( Gallagher et al., 2011 ).

Neurocognitive processes that are mediated by the hippocampus, such as declarative memory and pattern separation, or by the dorsolateral prefrontal cortex (PFC), such as cognitive flexibility and working memory, are among the most vulnerable in aging ( Barense et al., 2002 ; Dumitriu et al., 2010 ; Morrison and Baxter, 2012 ). It is thought that decline in separate neural systems occurs independently from each other; thus, declines in PFC function do not necessarily imply deficits in hippocampal function and vice versa ( Moore et al., 2009 ; Gallagher et al., 2011 ; Fletcher and Rapp, 2012 ). We also know that cost-benefit decision making is altered with age, albeit to what extent is less clear. In humans, older individuals tend to choose sure gains and avoid sure losses ( Mather et al., 2012 ), avoid risky choices ( Lee et al., 2008 ), and gains and losses are encoded asymmetrically ( Samanez-Larkin et al., 2007 ). However, aging has also been associated with an increase in making risky choices in some rats and humans, and risk-aversion in others ( Mata et al., 2011 ; Gilbert et al., 2012 ; Samson et al., 2015 ). The findings of individual differences in probability discounting in aged rats are consistent with evidence for individual differences in aged rat, monkey, and human performance in other cognitive domains ( Gallagher et al., 1993 , 2011 ; Barense et al., 2002 ; Schoenbaum et al., 2006 ; Bizon et al., 2009 ; Morrison and Baxter, 2012 ). Probability discounting is the change in the subjective value of a reward based on the probability of receiving it, e.g., a reward has reduced value and is “discounted” if the probability of receiving it is low. On average, however, older individuals (animals and humans) tend to display more risk-aversion during risk-based decision making. Therefore, the goal of the current study was to investigate in more detail how normal aging affects risk-based decision making. We selected a rodent model that features increased individual differences in memory with aging, providing a setting for asking whether these effects are coupled with changes in decision making, potentially pointing to shared underlying substrates. Our probability discounting task tested rats’ preference for a lever that led to a large reward with varying probability (the risky option) or a lever that lead to a small reward 100% of the time (the certain option). The probabilities associated with the risky lever were 100%, 50%, 25%, and 12.5%. The central question was whether aged rats show a preference for the small, certain reward over the risky one, even when the outcome would be better if the risky option was selected. Like aged humans tested on similar tasks, we expected aged rats to display significantly reduced choice of the large risky reward, or increased discounting behavior, compared to young controls.

Also of interest was the question of how aging may influence risk-based decision making because we know that normal aging does not affect all brain systems equally as some systems are more prone to functional decline over time. For example, dopamine function declines across the life-span, and this decline has been correlated with age-related cognitive deficits ( Bäckman et al., 2006 , 2010 ; Rollo, 2009 ). Midbrain dopamine (DA) neurons of the ventral tegmental area (VTA) appear highly evolved to have strong responses to rewards and biologically salient events. They exhibit burst-like activity in response to primary rewards such as food and water ( Wise, 2004 ; Fields et al., 2007 ). It is thought that these specialized responses to salient and rewarding events are important for the learning and selection of appropriate behavioral responses. Therefore, DA functional decline over the lifespan could alter cognition reliant on DA. Indeed, evidence suggests that an age-related decrease in DA release is linked to poorer working memory function and perceptual speed ( Bäckman et al., 2010 ). Additionally, DA hypofunction has been implicated in decreased sensitivity to changes in reward magnitudes ( Dreher et al., 2008 ). However, it is not known if the functional decline in DA function is linked to altered risk-based decision making observed in old age. We know that midbrain DA neurons encode the uncertainty of probabilistic rewards ( Fiorillo et al., 2003 ) and that alteration of DA signaling alters risky choice ( St. Onge and Floresco, 2009 ; St. Onge et al., 2010 ). Therefore, another goal of the current project was to investigate how reward processing may change in the aged brain by recording neural activity from DA neurons of the VTA.

Materials and Methods

Fifteen 6–9 months and 19 25–26-month-old male Long-Evans rats were received from the Laboratory of Behavioral Neuroscience at the National Institute on Aging, National Institutes of Health (Baltimore, MD, USA). All rats are originally from the Charles River facility (Raleigh, NC, USA). Twenty-eight rats (12 young and 16 aged) were used in the behavioral experiments and six rats (three young and three aged) were used for the DA recording experiment. Upon arrival, rats were housed individually in Plexiglas cages and were maintained on a 12 h light/dark cycle (lights on at 7:00 AM). All behavioral experiments were performed during the light phase of the cycle. Each rat was allowed access to water ad libitum and food-deprived to 80–85% of its ad libitum feeding weight. In some cases of extreme obesity, rats were further restricted to 75% of their free-feeding weight with oversight from the veterinary staff. Rats were handled and weighed daily. Rats that developed any health concerns were evaluated by veterinary staff and excluded from the experiment immediately if appropriate. All animal care and use were conducted in accordance with the University of Washington’s and National Institute of Aging’s Institutional Animal Care and Use Committee guidelines.

Behavioral Testing Procedures

Water maze testing.

Prior to arrival at the University of Washington (UW), standard water maze training (for detailed description, see Gallagher et al., 1993 ) was conducted for all rats. Briefly, training took place across 8 consecutive days, three trials per day (each using a 90 s cutoff), with a 60-s intertrial interval. Every sixth trial was a probe test in which the escape platform was initially retracted to the bottom of the maze for 30 s and then made accessible to permit escape. The performance was evaluated according to learning index scores (LISs), as described previously ( Gallagher et al., 1993 ). Briefly, the LISs were calculated as the weighted average proximity (in centimeters) to the hidden escape location across probe trials. This measure is optimized for identifying reliable individual differences in memory and was used to classify aged animals as either aged unimpaired (AU), or aged impaired (AI), using criteria validated in earlier research ( Gallagher et al., 1993 ). By this measure, lower values reflect closer proximity to the escape location and better spatial learning. The day after completing the spatial protocol, rats were tested on a one-session, hippocampus-independent cued version of the Morris water maze. Six trials were given from multiple start locations, 60-s maximum trial length. No animals that failed cue training were included in the present experiment.

Probability-Discounting Task

All UW behavioral testing and neural recordings took place in Med PC operant chambers (30.5 × 24 × 21 cm; Med-Associates, St. Albans, VT, USA). Operant chambers were enclosed in sound-attenuating boxes with a fan to provide ventilation. While standard chambers are equipped with a metal grid floor, custom cut Plexiglas covered the floors for all rats to provide aged animals with a comfortable standing surface. The chamber was illuminated by a houselight on one side of the chamber. On the opposite wall from the houselight was a food cup in the center of one wall and two retractable metal levers on either side of the food cup. A single light was located above each lever. For recording experiments, the food receptacle was custom built to extend from the wall to allow implanted animals to easily obtain the food reward. Additionally, the metal levers were coated in plastic to dampen electrical noise during neural recording experiments. Food rewards were 45 mg sugar pellets (Bioserv) that were dispensed from a pellet dispenser. Nose-pokes, or when the animals’ nose-first approached the food cup, were recorded via two infrared photo beams on either side of the food cup.

Pre-training

Animals underwent identical pre-training procedures for both experiments. Training protocols were adapted from St. Onge and Floresco (2009) . Once food restricted to 80–85% of their free-feeding weight, rats were given sugar pellets in their home cage to avoid neophobia for 1–2 days prior to training in the operant chamber. Then, rats were randomly assigned to one of the operant chambers that would then stay consistent throughout the training. The first day of pretraining consisted of lever-press training at a fixed-ratio 1 schedule. Crushed sugar pellets were placed on the lever to encourage the rat to press the lever until the rat started pressing reliably. Sugar pellets were also placed in the food cup. Rats were trained on the FR1 schedule of lever pressing for a single lever until they reached the criteria of 60 lever presses within 30 min after which they switched to the same training protocol on the other lever. The first lever presentation was counterbalanced between rats. Then, rats trained on a simplified version of the probability discounting task. This task consisted of 90 trials during which one of the two levers was extended with the house light illuminated (left and the right lever was presented once for every two trials and the order of presentation was randomized within a pair of trials). A lever press during one of these trials resulted in the delivery of a sugar pellet with a 50% probability. Once pressed, the lever retracted, the house light turned off and another trial was initiated after an inter-trial interval. If the rat failed to press the lever within 10 s of lever presentation, the lever retracted, the house light turned off, and the trial was recorded as an omission. Every trial was 40 s long regardless of the behavior of the animal. This procedure was used to allow the rats to learn about the probabilistic nature of lever pressing. Rats were trained on this version of the task until they reached criterion, defined as an omission of 10 or fewer trials per session. Days to reach criterion for each task varied from rat to rat (see “Results” section).

Probability-Discounting Task: Behavioral Training

The task was based on previous studies investigating probability discounting ( Cardinal and Howes, 2005 ; St. Onge and Floresco, 2009 ). After pretraining, rats performed 48-min sessions consisting of 72 trials 5 days a week. Each session was separated into four different probability blocks of 18 trials. A trial began every 40 s with the illumination of the houselight and, 3 s later, insertion of one or both levers into the chamber. One lever was designated the Large/Risky lever, the other the Small/Certain lever, which remained consistent throughout training (counterbalanced left/right). Half of the rats were assigned their preferred lever as the risky lever and the other half were assigned their non-preferred lever as risky. If the rat did not respond by pressing a lever within 10 s of lever presentation, the lights turned off, the levers were retracted, and the trial was scored as an omission. When a lever was chosen, both levers retracted. Pressing a certain lever led to the delivery of one sugar pellet 100% of the time. Pressing the “risky” lever led to four sugar pellets with varying probability across trial blocks. The probability blocks were presented in descending order. The first block was associated with a 100% probability of receiving the large reward when the risky lever was pressed, then 50%, then 25%, and finally 12.5%. When food was delivered, the houselight remained on for another 4 s after a response was made, after which the chamber reverted to the intertrial state (darkness). Multiple pellets were delivered 0.5 s apart. The four probability blocks were separated into eight forced-choice trials where only one lever was presented (four trials for each lever, randomized in pairs) allowing rats to learn the amount of food associated with each lever press and the respective probability of receiving reinforcement over each block. This was followed by 10 free-choice trials, where both levers were presented and the animal chose either the Small/Certain or the Large/Risky lever. For each session and trial block, the probability of receiving the risky reward was drawn from a set probability distribution. Therefore, on any given day, the probabilities in each block may vary, but on average across many training days, the actual probability experienced by the rat will approximate the set value. Using these probabilities, selection of the Large/Risky lever would be advantageous in the first two blocks, and disadvantageous in the last block, whereas rats could obtain an equivalent number of food pellets after responding on either lever during the 25% block. Therefore, in the last three trial blocks of this task, the selection of the larger reward option carried with it an inherent “risk” of not obtaining any reward on a given trial.

Probability-Discounting During Neural Recording

A modified version of the probability discounting task was used for rats in the recording experiment to increase the number of like-choice trials (20 for each probability instead of 10) and ensure participation in the lower probability blocks, particularly for the aged rats. Rats were trained on a probability discounting task until behavioral criterion was reached using only two probability blocks: 80% and 20%. Each probability block consisted of 10 forced-choice trials and 20 free-choice trials. The behavioral criterion was defined as stable behavior over 5 days (two-way ANOVA: no effect for day, a significant effect of probability).

Neural Recording Procedures

Electrode preparation and surgical procedures.

Recording tetrodes were constructed from 20 μm lacquer-coated tungsten wires (California Fine Wire) and mounted on one of two independently adjustable custom-built microdrives (three tetrodes per microdrive). Tetrode tips were gold-plated to reduce impedances to 0.1–0.4 MΩ (tested at 1 kHz). To implant recording electrodes, each rat was placed in an induction chamber and deeply anesthetized under isoflurane (4% mix with oxygen at a flow rate of 1 L/min). Under deep anesthesia, the animal was placed in a stereotaxic instrument (David Kopf Instruments, Tujunga, CA, USA) and anesthesia was maintained throughout surgery by isoflurane (1–2.5%) delivered via a nosecone. The skull was exposed and adjusted to place bregma and lambda on the same horizontal plane. After small burr holes were drilled, the microdrives were unilaterally implanted into the VTA (Young: −6.0 mm posterior, 0.5 mm lateral, and 7.0 mm ventral to Bregma. Aged: −7.5 mm posterior, 0.5 mm lateral, and 7.0 mm ventral to Bregma). An additional set of aged rats underwent a lesioning procedure to obtain the optimal stereotaxic coordinates for the VTA of young and aged rats since initial surgeries were unsuccessful in targeting the VTA of aged rats using standard atlas coordinates. Microdrive arrays were secured in place with anchoring screws and dental cement. Rats recovered for 7 days, during which they were weighed and handled daily.

Single-Unit Recording and Postsurgical Procedures

After a week of recovery, rats were returned to a food-restricted diet and spontaneous neural activity in the VTA was monitored as follows: the electronic interface board (Neuralynx, Bozeman, MT, USA) of the microdrives were connected to preamplifiers, and the outputs were transferred to a Cheetah data acquisition system (Neuralynx). Signals were filtered between 0.6 and 6 kHz and digitized at 16 kHz. Acquired waveforms were not inverted. Neuronal spikes were recorded for 2 ms after the voltage deflection exceeded a predetermined threshold at 500–7000X amplification. If no units were encountered, tetrodes were lowered in 40 μm increments to target new units. A non-motorized commutator (SwivElectra, Crist Instrument Company, Hagerstown, MD, USA) with custom Neuralynx adapters was mounted above the operant chamber and used to prevent the tether from getting tangled. A video camera mounted on the ceiling of the chamber recorded video, and data were relayed to the acquisition system. Once clearly isolated and stable units were found, the probability discounting task and recording began. Daily recording sessions included behavioral data and simultaneously recorded neural data that were time-stamped in the form of TTL pulses instantaneously sent to the neural recording system via the SuperPort TTL card (DIG-726) and the Cheetah data acquisition system. The commands sent by Med PC to run the behavioral experiment were also recorded in the Neuralynx event file as TTL pulses via a SmartCtrl connection panel (SG-716B; Zheng and Ycu, 2012 ). After each completed session, tetrodes were advanced about 40 μm. This resulted in the appearance of stable records from new cells the following day. Experimental sessions continued until tetrodes passed through the VTA based on the distance traveled from the brain surface.

Neural Data Analysis

Single units were isolated using an Offline Sorter (Plexon). Various waveform features, such as the relative peak, valley, width, and principle component, were compared across multiple units simultaneously recorded from the four wires of a tetrode. Only units showing good recording stability across the entire recording session were included. Further analysis of the sorted units was performed with custom Matlab software (Mathworks). To examine the reward-related responses of histologically verified VTA neurons, peri-event time histograms (PETHs) were constructed at 4.0 s around the time of all reward acquisition-triggered events. A bin size of 100 ms was used for all PETHs. VTA cells were identified based on phasic responses to reward acquisition, lever cues, and pellet cues. A VTA neuron was considered a reward or cue responsive if it at least doubled its firing rate during the 300 ms from cue or reward acquisition onset compared to its baseline firing rate. Putative DA and non-DA cells were classified using a cluster analysis developed to identify DA cells in the rodent VTA ( Roesch et al., 2007 ; Jin and Costa, 2010 ; Jo et al., 2013 ). Detailed information on this analysis can be found in previous reports ( Roesch et al., 2007 ; Takahashi et al., 2009 , 2011 ; Jo et al., 2013 ). Briefly, using the tetrode channel with the largest peak-to-valley amplitude, two basic characteristics of the average spike waveform were determined for each cell: (1) the half time of the spike duration (i.e., measured between the first valley and the next peak); and (2) the amplitude ratio of the first positive peak and negative valley in a waveform [( n − p )/( n + p ), with n as the first negative valley and p as the first positive peak]. A scatter plot including all VTA cells was then constructed for young and aged rats together, as well as for each age group separately. The cluster that included neurons with waveforms showing a broad half duration and low amplitude ratio was putatively classified as DAergic. Neurons that fell into multiple clusters were not classified. Spontaneous firing properties of putative DA cells were calculated from data collected during the ITIs (i.e., when rats were not engaged in the task-related behavior).

After the completion of all recording sessions, tetrode locations were verified. Rats were deeply anesthetized under 4% isoflurane. The final position of each tetrode was marked by passing a 15A current through a subset of the tetrode tips for 15 s Then, the animals were given an overdose of sodium pentobarbital and transcardially perfused with 0.9% saline and a 10% formaldehyde solution. Brains were stored in a 10% formalin–30% sucrose solution at 4°C for 72 h. The brains were frozen and then cut in coronal sections (45 μm) on a freezing sliding microtome. The sections were then mounted on gelatin-coated slides, stained with cresyl violet, and examined under light microscopy. Only cells verified to be recorded in VTA were included in the data analysis.

Statistical Analyses

Statistical analyses were performed using SPSS 22.0, Graphpad Prism 6.0, or custom Matlab software (MathWorks, Natick, MA, USA). The statistical tests used are indicated in the “Results” section where appropriate.

Age Effects on Probability Discounting Behavior

After stable behavior was reached, the mean choice of the large reward at each probability block was calculated for each rat. Each rat’s performance was averaged over the five-day stable behavior period. Performance scores reflect rats’ choice of the large reward lever during the choice blocks. The choice of the large reward was calculated only from those trials in which rats participated (i.e., and not omission trials). Then, rats’ scores were grouped by age. A mixed factorial ANOVA was run to assess the effects of age and probability on choice behavior. There was a significant main effect of probability ( F (3,104) = 43.53, p < 0.001) and a significant main effect of age ( F (1,104) = 7.93, p < 0.01), with aged rats showing reduced choice of the large reward over the four probability blocks ( Figure 1A ). The choice of the large/risky reward was not significantly different between aged and young rats when outcomes were certain, i.e., when the probability of the “risky” reward was 100% (independent t -test, t (25) = 1.84, p = 0.08). Instead, differences arose when risk was introduced, even when it was more beneficial to continue choosing the risky option, i.e., during the 50% block (independent t -test, t (25) = 1.99, p = 0.02) but not choices were equal, i.e during the 25% block (independent t -test, t (25) = 1.48, p = 0.08). Based on the probability of receiving a reward, optimal behavior would be 100% choice of the large reward, risky lever during the 100% and 50% block, equal during the 25%, and 100% choice of the small reward certain lever during the 12.5% block. There was no difference between the groups’ behavior for 100% or 12.5% blocks. However, the young rats had more optimal behavior as they chose the large reward risky lever more frequently, as a group, during the 50% block. Additionally, young rats received more sugar pellets (reward) per session than aged rats ( Figure 1B ; unpaired t -test, t (24) = 2.54, p = 0.02). This difference in the amount of reward received was not due to more omissions of choice trials by aged rats ( Figure 1C ; unpaired t -test, t (24) = 1.82, p = 0.08). When viewing each rat’s discounting behavior individually ( Figures 1D,E ), the trend for aged animals to choose the large risky reward less often than their younger counterparts become more clear, albeit there is a certain amount of individual variability in the level of discounting within both age groups.

www.frontiersin.org

Figure 1. (A) Mean (± SEM) choice of large rewards for all probability blocks grouped by age. Scores reflect rats’ choice of the large reward lever during the choice trials in which rats participated (i.e., not including omission trials). Aged rats chose the large risky lever significantly less often than young rats ( F (1,104) = 7.93, p < 0.01). (B) The Mean number of pellets received per session by age group. Young rats received more pellets per session on average than aged rats ( t (24) = 2.54, p = 0.02). (C) Mean number of omitted trials per session. There was not a significant difference in the number of omitted trials between groups ( t (24) = 1.82, p = 0.08). (D) Individual aged rats’ choice behavior across all four probability blocks. (E) Individual young rats’ choice behavior across all four probability blocks. Open circles: aged rats, n = 16; Closed circles: young rats, n = 12.

To gain more insight into how aged rats’ choices led to increased discounting, we next conducted a win-stay, lose-shift analysis. The ability to make flexible decisions based on choice feedback is important for optimal decision making. However, being too sensitive to losses or not sensitive enough to positive feedback may reduce gains in probabilistic paradigms. Aged rats have previously been shown to have win-stay and lose-shift deficits ( Means and Holsten, 1992 ). A win-stay deficit is defined as not repeating a choice that was previously rewarded, even when it is beneficial to do so. A lose-shift deficit is defined as not switching to the other option after a choice was not rewarded/correct. Win-stay and lose-shift behavior are thought to reflect sensitivity to positive and negative reinforcement, respectively. Thus, we wanted to assess if the increase in probability discounting observed with age-related to distinct changes in sensitivity to positive or negative reinforcement. To do this, a win-stay and lose-shift choice ratio were calculated for each rat. For win-stay, the ratio reflected the number of choices for the risky lever after a risky win out of the total number of trials for which there was a risky gain. The ratio of lose-shift choices was calculated as the number of choices for the small certain lever after the risky lever was chosen and no reward was delivered, i.e., a loss, out of the total number of loss trials. These ratios were calculated for each rat, and rats were then grouped by age for analysis. It was found that aged rats had significantly reduced win-stay choice ratios compared to young rats (independent t -test, t (24) = 2.42, p = 0.02; Figures 2A,B ). We examined if this effect was being driven primarily by the 50% and 25% blocks and found that aged rats had significantly reduced win-stay choice ratios in these blocks (two-way ANOVA, significant main effect of age p < 0.01; Bonferroni post hoc comparisons 50%: p < 0.05; 25%: p < 0.01; Figure 2B ). Win-stay choice ratios were not significantly different during the 12.5% block ( p > 0.05; Figure 2B ). However, when lose-shift choices were compared, aged and young rats displayed similar lose-shift behavior collapsed across all probabilities (independent t -test, t (24) = 0.69, p = 0.25; Figure 2C ), as well when examining each probability separately ( Figure 2D ; two-way ANOVA, F (1,68) = 0.02; p = 0.89). This suggests that aged rats have a selective deficit in maintaining risky choices after positive reinforcement and are not necessarily more loss-averse than their younger counterparts.

www.frontiersin.org

Figure 2 . Win-stay (A,B) and lose-shift choice ratios (C,D) . A win is defined as a choice for the large/risky lever that resulted in a reward. (Top) The choice ratio for win-stay represents the ratio of choices of the risky (as opposed to safe) lever following a “win” risky trial. (A) Individual rat win-stay choice ratios for aged and young groups. Aged rats showed significantly reduced win-stay ratios compared to young rats (independent t -test, t (24) = 2.42, p = 0.01). (B) This effect was specifically driven by a decrease in the choice ratio in the 50% and 25% blocks ( p’s < 0.05). (Bottom) A loss is defined as a choice for the large/risky lever that did not yield a reward. The choice ratio for lose-shift represents the ratio of choices for the small/certain lever after a “loss.” (C) Individual rat lose-shift choice ratios for aged and young animals. There was no difference in lose-shift behavior by age group (independent t -test, t (24) = 0.69, p = 0.25). (D) There was no difference in the lose-shift choice ratio in 50% and 25% blocks. Open symbols: aged rats, n = 16; Closed symbols: young rats, n = 12. * p < 0.05; ns, not significant, p > 0.05.

We next assessed if age-related changes on risk-based decision making were related to deficits in spatial memory. Previous research suggests that deficits in spatial memory tasks with age can be selective, indicating that age-related decline in the hippocampus happens independently of changes in other brain regions ( Gilbert et al., 2012 ; Samson et al., 2015 ). To assess this in the current study, individual choice behavior on the probability discounting task was correlated with performance on the water maze. A LIS was calculated based on performance on multiple probe trials on the water maze. LIS reflects the average proximity of the animal during probe trials to the training location of the escape platform; lower index scores indicate more accurate proximity and is used as an approximation of spatial memory ( Gallagher et al., 1993 ). Rats are considered “impaired” with an LIS ≥250. LISs have been shown to be associated with age-related changes in spatial memory ( Gallagher et al., 1993 ; Nicolle et al., 1999 ). As expected, aged rats had significantly higher LISs compared to young rats, even when collapsing aged impaired and unimpaired groups (one-tailed independent t -test, t (26) = 5.23, p < 0.0001; Figure 3A ). When grouping all 28 young and old rats together, no significant relationship between choice behavior on the probability discounting task and water maze performance was observed (Pearson correlation, R = -0.14, p = 0.47; Figure 3B ). We then compared choice behavior and water maze performance separately by age group. While not statistically significant, there was a positive relationship between large reward choice and LIS in aged rats ( R = 0.39, p = 0.13; Figure 3C ), and a negative relationship between large reward choice behavior and LIS in young rats ( R = -0.55, p = 0.06; Figure 3D ). There was also a significant difference between the two correlations (Fisher’s z = 2.26; p = 0.02), showing the relationship between risky reward choice and spatial memory performance was different for the two age groups. In other words, the more an aged rat chose the risky reward the more spatially impaired they were on the water maze task, while young rats displayed the inverse relationship between the two variables.

www.frontiersin.org

Figure 3. (A) Mean learning index score (LIS) by age group. Aged rats had significantly higher LISs (indicating poorer spatial memory) compared to young rats, even when collapsing aged impaired and unimpaired groups (one-tailed independent t -test, t (26) = 5.23, p < 0.0001). (B) The Mean choice of large reward collapsed across all probability blocks on the probability discounting task (%, Y-axis) correlated to LIS on the water maze (X-axis). No significant relationship was observed between choice behavior on the probability discounting task and water maze performance (Pearson correlation, R = −0.14, p = 0.47). The dotted line represents the threshold after which (to the right) rats are considered impaired on the water maze. Open circles: aged rats, n = 16; Closed circles: young rats, n = 12. (C) Relationship between risky choice and water maze performance for aged rats. No statistically significant relationship was found between these two variables (Pearson correlation, R = 0.39, p = 0.13). (D) Relationship between risky choice and water maze performance for young rats. No significant relationship was found between these two variables (Pearson correlation, R = −0.55, p = 0.06) although a trend is suggested. * p < 0.05.

Age Effects on Neural Responses During Probability Discounting

Another aim of the current study was to investigate VTA (both DA and non-DA neurons) neural responses to probabilistic rewards in aged and young animals. Loss of the appropriate phasic response to rewards may lead aged rats to interpret the risky reward as less reinforcing and thus be less sensitive to positive reinforcement. This could be a possible mechanism that reduces the ratio of win-stay choice behavior in aged rats. We recorded 269 neurons from the VTA in six rats ( Figures 4C,D ) while rats performed a two-probability version of the task. Aged and young rats did not have significantly different choice behavior in this version of the task ( Figures 4A,B ).

www.frontiersin.org

Figure 4. (A) Risk-based decisions by age group for recording experiment: mean (±SEM) choice of large rewards for 80% and 20% probability blocks grouped by age. Scores reflect rats’ choice of the large reward lever during the choice trials. There was no significant difference in choice behavior between age groups. (B) Win-stay and lose-shift choice behavior for aged and young rats. Open circles: aged rats, n = 3; Closed circles: young rats, n = 3. (C,D) Histological reconstruction of terminal tetrode tip locations in aged and young rats. ns = not significant, p > 0.05.

Of those 269 recorded neurons, 38 (14.1%) were identified as dopamine neurons based on previously established electrophysiological criteria (see “Materials and Methods” section; Roesch et al., 2007 ; Jo et al., 2013 ; Figure 5A ). Putative DA neuron numbers were similar to previous reports ( Roesch et al., 2007 ; 36/258; 13.9% and Jo et al., 2013 ; 203/905; 22%). However, there were proportionally fewer DA neurons recorded from aged rats than young [Aged: 11 DA neurons out of 115 total (10.4%); Young: 26 DA neurons out of 154 total (16.9%)]. However, this proportional frequency was not significantly different between groups ( χ 2 = 2.97, p = 0.08). Aged rats’ DA neurons had a significantly lower firing rate than young rats’ DA neurons ( p < 0.05; Figure 5B ). To potentially account for this age-difference, it is important to note that DA neurons show functional changes with age. While less sensitive to neuronal loss than the nigrostriatal system, DA neurons of the VTA display functional decline as DA metabolites (DOPAC, HVA) decline with age in the rat VTA ( Goudsmit et al., 1990 ). VTA DA neurons display axonal degeneration, a build-up of amyloid precursor protein and alpha-synuclein, and loss of tyrosine hydroxylase (TH) despite no obvious neurodegeneration ( Cruz-Muros et al., 2007 ). Most importantly, age can affect the electrophysiological properties of recorded DA neurons. In a previous report, it was found that DA neurons from aged mouse substantia nigra showed lower spontaneous firing rates, impaired firing fidelity, narrower spike widths, and decreased L-type calcium currents ( Branch et al., 2014 ). Therefore, using previously established methods to identify putative young adult DA neurons in the VTA may lead to exclusion of DA neurons in the aged rat. Due to the above-described age changes in DA neural integrity, we initially analyzed DA cell responses after identifying cells as DAergic based on published criteria (e.g., Jo et al., 2013 ; Figure 5A ). Then, we performed the same analysis on all VTA cell records from young and old rats.

www.frontiersin.org

Figure 5. (A) Results of cluster analysis for all ventral tegmental area (VTA) neurons ( N = 269). Putative dopamine (DA) cells were identified based on half spike duration and the amplitude ratio of the initial negative valley ( n ) and positive peak ( p ). Closed circles are putative DA neurons ( n = 38); open circles are classified as non-DA neurons ( n = 231). (B) The overall rate of the putative aged DA neurons (0.77 ± 0.24 spikes/s) was significantly reduced compared to young rats (3.25 ± 0.82 spikes/s; t (35) = 2.24, p = 0.02). (C) Mean firing rate (Hz) of young rats’ putative DA neurons ( n = 26) centered around the lever cue (±4.0 s). Blue traces indicate responses during the 80% block; gray traces represent the 20% block. DA neurons’ responses to the lever cue was significantly greater than baseline firing rates (repeated measures ANOVA, significant main effect of time, F (2,25) = 10.92, p < 0.001) and this response was greater during the high probability block (Bonferroni post hoc multiple comparisons test, t = 2.32, p = 0.01). (D) Mean firing rate (Hz) of young rats’ putative DA neurons centered around the first pellet cue (±4.0 s). The neurons significantly increased firing rate to the pellet cue compared to baseline (repeated measures ANOVA, significant main effect of time, F (2,25) = 10.49, p = 0.01), but they did not show significantly increased firing rates to this cue during the 80% block compared to the 20% block (Bonferroni post hoc multiple comparisons tests, t = 0.77, p > 0.05). (F) Mean rate (Hz) of aged rats’ putative DA neurons ( n = 11) centered around the lever cue (±4.0 s). Red traces indicate responses during the 80% block; gray traces represent the 20% block. (G) Mean rate (Hz) of aged rats’ putative DA neurons centered around the first pellet cue (±4.0 s). (E) Individual DA neurons’ normalized firing rate change from baseline to the first pellet cue ( Z scores). (H) Individual DA neurons’ normalized firing rate change from baseline to the lever cue ( Z scores). Blue dots represent young rats’ neurons; red dots represent aged rats’ neurons. These changes in firing rate are separated by the high and low probability blocks. * p < 0.05.

Of interest were DA neuron responses to probabilistic cues that indicate the availability of reward (lever cue), cues indicating when a choice resulted in a reward (pellet cue), and the receipt of the reward itself (first nose poke after a reward was delivered). Based on previous research, it is known that DA neurons encode the value of probabilistic rewards: neurons exhibit a lower firing rate to cues that are associated with a lower probability of receiving reward compared to cues that are associated with a high probability of receiving reward, and DA neurons respond more to a primary reward that was preceded by a cue associated with a low probability of receipt ( Fiorillo et al., 2003 ). Therefore, we hypothesized that DA neurons would exhibit phasic responses to the lever cue, with a higher firing rate to the cue during the 80% block compared to the 20% block. Additionally, we expected higher phasic firing of DA neurons to the pellet cue, with a higher firing rate during the 20% block compared to the 80% block. In the young rats, DA neurons indeed exhibited phasic firing to the lever cue that was significantly greater than baseline firing rates (repeated measures ANOVA, significant main effect of time, F (2,25) = 10.92, p < 0.001; Figure 5C ) and this phasic response was greater during the high probability block (Bonferroni post hoc multiple comparisons tests, t = 2.32, p = 0.01). When examining these same neurons’ responses to the pellet cue, we found that they exhibited significantly increased firing rate to the pellet cue compared to baseline (repeated measures ANOVA, significant main effect of time, F (2,25) = 10.49, p = 0.01; Figure 5D ), but they did not show significantly increased firing rate to this cue during the 80% block compared to the 20% block (Bonferroni post hoc multiple comparisons tests, t = 0.77, p > 0.05). While there was a significant increase in firing rate to the first pellet cue and lever cue in the majority of young rats’ DA, we did find great variability in the magnitude and direction of change when examining the neurons individually ( Figures 5G,H ). The aged rats’ neurons, by comparison, did not show any detectable responses to any of these cues as a group ( Figures 5E,F ). Additionally, overall rate of the putative aged DA neurons (0.77 ± 0.24 spikes/s) was significantly reduced compared to young rats (3.25 ± 0.82 spikes/s; t (35) = 2.24, p = 0.02). Compared to young rats, individual DA neurons from aged rats showed a consistent pattern of a lower magnitude of change in firing rate to these cues ( Figures 5G,H ).

Since the DA waveform analysis could exclude DA neurons in the aged rats due to functional changes in the neurons over time, we could be neglecting neurons that do indeed encode the salient events of the task. Thus, the neural data were reanalyzed to include all VTA neurons for both age groups. Previous reports show that even non-DA neurons of the VTA encode salient events ( Puryear et al., 2010 ), albeit with different firing patterns in response to reward-predicting cues ( Cohen et al., 2012 ). When examining all young rats’ VTA neurons response to reward, or first pellet cue, we found that the firing rate was significantly higher during this cue period compared to baseline (two-way ANOVA, significant main effect of time, F (1,568) = 10.87, p = 0.001; Figure 6A ), but there was no significant difference between probability blocks to this cue (no main effect of probability, F (1,568) = 0.13, p = 0.72; Figure 6A ). In the same analysis on aged rats’ VTA neurons, there was no significant difference in firing rate between baseline and the cue, or between probability blocks (two-way ANOVA, no effect of time, F (1,330) = 0.33, p = 0.56; or probability, F (1,330) = 1.83, p = 0.18; Figure 6D ).

www.frontiersin.org

Figure 6. (A) Mean firing rate (Hz) of young rats’ VTA neurons ( n = 151) centered around the first pellet cue (±4.0 s). (B) Mean firing rate (Hz) of young rats’ lever excited VTA (DA + nonDA) neurons ( n = 37) centered around the lever cue (±4.0 s). Blue traces indicate responses during the 80% block; gray traces represent the 20% block. The neurons displayed significantly greater firing rate responses to the lever cue during the high probability block compared to the low probability block (one-tailed dependent t -test, t (36) = 1.73, p = 0.04). (C) Peri-event time histogram (PETH) of a lever excited young neuron’s responses to the lever cue during high and low probability blocks. Left Y-axis represents the mean firing rate (Hz) over all the events; time 0 indicates the onset of lever cue. Raster diagrams show individual spikes around the lever cue. Top: 80% block; bottom: 20% block. (D) Mean firing rate (Hz) of aged rats’ VTA neurons ( n = 107) centered around the first pellet cue (±4.0 s). (E) Mean firing rate (Hz) of aged rats’ lever excited VTA (DA + nonDA) neurons ( n = 9) centered around the lever cue (±4.0 s). Red traces indicate responses during the 80% block; gray traces represent the 20% block. The neurons did not display a significant difference in firing rate between the two probability blocks (one-tailed dependent t -test, t (8) = 0.09, p = 0.46). (F) PETH of a lever excited aged neuron’s responses to the lever cue during high and low probability blocks. Left Y-axis represents the mean firing rate (Hz) over all the events; time 0 indicates the onset of lever cue. Raster diagrams show individual spikes around the lever cue. Top: 80% block; bottom: 20% block. * p < 0.05; ns = not significant, p > 0.05.

In the present study, for the young rats, 37/154 (24.0%) of neurons displayed a pattern of phasic excitation to the lever cue ( Figure 6B ). The responses of these neurons during the first 300 ms after the onset of the lever cue were compared between the high and low probability blocks to see if the phasic excitation of these neurons scaled with the probability of receiving the large reward. Indeed, the neurons displayed significantly greater firing rate responses to the lever cue during the high probability block compared to the low probability block (one-tailed dependent t -test, t (36) = 1.73, p = 0.04; Figure 6B ). For the aged rats, only 9/115 (7.8%) of neurons showed excitation to the lever cue and when these responses were analyzed to examine if they encoded the probability of available reward it was found that the neurons did not display a difference in firing rate between the two probability blocks (one-tailed dependent t -test, t (8) = 0.09, p = 0.46). Therefore, the aged rats’ VTA neurons did not distinguish between the cues that predict the probabilistic rewards with high and low probabilities while the young rats VTA neurons did. This is evident even in individual neurons’ responses to the lever cue in young and aged rats ( Figures 6C,F ). These results indicate that the increase in probability discounting in aged rats may be due to the blunting of the dynamic range of the VTA neurons’ responses, which might prevent the aged rats from correctly assessing the value of the risky option.

Age-Related Increase in Probability Discounting

It is important to understand how and why risk-based decision making changes over the lifespan. The current data show that aged rats display a decreased choice of the large risky reward, i.e., more discounting on the probability discounting task, reflecting an age-related change in risk-based decision making. Previous reports have not found this same age-related effect on probability discounting in rats ( Gilbert et al., 2012 ; Samson et al., 2015 ). This apparent discrepancy could be due to rat strain differences, i.e., F344 rats in Gilbert et al. (2012) and Samson et al. (2015) , or a difference in methodologies rather than a lack of an age-related effect on probability discounting. For example, in Gilbert et al. (2012) , the reward was grain-based rather than sucrose and the large reward was twice the size of the small, vs. four times in our current study. These differences in reward palatability and size could affect the computations animals make when considering their options. The findings here are in line with increased risk-aversion observed in older humans (e.g., Lee et al., 2008 ). Importantly, differences in choice behavior were not due to increased omitted trials for aged rats. Thus, we believe our model of risk-based decision making can be used to study the underlying physiological changes in the brain that contribute to the observed alterations in decision making with age.

Age-Related Reduction in Response to Positive, Not Negative, Reinforcement

We found that the increased discounting was related to alterations in choices after positive reinforcement as aged rats show decreased win-stay behavior compared to their younger counterparts. It has previously been shown that aged rats have a deficit in repeating a choice that was previously rewarded ( Means and Holsten, 1992 ). On the other hand, we did not find a significant difference between the age groups in lose-shift behavior, indicating that aged animals are not necessarily more loss-averse than their younger counterparts. Therefore, age-related increases in discounting on the probability task are more likely to be due to changes in adequately assessing the positive outcomes of past choices. This effect seems to be selective for risky choices, as the choice of the large/risky reward was not significantly different between aged and young rats when outcomes were certain, i.e., when the probability of the “risky” reward was 100%. Instead, deficits arose when risk was introduced, even when it is more beneficial to continue choosing the risky option, i.e., during the 50% block. This is in line with human research that shows older individuals selectively avoid risky options, even if such choices reduce overall gains ( Lee et al., 2008 ; Rutledge et al., 2016 ).

Age-Related Changes in Spatial Memory and Probability Discounting Are Not Strongly Coupled

Age-related decline of hippocampal function has been vigorously studied in rats using spatial memory tasks like the Morris water maze. These tests reveal that aged rats consistently show impairment on hippocampal-dependent tasks compared to young rats ( Gallagher et al., 2011 ). Indeed, in vivo investigations into hippocampal function reveal that place cells are less likely to update in new environments, and this decrease in adaptability is correlated with deficits in spatial memory ( Wilson et al., 2004 ). Consistent with this literature, aged rats tested in this study showed significantly impaired water maze performance compared to younger animals. We were interested in whether this decline in spatial memory was correlated with altered risk-based decision making. Although it is thought that cognitive decline with age in one brain region is not necessarily predictive of decline in other regions, most of this research is restricted to assessing the relationship between PFC and hippocampal-dependent tasks ( Fiorillo et al., 2003 ; Moore et al., 2009 ; Fletcher and Rapp, 2012 ). We found that choice behavior on the probability discounting task was not significantly related to water maze performance. This is consistent with previous similar studies that assessed whether there is a relationship between the two types of behavioral performances ( Gilbert et al., 2012 ; Samson et al., 2015 ). Therefore, there is insufficient data at this time to link spatial working memory decline in age with increased probability discounting. With additional study, it would be of interest to determine whether, as has been done with studies of age-related memory changes, further exploitation of individual differences in discounting task performance will reveal new insights into underlying age-related brain mechanisms of decision making ( Orsini et al., 2019 ).

Age-Related Reduction in VTA Dopamine Responses to Reward

The in vivo electrophysiological data recorded from young and aged rats’ VTA show a striking lack of dynamic range in aged VTA neuron responses. This result supports the view that a functional decline in these neurons could ultimately contribute to altered risk-based decision making. The reduced responsiveness of aged VTA neurons to rewarding stimuli may relate to the age-related reduced sensitivity to positive reinforcement observed in the win-stay analysis. Known cellular changes in aged DA neural properties (e.g., Branch et al., 2014 ) may underlie the low rate and lack of responses by putative DA neurons. For this reason, at this time, we cannot be confident that the neurons classified as dopaminergic in the aged rats are truly dopaminergic due to the possibility that the waveform characteristics may have changed with age ( Rollo, 2009 ; Branch et al., 2014 ). Similarly, cells classified as non-dopaminergic may indeed be (aged) dopamine neurons. Even so, reduce dopaminergic function in aged rats could contribute to the reduced selection of the risky reward during probability discounting. Previous research indicates that both D1 and D2 dopamine receptor antagonism can reduce the preference for the large risky reward in a similar probability discounting task ( St. Onge and Floresco, 2009 ). Additionally, there is a positive correlation between enhanced DA release in the nucleus accumbens and increased risky decision-making ( Freels et al., 2020 ). It remains to be determined whether the increased risky choice could be accounted for by a change in the impacts of uncertainty on reward value and/or a change in the salience of rewards that drive behaviors (e.g., Roesch et al., 2012 ). Additionally, future work is needed to confirm the neurochemical identity of recorded VTA neurons of aged rats in order to elucidate if the blunted responses are specifically due to changes in dopamine neurons and their impact on the broader neural circuits within which they operate to affect learning and memory.

There was not a significant difference in behavior between the two age groups in the two-probability block recording task. However, the lack of a striking behavioral effect between the groups in the two-probability block is not surprising given 80% and 20% of four times a reward is a clearly superior and inferior choice, respectively. The behavioral differences between the two age groups are more obvious when the choice is more difficult if one is risk-averse.

Normal aging is known to affect a wide range of cognitive abilities. The current data support previous research that showed advanced aging affects how decisions are made in risky situations. In particular, we showed that this change may be related to reduced responses to positive, not negative, reinforcement. The latter was supported by additional findings that the aged VTA DA cells are less responsive to rewards when compared to similar cells recorded from young animals. A useful model for examining risky decision-making alteration late into the lifespan can help us understand how older individuals make decisions when faced with uncertain situations. Given the projected increases in the aged population across the world, understanding these changes has important implications for society. Additionally, understanding the physiological changes the brain undergoes with age can aide us in developing treatments to ameliorate any deleterious effects aging may have on cognition.

Data Availability Statement

All datasets generated for this study are included in the article.

Ethics Statement

The animal study was reviewed and approved by University of Washington’s and National Institute of Aging’s Institutional Animal Care and Use Committees.

Author Contributions

VT and SM contributed to the conception and design study. VT, PB, and JL performed the described experiments. VT and JL performed data analysis. VT submitted an earlier draft of the manuscript in partial fulfillment of a Ph.D. dissertation. All authors contributed to the comments and then approved the submitted version.

This work was supported by the National Institutes of Health (NIA grant T32AG000057 to fellow VT; NIMH grants MH58755 and MH119391 to SM) and in part by the Intramural Research Program of the NIH, National Institute on Aging. The the Intramural Research Program of the NIH, National Institute on Aging provided subjects and water maze assessments. NIMH to SM and NIA to VT provided funds to conduct the operant testing of subjects and neural recordings and analysis.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Bäckman, L., Lindenberger, U., Li, S. C., and Nyberg, L. (2010). Linking cognitive aging to alterations in dopamine neurotransmitter functioning: recent data and future avenues. Neurosci. Biobehav. Rev. 34, 670–677. doi: 10.1016/j.neubiorev.2009.12.008

PubMed Abstract | CrossRef Full Text | Google Scholar

Bäckman, L., Nyberg, L., Lindenberger, U., Li, S. C., and Farde, L. (2006). The correlative triad among aging, dopamine, and cognition: current status and future prospects. Neurosci. Biobehav. Rev. 30, 791–807. doi: 10.1016/j.neubiorev.2006.06.005

Barense, M. D., Fox, M. T., and Baxter, M. G. (2002). Aged rats are impaired on an attentional set-shifting task sensitive to medial frontal cortex damage in young rats. Learn. Mem. 9, 191–201. doi: 10.1101/lm.48602

Bizon, J. L., LaSarge, C. L., Montgomery, K. S., McDermott, A. N., Setlow, B., and Griffith, W. H. (2009). Spatial reference and working memory across the lifespan of male Fischer 344 rats. Neurobiol. Aging 30, 646–655. doi: 10.1016/j.neurobiolaging.2007.08.004

Branch, S. Y., Sharma, R., and Beckstead, M. J. (2014). Aging decreases L-type calcium channel currents and pacemaker firing fidelity in substantia nigra dopamine neurons. J. Neurosci. 34, 9310–9318. doi: 10.1523/jneurosci.4228-13.2014

Cardinal, R. N., and Howes, N. J. (2005). Effects of lesions of the nucleus accumbens core on choice between small certain rewards and large uncertain rewards in rats. BMC Neurosci. 6:37. doi: 10.1186/1471-2202-6-37

Cohen, J. Y., Haesler, S., Vong, L., Lowell, B. B., and Uchida, N. (2012). Neuron-type-specific signals for reward and punishment in the ventral tegmental area. Nature 482, 85–88. doi: 10.1038/nature10754

Cruz-Muros, I., Afonso-Oramas, D., Abreu, P., Barroso-Chinea, P., Rodríguez, M., González, M. C., et al. (2007). Aging of the rat mesostriatal system: differences between the nigrostriatal and the mesolimbic compartments. Exp. Neurol. 204, 147–161. doi: 10.1016/j.expneurol.2006.10.004

Dreher, J. C., Meyer-Lindenberg, A., Kohn, P., and Berman, K. F. (2008). Age-related changes in midbrain dopaminergic regulation of the human reward system. Proc. Natl. Acad. Sci. U S A 105, 15106–15111. doi: 10.1073/pnas.0802127105

Dumitriu, D., Hao, J., Hara, Y., Kaufmann, J., Janssen, W. G., Lou, W., et al. (2010). Selective changes in thin spine density and morphology in monkey prefrontal cortex correlate with aging-related cognitive impairment. J. Neurosci. 30, 7507–7515. doi: 10.1523/jneurosci.6410-09.2010

Fields, H. L., Hjelmstad, G. O., Margolis, E. B., and Nicola, S. M. (2007). Ventral tegmental area neurons in learned appetitive behavior and positive reinforcement. Annu. Rev. Neurosci. 30, 289–316. doi: 10.1146/annurev.neuro.30.051606.094341

Fiorillo, C. D., Tobler, P. N., and Schultz, W. (2003). Discrete coding of reward probability and uncertainty by dopamine neurons. Science 299, 1898–1902. doi: 10.1126/science.1077349

Fletcher, B. R., and Rapp, P. R. (2012). Normal Neurocognitive Aging. Handbook of Psychology. 2nd Edn.

Google Scholar

Freels, T. G., Gabriel, D. B., Lester, D. B., and Simon, N. W. (2020). Risky decision-making predicts dopamine release dynamics in nucleus accumbens shell. Neuropsychopharmacology 45, 266–275. doi: 10.1038/s41386-019-0527-0

Gallagher, M., Burwell, R., and Burchinal, M. R. (1993). Severity of spatial learning impairment in aging: development of a learning index for performance in the Morris water maze. Behav. Neurosci. 107, 618–626. doi: 10.1037/0735-7044.107.4.618

Gallagher, M., Bizon, J. L., Hoyt, E. C., Helm, K. A., and Lund, P. K. (2003). Effects of aging on the hippocampal formation in a naturally occurring animal model of mild cognitive impairment. Exp. Gerontol. 38, 71–77. doi: 10.1016/s0531-5565(02)00159-6

Gallagher, M., Stocker, A. M., and Koh, M. T. (2011). Mindspan: lessons from rat models of neurocognitive aging. ILAR J. 52, 32–40. doi: 10.1093/ilar.52.1.32

Gilbert, R. J., Mitchell, M. R., Simon, N. W., Bañuelos, C., Setlow, B., and Bizon, J. L. (2012). Risk, reward, and decision-making in a rodent model of cognitive aging. Front. Neurosci. 5:144. doi: 10.3389/fnins.2011.00144

Goudsmit, E., Feenstra, M. G. P., and Swaab, D. F. (1990). Central monoamine metabolism in the male Brown-Norway rat in relation to aging and testosterone. Brain Res. Bull. 25, 755–763. doi: 10.1016/0361-9230(90)90054-4

Jin, X., and Costa, R. M. (2010). Start/stop signals emerge in nigrostriatal circuits during sequence learning. Nature 466, 457–462. doi: 10.1038/nature09263

Jo, Y. S., Lee, J., and Mizumori, S. J. (2013). Effects of prefrontal cortical inactivation on neural activity in the ventral tegmental area. J. Neurosci. 33, 8159–8171. doi: 10.1523/jneurosci.0118-13.2013

Lee, T. M., Leung, A. W., Fox, P. T., Gao, J. H., and Chan, C. C. (2008). Age-related differences in neural activities during risk taking as revealed by functional MRI. Soc. Cogn. Affect. Neurosci. 3, 7–15. doi: 10.1093/scan/nsm033

Mata, R., Josef, A. K., Samanez-Larkin, G. R., and Hertwig, R. (2011). Age differences in risky choice: a meta-analysis. Ann. N Y Acad. Sci. 1235, 18–29. doi: 10.1111/j.1749-6632.2011.06200.x

Means, L. W., and Holsten, R. D. (1992). Individual aged rats are impaired on repeated reversal due to loss of different behavioral patterns. Physiol. Behav. 52, 959–963. doi: 10.1016/0031-9384(92)90377-e

Moore, T. L., Schettler, S. P., Killiany, R. J., Rosene, D. L., and Moss, M. B. (2009). Effects on executive function following damage to the prefrontal cortex in the rhesus monkey ( Macaca mulatta ). Behav. Neurosci. 123, 231–241. doi: 10.1037/a0014723

Morrison, J. H., and Baxter, M. G. (2012). The ageing cortical synapse: hallmarks and implications for cognitive decline. Nat. Rev. Neurosci. 13, 240–250. doi: 10.1038/nrn3200

Morrison, J. H., and Hof, P. R. (1997). Life and death of neurons in the aging brain. Science 278, 412–419. doi: 10.1126/science.278.5337.412

Mather, M., Mazar, N., Gorlick, M. A., Lighthall, N. R., Burgeno, J., Schoeke, A., et al. (2012). Risk preferences and aging: the “certainty effect” in older adults’ decision making. Psychol. Aging 27:801. doi: 10.1037/a0030174

Nicolle, M. M., Colombo, P. J., Gallagher, M., and McKinney, M. (1999). Metabotropic glutamate receptor-mediated hippocampal phosphoinositide turnover is blunted in spatial learning-impaired aged rats. J. Neurosci. 19, 9604–9610. doi: 10.1523/jneurosci.19-21-09604.1999

Orsini, C. A., Hernandez, C. M., Bizon, J. L., and Setlow, B. (2019). Deconstructing value-based decision making via temporally selective manipulation of neural activity: insights from rodent models. Cogn. Affect. Behav. Neurosci. 19, 459–476. doi: 10.3758/s13415-018-00649-0

Puryear, C. B., Kim, M. J., and Mizumori, S. J. (2010). Conjunctive encoding of movement and reward by ventral tegmental area neurons in the freely navigating rodent. Behav. Neurosci. 124, 234–247. doi: 10.1037/a0018865

Rapp, P. R., Deroche, P. S., Mao, Y., and Burwell, R. D. (2002). Neuron number in the parahippocampal region is preserved in aged rats with spatial learning deficits. Cereb. Cortex 12, 1171–1179. doi: 10.1093/cercor/12.11.1171

Rapp, P. R., and Gallagher, M. (1996). Preserved neuron number in the hippocampus of aged rats with spatial learning deficits. Proc. Natl. Acad. Sci. U S A 93, 9926–9930. doi: 10.1073/pnas.93.18.9926

Roesch, M. R., Calu, D. J., and Schoenbaum, G. (2007). Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards. Nat. Neurosci. 10, 1615–1624. doi: 10.1038/nn2013

Roesch, M. R., Esber, G. R., Bryden, D. W., Cerri, D. H., Haney, Z. R., and Schoenbaum, G. (2012). Normal aging alters learning and attention-related teaching signals in basolateral amygdala. J. Neurosci. 32, 13137–13144. doi: 10.1523/jneurosci.2393-12.2012

Rollo, C. D. (2009). Dopamine and aging: intersecting facets. Neurochem. Res. 34, 601–629. doi: 10.1007/s11064-008-9858-7

Rosenzweig, E. S., and Barnes, C. A. (2003). Impact of aging on hippocampal function: plasticity, network dynamics, and cognition. Prog. Neurobiol. 69, 143–179. doi: 10.1016/s0301-0082(02)00126-0

Rutledge, R. B., Smittenaar, P., Zeidman, P., Brown, H. R., Adams, R. A., Lindenberger, U., et al. (2016). Risk taking for potential reward decreases across the lifespan. Curr. Biol. 26, 1634–1639. doi: 10.1016/j.cub.2016.05.017

Samanez-Larkin, G. R., Gibbs, S. E., Khanna, K., Nielsen, L., Carstensen, L. L., and Knutson, B. (2007). Anticipation of monetary gain but not loss in healthy older adults. Nat. Neurosci. 10, 787–791. doi: 10.1038/nn1894

Samson, R. D., Venkatesh, A., Lester, A. W., Weinstein, A. T., Lipa, P., and Barnes, C. A. (2015). Age differences in strategy selection and risk preference during risk-based decision making. Behav. Neurosci. 129, 138–148. doi: 10.1037/bne0000037

Schoenbaum, G., Setlow, B., Saddoris, M. P., and Gallagher, M. (2006). Encoding changes in orbitofrontal cortex in reversal-impaired aged rats. J. Neurophysiol. 95, 1509–1517. doi: 10.1152/jn.01052.2005

St. Onge, J. R., Chiu, Y. C., and Floresco, S. B. (2010). Differential effects of dopaminergic manipulations on risky choice. Psychopharmacology 211, 209–221. doi: 10.1007/s00213-010-1883-y

St. Onge, J. R., and Floresco, S. B. (2009). Dopaminergic modulation of risk-based decision making. Neuropsychopharmacology 34, 681–697. doi: 10.1038/npp.2008.121

Takahashi, Y. K., Roesch, M. R., Stalnaker, T. A., Haney, R. Z., Calu, D. J., Taylor, A. R., et al. (2009). The orbitofrontal cortex and ventral tegmental area are necessary for learning from unexpected outcomes. Neuron 62, 269–280. doi: 10.1016/j.neuron.2009.03.005

Takahashi, Y. K., Roesch, M. R., Wilson, R. C., Toreson, K., O’Donnell, P., Niv, Y., et al. (2011). Expectancy-related changes in firing of dopamine neurons depend on orbitofrontal cortex. Nat. Neurosci. 14, 1590–1597. doi: 10.1038/nn.2957

Wilson, I. A., Ikonen, S., Gureviciene, I., McMahan, R. W., Gallagher, M., Eichenbaum, H., et al. (2004). Cognitive aging and the hippocampus: how old rats represent new environments. J. Neurosci. 24, 3870–3878. doi: 10.1523/jneurosci.5205-03.2004

Wise, R. A. (2004). Dopamine, learning and motivation. Nat. Rev. Neurosci. 5, 483–494. doi: 10.1038/nrn1406

Zheng, W., and Ycu, E. A. (2012). A fully automated and highly versatile system for testing multi-cognitive functions and recording neuronal activities in rodents. J. Vis. Exp. 63:e3685. doi: 10.3791/3685

Keywords: aging, probability discounting, ventral tegmentum, dopamine reward responses, memory

Citation: Tryon VL, Baker PM, Long JM, Rapp PR and Mizumori SJY (2020) Loss of Sensitivity to Rewards by Dopamine Neurons May Underlie Age-Related Increased Probability Discounting. Front. Aging Neurosci. 12:49. doi: 10.3389/fnagi.2020.00049

Received: 14 November 2019; Accepted: 12 February 2020; Published: 06 March 2020.

Reviewed by:

Copyright © 2020 Tryon, Baker, Long, Rapp and Mizumori. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Sheri J. Y. Mizumori, [email protected]

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here .

Loading metrics

Open Access

Peer-reviewed

Research Article

Delay and Probability Discounting of Sexual and Monetary Outcomes in Individuals with Cocaine Use Disorders and Matched Controls

* E-mail: [email protected]

Affiliation Behavioral Pharmacology Research Unit, Department of Psychiatry and Behavioral Sciences, Johns Hopkins University School of Medicine, Baltimore, Maryland, United States of America

  • Matthew W. Johnson, 
  • Patrick S. Johnson, 
  • Evan S. Herrmann, 
  • Mary M. Sweeney

PLOS

  • Published: May 27, 2015
  • https://doi.org/10.1371/journal.pone.0128641
  • Reader Comments

Table 1

Individuals with cocaine use disorders are disproportionately affected by HIV/AIDS, partly due to higher rates of unprotected sex. Recent research suggests delay discounting of condom use is a factor in sexual HIV risk. Delay discounting is a behavioral economic concept describing how delaying an event reduces that event’s value or impact on behavior. Probability discounting is a related concept describing how the uncertainty of an event decreases its impact on behavior. Individuals with cocaine use disorders ( n = 23) and matched non-cocaine-using controls ( n = 24) were compared in decision-making tasks involving hypothetical outcomes: delay discounting of condom-protected sex (Sexual Delay Discounting Task), delay discounting of money, the effect of sexually transmitted infection (STI) risk on likelihood of condom use (Sexual Probability Discounting Task), and probability discounting of money. The Cocaine group discounted delayed condom-protected sex (i.e., were more likely to have unprotected sex vs. wait for a condom) significantly more than controls in two of four Sexual Delay Discounting Task partner conditions. The Cocaine group also discounted delayed money (i.e., preferred smaller immediate amounts over larger delayed amounts) significantly more than controls. In the Sexual Probability Discounting Task, both groups showed sensitivity to STI risk, however the groups did not differ. The Cocaine group did not consistently discount probabilistic money more or less than controls. Steeper discounting of delayed, but not probabilistic, sexual outcomes may contribute to greater rates of sexual HIV risk among individuals with cocaine use disorders. Probability discounting of sexual outcomes may contribute to risk of unprotected sex in both groups. Correlations showed sexual and monetary results were unrelated, for both delay and probability discounting. The results highlight the importance of studying specific behavioral processes (e.g., delay and probability discounting) with respect to specific outcomes (e.g., monetary and sexual) to understand decision making in problematic behavior.

Citation: Johnson MW, Johnson PS, Herrmann ES, Sweeney MM (2015) Delay and Probability Discounting of Sexual and Monetary Outcomes in Individuals with Cocaine Use Disorders and Matched Controls. PLoS ONE 10(5): e0128641. https://doi.org/10.1371/journal.pone.0128641

Academic Editor: Peter G. Roma, Institutes for Behavior Resources and Johns Hopkins University School of Medicine, UNITED STATES

Received: January 26, 2015; Accepted: April 29, 2015; Published: May 27, 2015

Copyright: © 2015 Johnson et al. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited

Data Availability: De-identified data are available upon request from Johns Hopkins Medicine Institutional Review Board 3 and author MWJ. The source of the data are human participants. The Johns Hopkins Medicine Institutional Review Board requires approval before sending study data to an outside source.

Funding: This research was supported by National Institute on Drug Abuse ( http://www.drugabuse.gov ) grant R01DA032363 (MWJ). PSJ, ESH, and MMS were supported by an institutional training grant from the National Institute on Drug Abuse (T32DA007209). The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The authors have declared that no competing interests exist.

Introduction

Approximately 1.5 million people in the U.S. have used cocaine within the past month [ 1 ]. HIV prevalence among individuals who use cocaine (4–22%) [ 2 – 6 ] is many times higher than the national average (0.4%) [ 7 ]. Only about ten percent of individuals who use cocaine inject [ 8 – 10 ], and HIV rates among injecting vs. non-injecting individuals who use cocaine are similar [ 2 , 5 , 11 ], suggesting that risky sexual behavior is the most prominent HIV transmission vector among individuals who use cocaine.

Most sexual HIV risk reduction interventions for individuals who use cocaine target HIV risk reduction knowledge (e.g., [ 12 – 14 ]) and condom use skills (e.g., [ 15 ]). Although these interventions increase knowledge and skills, meta-analyses have demonstrated less robust effectiveness in reducing risk behavior [ 16 – 17 ]. The observation that individuals who use cocaine continue to engage in risky sexual behavior despite knowledge/skills improvements prompts examination of other factors that may underlie risk behavior, including decision-making processes.

Delay discounting provides a useful framework for examining relations between decision making and risk behavior. Delay discounting is a concept from the field of behavioral economics describing how delaying an event reduces that event’s value or impact on behavior. This is shown, for example, by the observation that individuals typically prefer immediate over delayed rewards. Most delay discounting studies ask participants to make choices between receiving smaller amounts of money available immediately vs. larger amounts available after various delays. Steeper discounting of monetary rewards is related to cocaine use (e.g., [ 18 – 21 ]) and use of other substances [ 22 ], as well as a variety of non-drug-related problem behaviors, including pathological gambling [ 23 – 24 ], obesity [ 25 – 26 ], and failing to engage in preventive health behaviors [ 27 – 29 ]. However, choices between immediate and delayed outcomes involve a variety of reinforcers other than money. For example, in a casual sex scenario, one may prefer to use a condom because it decreases the risk of sexually transmitted infection (STI). However, if a condom is not readily available, the same person might prefer immediate unprotected sex over waiting to obtain a condom. In other words, the value of condom protection may be discounted due to delay.

The Sexual Delay Discounting Task (previously referred to as the “Sexual Discounting Task”) was developed to assess the influence of delay on choices related to condom use in casual sex scenarios. Studies using the task in individuals with cocaine use disorders [ 30 – 31 ] reported several findings. First, individuals with cocaine use disorders generally indicated that they would be less likely to use condoms as the delay to condom availability increased. Second, participants discounted condom-protected sex more steeply for partners with whom they most vs. least wanted to have sex, and for partners they judged least vs. most likely to have an STI. Third, steeper discounting of condom-protected sex was significantly associated with higher rates of self-reported sexual HIV risk behavior. Fourth, the Sexual Delay Discounting Task showed good 1-week test-retest reliability. Together, these findings suggest the Sexual Delay Discounting Task has both external and internal validity among individuals with cocaine use disorders.

Despite the reliability and validity of the Sexual Delay Discounting Task within individuals with cocaine use disorders, there are no reports comparing delay discounting of condom-protected sex between individuals with cocaine use disorders and those who do not use cocaine. A recent study using the Sexual Delay Discounting Task demonstrated that opioid-dependent women discounted delayed condom-protected sex and monetary rewards more steeply than non-drug-using control women, and that participants in both groups discounted in an orderly manner that was sensitive to partner characteristics [ 32 ]. Moreover, a recent study in 18–24 year old youth found increased delay discounting of condom-protected sex to be significantly associated with greater self-reported drug use [ 33 ]. These findings suggest that steeper discounting may be related to higher rates of sexual HIV risk in drug-using populations, but it is unknown whether steeper discounting is related to the higher rates of sexual HIV risk behavior observed specifically among individuals who use cocaine. It is worthwhile to examine the relationship between the discounting of sexual outcomes and cocaine use because individuals who use cocaine have shown higher rates of sexual risk behavior [ 34 ] and greater delay discounting of monetary rewards [ 35 ] than individuals who use heroin. The present study therefore compared discounting of delayed sexual and monetary outcomes between individuals with cocaine use disorders and matched non-cocaine-using controls.

Beyond delay, at least one additional factor that may influence condom use is the probability of contracting an STI with unprotected sex. Indeed, one reason condoms are ever preferred is likely because they decrease the probability of aversive outcomes (i.e., STIs or unwanted pregnancy). Therefore, we also compared probability discounting of sexual and monetary outcomes between the two groups. With methods analogous to delay discounting, probability discounting tasks systematically examine how uncertainty influences an event’s value or impact on behavior [ 36 – 37 ]. While delay and probability discounting are only weakly correlated, the two processes have shown independent associations with clinically relevant behavior (i.e., gambling) and show differential effects of experimental manipulations (i.e. reward magnitude manipulations show directionally opposite effects on delay and probability discounting). Therefore, delay and probability discounting likely represent separate behavioral processes (e.g., [ 38 – 41 ]). With respect to the influence of probability on sexual outcomes, previous studies using the Sexual Delay Discounting Task showed that a partner’s perceived likelihood of having an STI influenced delay discounting of condom use. However, these studies did not explicitly manipulate the probability of STI contraction. We developed the Sexual Probability Discounting Task to quantitatively examine how specified risk of STI contraction resulting from unprotected sex influences condom use. The reliably weak correlations observed in previous studies between delay and probability discounting of monetary rewards also prompted us to examine correlations between discounting in the Sexual Delay and Sexual Probability Discounting tasks.

Materials and Methods

Ethics statement.

Study procedures were approved by the Johns Hopkins Medicine Institutional Review Board 3 (Office for Human Research Protections Registration #00001656). The study was conducted according to the principles expressed in the Declaration of Helsinki. Written informed consent was obtained from the participants.

Participants

Volunteers were recruited using flyers, Internet, newspaper, and radio advertisements, and word of mouth referral. Inclusion criteria for both the cocaine use disorder (Cocaine) and non-cocaine-using (Control) groups included being at least 18 years of age, having at least an 8 th grade reading level, and reporting having vaginal or anal intercourse with another person during their lifetime. Participants in the Cocaine group met Diagnostic and Statistical Manual of Mental Disorders (4 th edition, DSM-IV) criteria for cocaine abuse or dependence, whereas participants in the Control group reported no lifetime use of cocaine. Participants in both groups could meet criteria for abuse for drugs other than cocaine, but could not meet dependence criteria for other drugs (excluding nicotine and caffeine). Exclusion criteria for both groups included self-reported serious head trauma, dementia, significant cognitive impairment, or diagnosis of major psychiatric disorder besides substance abuse/dependence.

After an initial telephone screening assessing basic inclusion/exclusion criteria, initially qualified participants were scheduled for an in-person screening. If qualified, participants remained in the laboratory for approximately four hours to complete a variety of behavioral tasks. During the in-person screening, participants provided informed consent and a urine sample to test for drug use. Participants also completed a demographic questionnaire, a verbal intelligence assessment (Quick Test) [ 42 ], a reading comprehension assessment (Wide Range Achievement Test) [ 43 ], a lifetime drug use questionnaire, and a checklist to assess current and past drug abuse and dependence [ 44 ]. Occurrence and frequency of HIV risk behaviors in the past month were assessed using the HIV Risk-Taking Behavior Scale (HRBS) [ 45 – 46 ]. The HRBS is a psychometrically reliable and valid questionnaire featuring 11 items scored on a 6-point scale (scores of 0–5, with higher scores indicating higher risk) pertaining to injection drug use (6 items) and sexual risk behavior (5 items). Only scores on the sexual risk behavior subscale, which assessed participants’ number of sexual partners in the past month, frequency of condom use with regular and casual partners and when paid for sex, and frequency of anal sex, were compared between groups. Several personality measures and behavioral tasks were also obtained but are not relevant to the present analyses.

As in our previous sexual discounting studies [ 30 – 33 ], participants then used a computer to view 60 individually-presented color photographs of diverse, clothed people (30 male, 30 female) and were asked to select photographs of individuals that they would consider having casual sex with based on physical appearance. The photographs were assembled from a variety of publicly available online repositories to provide a range of physical appearances that would be conducive to the multiple hypothetical partner conditions described below. Before viewing photographs (and before subsequent sexual tasks), participants were instructed to pretend that they were not in a committed relationship. Next, participants identified from the subset of initially selected photographs the person they (1) most wanted to have sex with, (2) least wanted to have sex with, (3) judged was most likely to have an STI, and (4) judged was least likely to have an STI. A single photograph could be assigned to multiple partner conditions, but not for “most” and “least” categories within one dimension. Participants who selected fewer than two photographs ( n = 2 potential Control group participants) were disqualified from further participation. Female participants who selected only photographs of females would have been disqualified from further participation because the risk of HIV infection from female-female sex is extremely low [ 47 ], although this criterion resulted in no exclusions for the current study. Participants who were disqualified from further participation were compensated $30.

Qualified participants were then trained on using a visual analog scale. Next, they completed the four discounting tasks described below, in addition to other decision-making tasks not relevant to the present analyses. Monetary tasks were administered before sexual tasks, and delay tasks were administered before probability tasks.

Sexual Delay Discounting Task.

Delay discounting of condom-protected sex was assessed using a computerized version of the Sexual Delay Discounting Task [ 32 ]. At the beginning of each of the four partner conditions (presented in a pseudo-randomized order), the participant was shown the relevant photograph and instructed to imagine the person was interested in having sex now, that there was no chance of pregnancy, and that a condom was readily and immediately available. The participant indicated his/her likelihood of using a condom by clicking on a visual analog scale that ranged from “I will definitely have sex with this person without a condom” (0%) to “I will definitely have sex with this person with a condom” (100%). In subsequent trials, the participant was asked to rate his/her likelihood of waiting a given delay (ascending order; 1 hour, 3 hours, 6 hours, 1 day, 1 week, 1 month, and 3 months) to have sex with a condom. The visual analog scales for these trials ranged from “I will definitely have sex with this person now without a condom” (0%) to “I will definitely wait [delay] to have sex with this person with a condom” (100%). In the event that a photograph was assigned to multiple partner conditions, the participant completed the 8-trial series only once for that photograph.

Monetary Delay Discounting Task.

Delay discounting of hypothetical money was assessed using a computerized task used previously [ 19 – 20 , 48 – 53 ]. Participants made choices between smaller amounts of money delivered immediately vs. a larger amount ($100) delivered after a delay. Unlike the Sexual Delay Discounting Task, which compared groups with respect to decisions involving each of four hypothetical partners, for the Monetary Delay Discounting only a single condition (i.e., reinforcer magnitude) was evaluated for group differences. Participants were instructed to treat choices as if the outcomes were real and that they should take their financial circumstances into account when making their choices. Based on the pattern of a participant’s choices, the task algorithm calculated an indifference point (i.e., a smaller amount of money subjectively equivalent to delayed $100) at each of 7 delays: 1 day, 1 week, 1 month, 6 months, 1 year, 5 years, and 25 years (see [ 49 ] and [ 54 ] for a description of the algorithm used to determine indifference points). Although some delays were common to both discounting tasks (i.e., 1 day, 1 week, and 1 month), the standard range of delays typically assessed in the Monetary Delay Discounting Task exceeded that of the Sexual Delay Discounting Task. The order in which delays were assessed (ascending or descending) was randomly determined.

Sexual Probability Discounting Task.

In this task, participants were asked to imagine in each decision that having sex with a photographed individual was associated with a specified risk of contracting an STI. We administered the task only for the “most want to have sex with” and “least want to have sex with” partner conditions to avoid explicitly confounding our experimental manipulation of risk with the perceived risk of partners. Presentation order was identical to that of the Sexual Delay Discounting Task. At the beginning of each partner condition, a research assistant placed a printed copy of the relevant photograph (21.59 cm x 27.94 cm) on the desk in front of the participant and reminded her or him to imagine the person was interested in having sex now, and that there was no chance of pregnancy. For the first trial, the research assistant read text shown to the participant specifying that if he/she did not use a condom, then there was a 1 in 1 (100%) chance of contracting an STI from the photographed individual. A visual analog scale located below the prompt ranged from “I will definitely have sex with this person without a condom” to “I will definitely have sex with this person with a condom” for this and all other probability trials (see [ 55 ] for use of a visual analog scale to assess probability discounting). No trials involved delays. In all trials, risk was described both as odds in favor and percent chance of contracting an STI. Other risk values assessed (in descending order) were 1 in 3 (33%), 1 in 13 (8%), 1 in 100 (1%), 1 in 400 (0.25%), 1 in 700 (0.14%), 1 in 2,000 (0.05%), and 1 in 10,000 (0.01%).

Monetary Probability Discounting Task.

Probability discounting of hypothetical money ($100) was assessed using a computerized task used in a previous study [ 52 ]. Choices were between smaller amounts of money delivered immediately with 100% certainty vs. a larger amount ($100) delivered immediately, but with a specified probability of delivery. Indifference points were obtained at each of 7 probabilities of receiving $100: 99%, 90%, 75%, 50%, 25%, 10%, and 1%. Hypothetical rewards instructions and the choice-adjustment algorithm were identical to the Monetary Delay Discounting Task. The order in which probability values were assessed (ascending or descending) was matched to the Monetary Delay Discounting Task for each participant.

HIV testing.

Upon completion of all experimental tasks, a 3.5 cc blood specimen was collected and sent to a commercial laboratory (Quest Diagnostics Incorporated, Baltimore, MD) for HIV-1/HIV-2 antibody testing. Immediately after the blood draw, participants were discharged and compensated $75 for study completion.

Evaluation of group characteristics.

Group characteristics were compared using independent-samples t -tests for continuous variables and Fisher’s exact tests for categorical variables. Groups were matched (i.e., no significant or trend-level differences, t s[ 45 ] ≤ 1.62, p s ≥. 11) on the following characteristics: age, sex, race, ethnicity, marital status, educational attainment, monthly income, Quick Test score, and cigarettes per day.

Orderliness of discounting data.

Likelihood values from the sexual tasks were calculated as a proportion of the total length of the visual analog scale and indifference points from the monetary tasks were expressed as a proportion of $100. Nonsystematic discounting data were identified according to criteria adapted from a previously established algorithm [ 56 – 57 ]. For the sexual tasks, two criteria were used to identify nonsystematic data. First, starting with the second delay value (1 hour) or probability value (33%), the likelihood of condom use at a given delay or probability could not exceed the immediately preceding likelihood by more than 0.2. Second, the likelihood of condom use at the longest delay (3 months) or smallest probability (0.01%) could not exceed the likelihood when a condom was immediately available or the risk of contracting an STI was 100% by more than 0.1. For the monetary tasks, the analogous two criteria were applied as well as an additional criterion, which specified that the final indifference point could not be greater than 0.9. Participants whose data violated one or more of the aforementioned criteria in a condition were excluded in analyses of that condition.

Discounting data analysis.

Groups were compared using extra sums-of-squares F tests analyzing all individual participant data as previously described [ 20 , 58 ] (GraphPad Prism version 6.05 for Windows, GraphPad Software, La Jolla, CA). For delay tasks, these analyses regressed proportion likelihood of condom use (sexual tasks) or indifference points (monetary tasks) against delay (hours). For probability tasks, these measures were regressed against odds against, which were calculated as (1/ p )-1, wherein p is the probability in favor of an event’s occurrence [ 37 ]. Nonlinear regressions were performed using a two-parameter hyperbolic discounting equation [ 59 – 61 ]: Proportion likelihood or indifference point = 1/(1+ r X) s , wherein X is the task-specific independent variable (i.e., either hours or odds against), r is a parameter proportional to discounting rate, and s is a parameter describing the nonlinear scaling of the dependent variable (monetary amount or likelihood of condom use) and delay or odds against. The free parameters r and s were unconstrained in regression analyses. The F tests compared the difference in nonlinear regression model error when free parameters were shared (i.e., one curve best-fit to all discounting data collapsed across groups) vs. when free parameters were unshared (i.e., a separate best-fit for each group). A significant p value indicates significantly less error when model parameters are unshared, indicating that the groups differ.

In addition to analyzing raw group likelihood data from the Sexual Delay Discounting and Sexual Probability Discounting tasks, we also compared 0-delay (and 100%-probability) likelihood values, which were non-normally distributed, between Cocaine and Control groups using Mann-Whitney U tests.

For the Sexual Delay Discounting Task we also isolated the effect of delay on likelihood of waiting to use delayed condoms (i.e., discounting) from differences in preference for using immediately available condoms (0-delay). Specifically, individual participant raw likelihood values for each condition were standardized by dividing each non-zero delay trial likelihood value by its respective 0-delay trial value. In the event that a standardized likelihood value exceeded 1 (i.e., a non-zero delay trial likelihood exceeded its respective 0-delay trial value), the value was replaced with a value of 1. Participants who reported zero likelihood of using an immediately available condom were excluded from standardized data analyses because the relative effect of delay on reward value is undefined if the initial value is zero. An identical procedure was employed to standardize the Sexual Probability Discounting Task data with respect to 100%-probability trial data. Extra sums-of-squares F tests were used to compare standardized discounting data between Cocaine and Control groups in each condition.

Correlations among all discounting measures (using standardized likelihood in the sexual tasks) were conducted using an area-under-the-curve (AUC; [ 62 ]) metric. Spearman’s rank-order correlations were calculated because AUC values were non-normally distributed. The criterion for significance in all tests was p <. 05.

Sample characteristics

Table 1 presents characteristics of the Cocaine ( n = 23) and Control ( n = 24) groups. There were no significant differences between the groups with the exception of substance use, self-reported sexual risk behavior, and HIV variables. All 23 participants in the Cocaine group met criteria for cocaine abuse, and 20 participants also met criteria for cocaine dependence. Eighteen participants (78%) reported inhalation (smoking) as their preferred route of cocaine administration, and 4 (17%) and 1 (4%) participants preferred intranasal (snorting) and intravenous routes, respectively. Compared to the Control group, the Cocaine group reported significantly greater alcohol and cannabis use (number of past-year users; days used per month), and included significantly more participants meeting diagnostic abuse criteria for these two drugs. There were no significant group differences in measures of opioid and cigarette use. Participants in the Cocaine group also had significantly higher HRBS sexual risk subscale scores, and had a greater proportion of HIV-positive participants, than the Control group. Five of six Cocaine group participants with HIV knew they were HIV-positive prior to participating in the study and completing discounting assessments. Excluding these five HIV-positive participants did not alter whether 13 of 14 between-groups comparisons of discounting data met significance or not (the exception being a significant between-group difference in the standardized analysis of the “least likely to have an STI” partner condition, one that was not significant in the full sample). We therefore we report results from the full sample. In preparation for the sexual discounting tasks, 13 men in the Control group and 11 men in the Cocaine group selected exclusively female partners, while 7 women in the Control group and 10 women in the Cocaine group selected exclusively male partners. One man in the Control group and 2 men in the Cocaine group selected exclusively male partners, and 1 man in the Control group selected both male and female partners. Two women in the Control group selected both male and female partners.

thumbnail

  • PPT PowerPoint slide
  • PNG larger image
  • TIFF original image

https://doi.org/10.1371/journal.pone.0128641.t001

Sexual discounting data from partner conditions in which female participants selected a female partner ( n = 4; 1 partner condition for 1 participant and 3 partner conditions for 1 participant) were excluded prior to analysis because the risk of female-female HIV transmission via sexual behavior is extremely low [ 47 ].

Orderliness of data

Across all four partner conditions of the Sexual Delay Discounting Task, 77% and 85% of discounting functions were systematic for Cocaine and Control groups, respectively. Similarly, 87% (Cocaine) and 88% (Control) of Monetary Delay Discounting Task functions were systematic.

Across both partner conditions of the Sexual Probability Discounting Task, 98% (Cocaine) and 100% (Control) of discounting functions were systematic and 100% (Cocaine) and 96% (Control) of Monetary Probability Discounting Task functions were systematic.

Excluding nonsystematic data and data that could not be standardized (because of a zero initial likelihood of condom use) resulted in n that differed somewhat from analysis to analysis, however, these exclusions did not alter whether any group difference reported in Table 1 reached significance or a trend toward significance.

Between-group comparisons of discounting data

Sexual delay discounting\..

Reported likelihood of waiting to use a condom decreased as a function of delay to condom availability in all four Sexual Delay Discounting Task partner conditions for both groups. Significant group differences (i.e., lower likelihood of waiting to use a condom among Cocaine participants than among Control participants) were observed in two partner conditions: the “most want to have sex with” partner condition [Cocaine n = 19, Control n = 19, F (2, 300) = 5.81, p <. 01], and the “least want to have sex with” partner condition [Cocaine n = 13, Control n = 20, F (2, 260) = 6.38, p <. 01].

Fig 1 (left column) shows best-fit curves to mean standardized likelihood of condom use in Cocaine and Control groups within each partner condition of the Sexual Delay Discounting Task. The right column of Fig 1 displays these same data, except with delay to condom availability expressed ordinally to facilitate visual inspection at short delays. When data were standardized to isolate the effect of delay, participants in the Cocaine group discounted significantly more steeply than those in the Control group in two of four partner conditions: the “most want to have sex with” partner condition [Cocaine n = 14, Control n = 15, F (2, 228) = 5.51, p <. 01], and the “least want to have sex with” partner condition [Cocaine n = 13, Control n = 19, F (2, 252) = 8.56, p <. 001]. Discounting did not differ significantly between Cocaine and Control groups with the “most likely to have an STI” partner condition [Cocaine n = 18, Control n = 19, F (2, 292) = 0.55, p =. 58], or the “least likely to have an STI” partner condition [Cocaine n = 15, Control n = 16, F (2, 244) = 2.02, p =. 13]. No significant group differences in likelihood of using an immediately available condom were observed ( U s ≥ 116.5, p s ≥. 25).

thumbnail

Left column: Best-fit curves to mean standardized likelihood of condom use (proportion of visual analog scale) in each of the Sexual Delay Discounting Task partner conditions in the Control and Cocaine groups. Right column: Data from left column with delay to condom availability expressed ordinally on the x-axis. Error bars represent ± SEM .

https://doi.org/10.1371/journal.pone.0128641.g001

Monetary Delay Discounting.

The top row of Fig 2 shows best-fit curves to delay discounting data for hypothetical money (left graph; right graph shows data with delays expressed ordinally). Cocaine participants ( n = 20) discounted delayed $100 significantly more steeply than Control participants [ n = 21, F (2, 283) = 18.29, p <. 0001].

thumbnail

Top row: Best-fit curves to mean indifference points (proportion of $100) from the Monetary Delay Discounting Task (left graph; right graph shows data with delay expressed ordinally). Bottom row: Best-fit curves to mean indifference points (proportion of $100) from the Monetary Probability Discounting Task (left graph; right graph shows data with odds against expressed ordinally). Error bars represent ± SEM .

https://doi.org/10.1371/journal.pone.0128641.g002

Sexual Probability Discounting.

Reported likelihood of using an immediately available condom decreased with increased odds against contracting an STI for both partner conditions in both groups. However, no significant differences in discounting between Cocaine and Control groups were detected in unstandardized likelihood of condom use in the Sexual Probability Discounting Task for either the “most want to have sex with” partner condition [Cocaine n = 23, Control n = 24, F (2, 372) = 2.30, p =. 10] or for the “least want to have sex with” partner condition [Cocaine n = 22, Control n = 23, F (2, 356) = 1.88, p =. 16].

Fig 3 (left column) shows best-fit curves to mean standardized likelihood of condom use in Cocaine and Control groups within each partner condition of the Sexual Probability Discounting Task. The right column of Fig 3 displays these same data, except with odds against contracting an STI expressed ordinally. Analyses of standardized data suggested a trend for those in the Cocaine group to discount condom-protected sex less steeply as STI risk decreased for the “most want to have sex with” partner [Cocaine n = 23, Control n = 24, F (2, 372) = 2.95, p =. 054], with no discounting difference for the “least want to have sex with” partner [Cocaine n = 22, Control n = 23, F (2, 356) = 1.72, p =. 18]. The groups also did not differ statistically with respect to likelihood of condom use when the risk of contracting an STI was 100% ( U s ≥ 226, p s ≥. 23).

thumbnail

Left column: Best-fit curves to mean standardized likelihood of condom use (proportion of visual analog scale) in each of the Sexual Probability Discounting Task partner conditions in the Cocaine and Control groups. Right column: Data from left column with odds against STI contraction expressed ordinally on the x-axis. Error bars represent ± SEM .

https://doi.org/10.1371/journal.pone.0128641.g003

Monetary Probability Discounting.

The bottom row of Fig 2 shows best-fit curves to probability discounting data for hypothetical money (left graph; right graph shows data with odds against expressed ordinally). Cocaine ( n = 23) and Control ( n = 23) group participants showed a significantly different pattern of discounting for a probabilistic $100 reward [ F (2, 318) = 5.24, p <. 01]. However, the bottom right panel of Fig 2 shows that one group did not consistently discount to a greater or lesser extent than the other group. At lower odds against receiving $100 (i.e., 0.01, 0.11, 0.33, 1), mean indifference points for the Control group tended to be higher than mean indifference points for the Cocaine group, whereas at higher odds against receiving $100 (i.e., 9, 99), the Cocaine group tended to be higher than the Control group.

Correlations Among Discounting Tasks

Table 2 shows Spearman rank correlations for sexual (using standardized likelihood) and monetary discounting tasks. Upon elimination of nonsystematic discounting data sets and sexual data sets showing zero likelihood of condom use under 0-delay or 100% probability of STI conditions, n ranged from 22 to 46 across these correlations. Within the Sexual Delay Discounting Task, discounting measures among partner conditions were positively and significantly correlated in 4 of 6 instances. Similarly, within the Sexual Probability Discounting Task, discounting between the two partner conditions was positively and significantly correlated. The Sexual Delay Discounting Task and the Sexual Probability Discounting Task were significantly and positively correlated in 7 of 8 partner conditions. Conversely, the Monetary Delay Discounting Task and Monetary Probability Discounting Task were not significantly correlated. The Monetary Delay Discounting Task was not significantly correlated with the Sexual Delay Discounting Task for any of the 4 partner conditions. Similarly, the Monetary Probability Discounting Task was not significantly correlated with either partner condition in the Sexual Probability Discounting Task. Finally, although the Monetary Probability Discounting Task was significantly and positively correlated with the Sexual Delay Discounting Task for 2 of the 4 partner conditions, no significant relation was found between the Monetary Delay Discounting Task and either partner condition in the Sexual Probability Discounting Task.

thumbnail

https://doi.org/10.1371/journal.pone.0128641.t002

This study systematically examined discounting of delayed and probabilistic sexual and monetary outcomes among individuals with cocaine use disorders and demographically-matched controls. First, we found that individuals with cocaine use disorders discounted significantly more steeply than controls in two of the four Sexual Delay Discounting Task partner conditions, as well as in the Monetary Delay Discounting Task. Second, in the novel Sexual Probability Discounting Task, both groups showed an orderly effect in which odds against contracting an STI systematically decreased the likelihood of using an immediately available condom. Third, no robust group differences in probability discounting of sexual outcomes or monetary rewards were found. Finally, correlations showed sexual and monetary results were unrelated, for both delay and probability discounting tasks. Each finding will be discussed in turn.

To date, no other study has examined whether individuals with cocaine use disorders discount delayed condom-protected sex more than matched controls. After controlling for each individual’s likelihood of condom use when no delay was involved, individuals with cocaine use disorders discounted delayed condom-protected sex significantly steeper than controls in two of four partner conditions. We suspect that the relatively high rates of reported condom use in the “most likely to have an STI” partner condition were responsible for the inability to detect group differences. Although likelihood of condom use was relatively lower in the “least likely to have an STI” partner condition, a significant group difference was not obtained, perhaps due in part to similar ratings of condom use likelihood between groups at delays shorter than 1 day. There were no significant differences between individuals with cocaine use disorders and controls in likelihood of using immediately available condoms in any of the four Sexual Delay Discounting Task partner conditions, demonstrating that the between-group differences in likelihood of using delayed condoms were truly driven by differential responses to delay (i.e., delay discounting). Had this study examined only preferences about using immediately available condoms, an entire dimension of increased HIV risk behavior would have been overlooked. This highlights the importance of delay discounting as a contributor to the high rates of HIV risk behavior. For the Monetary Delay Discounting Task, the finding of steeper discounting in individuals with cocaine use disorders replicates several previous findings [ 18 – 21 ], contributing to overall confidence in our study findings.

In the novel Sexual Probability Discounting Task, decreased odds of contracting an STI systematically decreased the likelihood of using an immediately available condom in both groups, suggesting that perceived STI risk has a lawful effect on condom use regardless of cocaine use history. Probability discounting of sexual outcomes has been shown in college students using tasks assessing choices between certain shorter durations vs. uncertain longer durations of sexual activity [ 63 – 64 ] or choices between certain “less than ideal” and uncertain “ideal” sexual outcomes, with idealness represented visually by line length [ 65 ]. These tasks assessed a theoretically important issue, the effects of uncertainty of a sexual act on its value, while our task assessed the clinically-relevant effect of STI uncertainty on condom use. Orderly probability effects in all three tasks speak to the robust effect of probability on sexual outcomes, regardless of whether a probabilistic reward [ 63 – 65 ] or a probabilistic punishment (hypothetical STI contraction in the current study) is being assessed.

In contrast to the delay discounting results, no robust between-group differences in probability discounting of either sexual or monetary outcomes were observed. The only between-group difference detected was in the shape of the monetary probability discounting function; there was no reliable between-group difference across the range of different probabilities assessed. A prior study showed no association between drug use and probability discounting of sexual outcomes (although low drug use in the sample was a limitation [ 64 ]). Our comparisons between individuals with cocaine use disorders and matched controls regarding STI risk and condom use further suggest no robust relations between drug use and probability discounting of sexual outcomes. This conclusion is consistent with the larger literature on probability discounting, which shows mixed results regarding relations between drug use and probability discounting of monetary gains [ 66 – 72 ], and no apparent relation between drug use and probability discounting of monetary losses [ 69 , 73 ]. Our findings therefore support the somewhat paradoxical conclusion that behavioral processes underlying risk-taking may be unrelated to drug use, and specifically cocaine use in the present study. Likewise, loss aversion (i.e., the tendency to overweight losses relative to equivalent gains [ 74 ]), or negativity bias [ 75 ] may be similarly unrelated to cocaine use disorders, given our observation that individuals with cocaine use disorders did not differ from matched controls in terms of their sensitivity to STI contraction risk (i.e., a loss), but did differ significantly in their sensitivity to delayed condom-protected sex (i.e., a gain). Collectively, our differential findings between delay and probability discounting highlight the importance of examining multiple behavioral processes in relation to clinically relevant behavior, and suggests that, with respect to sexual outcomes, steeper delay discounting of condom-protected sex, but not reduced sensitivity to STI probability, contributes to increased sexual HIV risk among individuals with cocaine use disorders.

Correlations showed sexual and monetary results were unrelated, for both delay and probability discounting. The sexual partner conditions were generally positively related within a single type of sexual task (delay or probability). Moreover, delay and probability discounting in the sexual tasks were generally positively related. These findings were in contrast to the lack of significant association between delay and probability discounting of monetary rewards, and in contrast to the lack of significant associations between the monetary and sexual results for either delay or probability discounting. The nonsignificant correlation (in the positive direction) between delay and probability discounting of money should be viewed in the context of mixed evidence on this relationship, with studies typically showing correlations in the positive direction but varying from weak to strong in correlation strength, and varying between significance and nonsignificance [ 40 – 41 , 54 , 76 – 77 ]. The predominantly nonsignificant correlations between discounting of delayed condom-protected sex and delayed money replicates previous findings [ 30 , 32 ]. One potential explanation for the relation between the delay and probability sexual tasks is that choice behavior involving waiting for a delayed condom and avoidance of STI contraction recruit related processes. This seems especially plausible given the risk of STI contraction implicit in all sexual situations, including the “most/least want to have sex with” partner conditions in the Sexual Delay Discounting Task.

With respect to probability discounting, it should be noted that, unlike the delay discounting tasks, the probability tasks assess events of opposite valences, with the risk of contracting an STI or receiving $100 representing a probabilistic loss (punishment) and gain (reward), respectively. As a result of this difference, a general tendency toward risk-taking would be evidenced by steep discounting of STI risk and shallow discounting of the monetary reward (and vice versa for risk-aversion), resulting in a negative correlation between these measures [ 78 ]. The fact that we did not observe significant negative correlations between these measures lends some support to the conclusion that sexual outcomes are discounted uniquely relative to the discounting of monetary outcomes.

These results add to growing evidence [ 30 – 32 , 64 , 79 ] showing domain specificity in discounting results. In other words, discounting results differ depending upon the type of outcome studied. Most human discounting studies have examined only money as the outcome, with the implicit but questionable assumption that decision making for monetary outcomes is indicative of decision making in clinically relevant domains such as substance use disorders. Replicating previous findings with respect to opioid-dependent women [ 32 ], the present results suggest that delay discounting of condom-protected sex and delay discounting of money are different processes, although individuals with cocaine use disorders discount both more steeply than matched controls. Similarly, results suggest that probability discounting of STI contraction and probability discounting of money are different processes, but unlike with delay discounting, these different probability discounting processes do not differ between individuals with cocaine use disorders and matched controls who did not use cocaine.

One limitation is that despite matching participants demographically with respect to age, sex, race, ethnicity, marital status, education, monthly income, intelligence test score, and cigarettes per day, participants in the Cocaine group and the Control group differed on substance use variables other than cocaine use. Specifically, participants in the Cocaine group showed significantly higher rates of alcohol and cannabis use relative to participants in the Control group. Moreover, participants in the Cocaine group showed significantly higher self-reported sexual risk behavior and were significantly more likely to be HIV positive. We believe these differences are to be expected and indicate that we studied a representative sample of individuals with cocaine use disorders [2–6; 18–21]. Indeed, the higher rates of sexual risk behavior and HIV infection in the Cocaine group highlight the behavioral problems associated with cocaine use disorders, which prompted the present study. Further, there is some evidence to suggest that cocaine use may be a stronger predictor of discounting than alcohol and cannabis use. For example, individuals with problematic cocaine use discount more steeply than those with problematic alcohol use [ 21 ], and the relationship between cannabis use and discounting is less robust than previous studies with other drugs [ 51 ]. Finally, differences in other drug use between the Cocaine group and Control group are comparable to differences that may have been present in previous studies examining discounting in individuals who use cocaine relative to a matched control group [ 18 – 21 ].

An additional limitation is that the various discounting tasks involved hypothetical rather than real outcomes. However, delay and probability discounting studies have generally shown similar results when using real and hypothetical money ([ 20 , 48 – 50 , 80 – 86 ] but see [ 87 – 88 ]). Moreover, the Sexual Delay Discounting Task has demonstrated reliability and relationships with self-reported sexual risk [ 30 – 31 , 33 ]. Another potential limitation is that discounting in the sexual tasks may have been affected not only by delay or probability, but also by the effort associated with condom use, whereas this was not the case in the monetary tasks. Although a potential effort-related confound could have contributed to the lack of significant correlations between the task types, attempts to control for this variable could jeopardize the external validity of the sexual tasks. Moreover, it is unlikely that effort influenced group differences in the Sexual Delay Discounting Task, because the Sexual Probability Discounting Task also involves the same potential effort-related confound, yet did not show robust group differences. Another shortcoming was our relatively small sample size. However, the orderliness of the data and the detection of between-groups differences suggest the sample was sufficient to detect meaningful results. Finally, although the present findings suggest an association with cocaine use disorders, the etiology of increased sexual HIV risk still remains unclear. To examine the potential contribution of cocaine pharmacology on sexual risk behavior, future research should examine the effects of acute cocaine administration on the Sexual Delay Discounting Task and the Sexual Probability Discounting Task in individuals who use cocaine.

The translational nature of this research contributes a novel perspective to the prevention of HIV sexual risk behavior among individuals with and without cocaine use disorders. Both delay and probability discounting may serve a diagnostic role, identifying individuals who are at risk for HIV or STI contraction (or transmission to others, as highlighted by the non-trivial percentage of our sample that was HIV-positive). Among individuals who exhibit high rates of sexual risk behavior and steeply discount delayed condom-protected sex, behavioral treatment strategies aimed at reinforcing condom carrying or training delay tolerance may reduce the likelihood of HIV and other STI transmission. Among individuals who exhibit high rates of sexual risk behavior and steeply discount the possibility of uncertain STI contraction, training that increases the perceived likelihood that partners may have an STI may also decrease HIV and other STI transmission. The present translational research on basic behavioral processes using clinically relevant decisions may therefore be leveraged to improve public health.

Acknowledgments

The authors would like to thank Natalie R. Bruner, Ph.D., Crystal Fridy, and Grant Glatfelter for assistance in conducting this study, and Leticia Nanda, CRNP for collecting blood samples and providing HIV counseling.

Author Contributions

Conceived and designed the experiments: MWJ PSJ. Performed the experiments: MWJ PSJ ESH. Analyzed the data: MWJ PSJ. Contributed reagents/materials/analysis tools: MWJ. Wrote the paper: MWJ PSJ ESH MMS.

  • 1. Substance Abuse and Mental Health Services Administration, Results from the 2013 National Survey on Drug Use and Health: Summary of National Findings, NSDUH Series H-48, HHS Publication No. (SMA) 14–4863. Rockville, MD: Substance Abuse and Mental Health Services Administration, 2014.
  • View Article
  • PubMed/NCBI
  • Google Scholar
  • 7. Centers for Disease Control and Prevention: HIV and AIDS in the United States fact sheet, 2010. Department of Health and Human Services, Centers for Disease Control and Prevention. http://www.cdc.gov/hiv/resources/factsheets/us.htm .
  • 14. Winhusen TM, Somoza EC, Lewis DF, Kropp F, Theobald J, Elkashef A. An evaluation of substance abuse treatment and HIV education on safe sex practices in cocaine dependent individuals. ISRN Addiction. 2014; Article ID 912863.
  • 43. Wilkinson GS, 1993. WRAT-3: Wide range achievement test administration manual. Wilmington, DE: Wide Range.
  • 58. Motulsky HJ, Christopoulos A. Fitting models to biological data using linear and nonlinear regression: A practical guide to curve fitting. San Diego, CA: GraphPad Software, Inc.; 2003.
  • 61. Rachlin H. Judgment, decision, and choice: A cognitive/behavioral synthesis. WH Freeman/Times Books/Henry Holt Co.; 1989.

U.S. flag

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

  • Publications
  • Account settings
  • Advanced Search
  • Journal List
  • J Exp Anal Behav
  • v.71(2); 1999 Mar

Delay or probability discounting in a model of impulsive behavior: effect of alcohol.

Little is known about the acute effects of drugs of abuse on impulsivity and self-control. In this study, impulsivity was assessed in humans using a computer task that measured delay and probability discounting. Discounting describes how much the value of a reward (or punisher) is decreased when its occurrence is either delayed or uncertain. Twenty-four healthy adult volunteers ingested a moderate dose of ethanol (0.5 or 0.8 g/kg ethanol: n = 12 at each dose) or placebo before completing the discounting task. In the task the participants were given a series of choices between a small, immediate, certain amount of money and $10 that was either delayed (0, 2, 30, 180, or 365 days) or probabilistic (i.e., certainty of receipt was 1.0, .9, .75, .5, or .25). The point at which each individual was indifferent between the smaller immediate or certain reward and the $10 delayed or probabilistic reward was identified using an adjusting-amount procedure. The results indicated that (a) delay and probability discounting were well described by a hyperbolic function; (b) delay and probability discounting were positively correlated within subjects; (c) delay and probability discounting were moderately correlated with personality measures of impulsivity; and (d) alcohol had no effect on discounting.

The Full Text of this article is available as a PDF (210K).

Selected References

These references are in PubMed. This may not be the complete list of references from this article.

  • Ainslie G. Specious reward: a behavioral theory of impulsiveness and impulse control. Psychol Bull. 1975 Jul; 82 (4):463–496. [ PubMed ] [ Google Scholar ]
  • de Wit H, Doty P. Preference for ethanol and diazepam in light and moderate social drinkers: a within-subjects study. Psychopharmacology (Berl) 1994 Aug; 115 (4):529–538. [ PubMed ] [ Google Scholar ]
  • Doty P, de Wit H. Effect of setting on the reinforcing and subjective effects of ethanol in social drinkers. Psychopharmacology (Berl) 1995 Mar; 118 (1):19–27. [ PubMed ] [ Google Scholar ]
  • Doty P, Kirk JM, Cramblett MJ, de Wit H. Behavioral responses to ethanol in light and moderate social drinkers following naltrexone pretreatment. Drug Alcohol Depend. 1997 Aug 25; 47 (2):109–116. [ PubMed ] [ Google Scholar ]
  • Doty P, Zacny JP, de Wit H. Effects of ondansetron pretreatment on acute responses to ethanol in social drinkers. Behav Pharmacol. 1994 Aug; 5 (4-5):461–469. [ PubMed ] [ Google Scholar ]
  • Duffy JD. The neurology of alcoholic denial: implications for assessment and treatment. Can J Psychiatry. 1995 Jun; 40 (5):257–263. [ PubMed ] [ Google Scholar ]
  • Ericksen KP, Trocki KF. Behavioral risk factors for sexually transmitted diseases in American households. Soc Sci Med. 1992 Apr; 34 (8):843–853. [ PubMed ] [ Google Scholar ]
  • Eysenck SB, Eysenck HJ. Impulsiveness and venturesomeness: their position in a dimensional system of personality description. Psychol Rep. 1978 Dec; 43 (3 Pt 2):1247–1255. [ PubMed ] [ Google Scholar ]
  • Fendrich M, Mackesy-Amiti ME, Goldstein P, Spunt B, Brownstein H. Substance involvement among juvenile murderers: comparisons with older offenders based on interviews with prison inmates. Int J Addict. 1995 Sep; 30 (11):1363–1382. [ PubMed ] [ Google Scholar ]
  • Fromme K, Katz E, D'Amico E. Effects of alcohol intoxication on the perceived consequences of risk taking. Exp Clin Psychopharmacol. 1997 Feb; 5 (1):14–23. [ PubMed ] [ Google Scholar ]
  • Hingson R, Strunin L, Berlin B. Acquired immunodeficiency syndrome transmission: changes in knowledge and behaviors among teenagers, Massachusetts statewide surveys, 1986 to 1988. Pediatrics. 1990 Jan; 85 (1):24–29. [ PubMed ] [ Google Scholar ]
  • Lau MA, Pihl RO, Peterson JB. Provocation, acute alcohol intoxication, cognitive performance, and aggression. J Abnorm Psychol. 1995 Feb; 104 (1):150–155. [ PubMed ] [ Google Scholar ]
  • Martin WR, Sloan JW, Sapira JD, Jasinski DR. Physiologic, subjective, and behavioral effects of amphetamine, methamphetamine, ephedrine, phenmetrazine, and methylphenidate in man. Clin Pharmacol Ther. 1971 Mar-Apr; 12 (2):245–258. [ PubMed ] [ Google Scholar ]
  • McCusker J, Koblin B, Lewis BF, Sullivan J. Demographic characteristics, risk behaviors, and HIV seroprevalence among intravenous drug users by site of contact: results from a community-wide HIV surveillance project. Am J Public Health. 1990 Sep; 80 (9):1062–1067. [ PMC free article ] [ PubMed ] [ Google Scholar ]
  • Mischel W, Grusec J. Waiting for rewards and punishments: effects of time and probability on choice. J Pers Soc Psychol. 1967 Jan; 5 (1):24–31. [ PubMed ] [ Google Scholar ]
  • Myerson J, Green L. Discounting of delayed rewards: Models of individual choice. J Exp Anal Behav. 1995 Nov; 64 (3):263–276. [ PMC free article ] [ PubMed ] [ Google Scholar ]
  • Rachlin H, Castrogiovanni A, Cross D. Probability and delay in commitment. J Exp Anal Behav. 1987 Nov; 48 (3):347–353. [ PMC free article ] [ PubMed ] [ Google Scholar ]
  • Rachlin H, Green L. Commitment, choice and self-control. J Exp Anal Behav. 1972 Jan; 17 (1):15–22. [ PMC free article ] [ PubMed ] [ Google Scholar ]
  • Rachlin H, Raineri A, Cross D. Subjective probability and delay. J Exp Anal Behav. 1991 Mar; 55 (2):233–244. [ PMC free article ] [ PubMed ] [ Google Scholar ]
  • Richards JB, Mitchell SH, de Wit H, Seiden LS. Determination of discount functions in rats with an adjusting-amount procedure. J Exp Anal Behav. 1997 May; 67 (3):353–366. [ PMC free article ] [ PubMed ] [ Google Scholar ]
  • Steele CM, Southwick L. Alcohol and social behavior I: The psychology of drunken excess. J Pers Soc Psychol. 1985 Jan; 48 (1):18–34. [ PubMed ] [ Google Scholar ]
  • Virkkunen M, Linnoila M. Serotonin in early onset, male alcoholics with violent behaviour. Ann Med. 1990; 22 (5):327–331. [ PubMed ] [ Google Scholar ]
  • Welte JW, Abel EL. Homicide: drinking by the victim. J Stud Alcohol. 1989 May; 50 (3):197–201. [ PubMed ] [ Google Scholar ]

Group I metabotropic glutamate receptor antagonists impair discriminability of reinforcer magnitude, but not risky choice, in a probability-discounting task

Affiliations.

  • 1 Department of Psychological Science, Northern Kentucky University, USA. Electronic address: [email protected].
  • 2 Department of Psychological Science, Northern Kentucky University, USA.
  • PMID: 30831139
  • PMCID: PMC6431275
  • DOI: 10.1016/j.bbr.2019.02.047

The glutamatergic system has been identified as an important mediator of risky choice. However, previous studies have focused primarily on ionotropic glutamate receptors (e.g., NMDA receptors). Little research has examined the contribution of metabotropic glutamate receptors (mGluRs) on risky choice. The goal of the current experiment was to determine the effects of mGluR 1 and mGluR 5 antagonism on risky choice as assessed in probability discounting (PD). Male Sprague Dawley rats (n = 24) were trained in PD, in which consistently choosing a large, probabilistic reward (LR) reflects risky choice. For half of the rats, the odds against (OA) receiving the LR increased across blocks of trials, whereas the OA decreased across the session for half of the rats. Following training, rats received injections of the mGluR 1 antagonist JNJ 16,259,685 (JNJ; 0, 0.1, 0.3, or 1.0 mg/kg; i.p) and the mGluR 5 antagonist MTEP (0, 1.0, 3.0, or 10.0 mg/kg; i.p.). Regardless of which schedule was used, JNJ and MTEP decreased preference for the LR when its delivery was guaranteed. In contrast to delay discounting, in which blocking the mGluR 1 has been shown to alter impulsive choice, these results show that the Group I mGluR family does not selectively alter risky choice. Instead, blocking these receptors appears to impair discriminability of reinforcers of varying magnitudes in PD.

Keywords: Discriminability of reinforcer magnitude; Metabotropic glutamate receptor; Probability discounting; Rat; Risky choice; Sensitivity to probabilistic reinforcement.

Copyright © 2019 Elsevier B.V. All rights reserved.

Publication types

  • Research Support, N.I.H., Extramural
  • Research Support, Non-U.S. Gov't
  • Choice Behavior / drug effects*
  • Choice Behavior / physiology
  • Conditioning, Operant / drug effects
  • Delay Discounting / drug effects*
  • Delay Discounting / physiology
  • Impulsive Behavior / drug effects
  • Probability
  • Pyridines / pharmacology
  • Quinolines / pharmacology
  • Rats, Sprague-Dawley
  • Receptors, Metabotropic Glutamate / antagonists & inhibitors
  • Receptors, N-Methyl-D-Aspartate / antagonists & inhibitors
  • Reinforcement, Psychology
  • Risk-Taking
  • Thiazoles / pharmacology
  • (3,4-dihydro-2H-pyrano(2,3)b-quinolin-7-yl)-(cis-4-methoxycyclohexyl) methanone
  • 3-((2-methyl-1,3-thiazol-4-yl)ethynyl)pyridine
  • Receptors, Metabotropic Glutamate
  • Receptors, N-Methyl-D-Aspartate

Grants and funding

  • P20 GM103436/GM/NIGMS NIH HHS/United States

IMAGES

  1. Probabilistic Discounting Task design. A Cost/benefit contingencies

    probability discounting task

  2. Choice performance in the probability discounting task following

    probability discounting task

  3. Experimental design for the delay and probability discounting tasks

    probability discounting task

  4. Flow chart of a trial on the probability discounting task (ITI

    probability discounting task

  5. Clark Lab

    probability discounting task

  6. Normalized discount rates in the probability discounting tasks

    probability discounting task

VIDEO

  1. How to expand your PROBABILITY to SUCCESS 💸

  2. Probability distribution

  3. Identify high probability setups

  4. Probability distributions

  5. Probability distribution

  6. PERFORMANCE TASK IN STATISTICS AND PROBABILITY

COMMENTS

  1. Sequential delay and probability discounting tasks in mice reveal

    Probability discounting Modeling Touchscreen Mice 1. Introduction Delay discounting tasks measure value assessments against a temporal cost, while probability discounting tasks measure value assessments against risky reward [21], [40].

  2. Delay discounting: Concepts and measures.

    The present review evaluated tasks of delay discounting, their features, measures of evaluation and anomalies, and some variables that can affect delay discounting results and applications in the study of individual and intra-individual differences. delay discounting, decision making, impulsiveness, experimental tasks

  3. Probability Discounting of Gains and Losses: Implications for Risk

    Probability discounting refers to the observation that a probabilistic gain is usually considered to be worth less than the same amount of gain available for certain.

  4. An Investigation of Delay and Probability Discounting in Hoarding

    A commonly used and well-validated measure of impulsivity is the delay and probability discounting task, which consists of making decisions about receiving monetary rewards after varying delay intervals and delivery probabilities.

  5. Probability discounting of gains and losses: Implications for risk

    Sixty college students performed three discounting tasks: probability discounting of gains, probability discounting of losses, and delay discounting of gains. Each task used an adjusting-amount procedure, and participants' choices affected the amount and timing of their remuneration for participating. Both group and individual discounting functions from all three procedures were well fitted ...

  6. Discounting delayed and probabilistic rewards: Processes and traits

    Similarly, probability discounting refers to the fact that the subjective value of a reward decreases as the odds against receiving it increase (i.e., as the probability of its receipt decreases).

  7. Delay and Probability Discounting

    It summarizes findings regarding the discounting of delayed rewards and the discounting of probabilistic rewards from the perspective of this framework, and discusses the similarities and differences between these two types of discounting when the outcomes are positive.

  8. Effects of Amount on Probability Discounting: A Replication and

    Sixty-four undergraduate psychology students completed four probability discounting tasks that differed in the amount of the probabilistic outcome ($20, $3,000, $50,000, and $500,000). In an amount-independent hyperboloid model, the exponent () systematically increased with reward amount. These systematic changes were not explained by the model.

  9. Gain-Loss Asymmetry in Experiential Probability Discounting

    Probability discounting is loss in reinforcer value as a function of uncertainty. In typical tasks measuring probability discounting, participants repeatedly choose between a smaller, certain amount and a larger amount at one of several probabilities, and do not experience the outcome they select. Most participants show a gain-loss asymmetry, discounting gains more steeply than losses. We ...

  10. Moderate Stability among Delay, Probability, and Effort Discounting in

    Thus, probability discounting refers to the process by which an outcome loses value as the odds against its receipt increases (Rachlin et al., 1991 ), and steep probability discounting reflects a relative preference for smaller, certain rewards (i.e., less risk-taking, more risk aversion).

  11. A systematic review and meta-analysis on the clinical ...

    Wang Z et al (2019) Altered brain functional networks in Internet gaming disorder: independent component and graph theoretical analysis under a probability discounting task. Spectrums . 24 , 544-556

  12. Neural Dynamics of the Combined Discounting of Delay and Probability

    Using the event-related brain potentials (ERPs) technique, we designed a novel paradigm to investigate neural processes related to the combined discounting of delay and probability during the evaluation of a delayed risky reward. ERP results suggested distinct temporal dynamics for delay and probability processing during combined discounting.

  13. Pramipexole-Induced Increased Probabilistic Discounting: Comparison

    This criterion was held for phases 6-9 (each of which employed the probability discounting task), and overall, <2% of the blocks were excluded. Phase 7, (±)PPX treatment. At 1 day following the ...

  14. Moderate Stability among Delay, Probability, and Effort Discounting in

    The stability of delay discounting across time has been well-established. However, limited research has examined the stability of probability discounting, and no studies of the stability of effort discounting are available. The present study assessed the steady-state characteristics of delay, probability, and effort discounting tasks across time with hypothetical rewards in humans, as well as ...

  15. Adolescents' performance on delay and probability discounting tasks

    Delay/probability discounting task (Richards et al., 1999) On each trial, participants chose between an immediate amount of money or $10 available after a delay (i.e., 'Would you rather have $2 now or $10 in 30 days?'). On probability trials, the choice was between a certain amount of money or $10 available with a given probability.

  16. Development of a Probability Discounting Task of Communication for

    The present study aimed to develop and validate the Probability Discounting for Communication (PDC) task, a behavioral measure of risk taking that characterizes decreasing subjective value of ...

  17. Frontiers

    Our probability discounting task tested rats' preference for a lever that led to a large reward with varying probability (the risky option) or a lever that lead to a small reward 100% of the time (the certain option). The probabilities associated with the risky lever were 100%, 50%, 25%, and 12.5%. The central question was whether aged rats ...

  18. Discounting of Delayed and Probabilistic Losses Over a Wide Range of

    From an individual-differences perspective, researchers are seeking to understand the extent to which an individual's tendency to discount the value of outcomes that are delayed or probabilistic reflects a single underlying trait (e.g., impulsivity) or whether multiple traits are involved.

  19. Delay and Probability Discounting of Sexual and Monetary ...

    Monetary Probability Discounting Task. Probability discounting of hypothetical money ($100) was assessed using a computerized task used in a previous study . Choices were between smaller amounts of money delivered immediately with 100% certainty vs. a larger amount ($100) delivered immediately, but with a specified probability of delivery.

  20. Impaired risk evaluation in people with Internet gaming disorder: fMRI

    10.1016/j.pnpbp.2014.08.016 This study examined how Internet gaming disorder (IGD) subjects modulating reward and risk at a neural level under a probability-discounting task with functional magnetic resonance imaging (fMRI).

  21. Clocks & Sleep

    We discuss task designs and model validity. Our second objective was to apply this knowledge and perform a pilot study on the effect of sleep deprivation. We trained and tested male Wistar rats on a probability discounting task; a "safe" lever always resulted in 1 reward, a "risky" lever resulted in 4 or no rewards.

  22. Delay or probability discounting in a model of impulsive behavior

    Discounting describes how much the value of a reward (or punisher) is decreased when its occurrence is either delayed or uncertain. Twenty-four healthy adult volunteers ingested a moderate dose of ethanol (0.5 or 0.8 g/kg ethanol: n = 12 at each dose) or placebo before completing the discounting task.

  23. Group I metabotropic glutamate receptor antagonists impair

    Little research has examined the contribution of metabotropic glutamate receptors (mGluRs) on risky choice. The goal of the current experiment was to determine the effects of mGluR 1 and mGluR 5 antagonism on risky choice as assessed in probability discounting (PD). Male Sprague Dawley rats (n = 24) were trained in PD, in which consistently ...