The selective serotonin reuptake inhibitor sertraline alters learning from aversive reinforcements in patients with depression: evidence from a randomized controlled trial

Abstract

Background

Selective serotonin reuptake inhibitors (SSRIs) are first-line pharmacological treatments for depression and anxiety. However, little is known about how pharmacological action is related to cognitive and affective processes. Here, we examine whether specific reinforcement learning processes mediate the treatment effects of SSRIs.

Methods

The PANDA trial was a multicentre, double-blind, randomized clinical trial in UK primary care comparing the SSRI sertraline with placebo for depression and anxiety. Participants (N = 655) performed an affective Go/NoGo task three times during the trial and computational models were used to infer reinforcement learning processes.

Results

There was poor task performance: only 54% of the task runs were informative, with more informative task runs in the placebo than in the active group. There was no evidence for the preregistered hypothesis that Pavlovian inhibition was affected by sertraline. Exploratory analyses revealed that in the sertraline group, early increases in Pavlovian inhibition were associated with improvements in depression after 12 weeks. Furthermore, sertraline increased how fast participants learned from losses and faster learning from losses was associated with more severe generalized anxiety symptoms.

Conclusions

The study findings indicate a relationship between aversive reinforcement learning mechanisms and aspects of depression, anxiety, and SSRI treatment, but these relationships did not align with the initial hypotheses. Poor task performance limits the interpretability and likely generalizability of the findings, and highlights the critical importance of developing acceptable and reliable tasks for use in clinical studies.

Funding

This article presents research supported by NIHR Program Grants for Applied Research (RP-PG-0610-10048), the NIHR BRC, and UCL, with additional support from IMPRS COMP2PSYCH (JM, QH) and a Wellcome Trust grant (QH).

Introduction

Anxiety and depression are the most common mental health problems, often occurring together and constituting a significant fraction of the global disease burden (Bandelow et al., 2015; Evans-Lacko et al., 2018; Hirschfeld, 2001). Selective serotonin reuptake inhibitors (SSRIs) are commonly prescribed as first-line pharmacological treatments for both depression (Bogowicz et al., 2021; Kendrick, Stuart, Newell, Geraghty, & Moore, 2015) and anxiety disorders (Garakani et al., 2020). However, how SSRIs work beyond their initial pharmacological action on the serotonin transporter remains unclear (Harmer, Duman, & Cowen, 2017).

Neuropsychological models propose that antidepressants, including SSRIs, may alter cognitive processing, leading to improvements in depressive and anxiety symptoms (Harmer, Goodwin, & Cowen, 2009a). Reinforcement learning provides a framework for investigating links between cognitive and biological processes and hence the effect of SSRIs on cognition (Huys, Browning, Paulus, & Frank, 2021; Lan & Browning, 2022; Maia & Frank, 2011). Preclinical and experimental research has established that several cognitive functions relevant to the etiology of anxiety and depression are sensitive to SSRIs (e.g. Geurts, Huys, den Ouden, & Cools, 2013a; Guitart-Masip, Duzel, Dolan, & Dayan, 2014; Harmer, 2013; Michely, Eldar, Erdman, Martin, & Dolan, 2022; Michely, Eldar, Martin, & Dolan, 2020; Roiser, Elliott, & Sahakian, 2012a; Roiser et al., 2012b). However, there is little evidence tying these experimental effects of SSRIs on cognition to improvement in symptoms in clinical settings as only a few clinical randomized controlled trials (RCTs) have evaluated candidate mechanisms to explain treatment effects (Ahmed et al., 2022; Cuthbert & Insel, 2013; Morris et al., 2022; Pizzagalli et al., 2020). Evaluations in the context of RCTs comparing SSRIs and placebo provide a strong test of whether specific cognitive or learning processes are the mechanisms through which SSRIs alleviate symptoms of anxiety and depression.

The present study investigates whether SSRIs improve symptoms by modulating reinforcement learning processes, specifically aversive Pavlovian control. Aversive Pavlovian control refers to the automatic, stereotyped inhibition of actions in the face of negative expectations (Bolles, 1970; Dayan, Niv, Seymour, & Daw, 2006), an effect that can be robustly observed in humans using neurocognitive probes (Boureau & Dayan, 2011; Guitart-Masip et al., 2012; Huys, Moutoussis, & Williams, 2011b). Aversive Pavlovian control is a promising candidate mechanism for the treatment of SSRIs. It is sensitive to serotonergic functioning in animal (Abela et al., 2020; Amo et al., 2014; Doya, Miyazaki, & Miyazaki, 2021; Ohmura, Tanaka, Tsunematsu, Yamanaka, & Yoshioka, 2014) and preclinical studies (Crockett, Clark, Apergis-Schoute, Morein-Zamir, & Robbins, 2012; Crockett, Clark & Robbins, 2009; Geurts, Huys, den Ouden, & Cools, 2013b; Hebart & Gläscher, 2015). Moreover, aversive Pavlovian control is associated with symptoms of depression and anxiety in clinical samples (Huys et al., 2016; Nord, Lawson, Huys, Pilling, & Roiser, 2018) and in general population samples with anxiety traits (Mkrtchian, Aylward, Dayan, Roiser, & Robinson, 2017). Influential reviews have highlighted the prominence of inhibition in response to negative expectations in depression (Roiser et al., 2012a) and of avoidance driven by negative expectations in anxiety (LeDoux, Moscarello, Sears, & Campese, 2017). Modifying aversive Pavlovian control is hence clinically promising (Huys, Russek, Abitante, Kahnt, & Gollan, 2022; Martell, Dimidjian, & Herman-Dunn, 2010).

In terms of underlying mechanisms, computational models have proposed formal relationships between rumination, acute reductions in central serotonin levels, and the attenuation of aversive Pavlovian control (Dayan & Huys, 2008, 2009; Huys et al., 2012; Robinson et al., 2013). At the neural level, the subgenual anterior cortex has been implicated in aversive Pavlovian control in research involving primates (Amemori & Graybiel, 2012) and healthy volunteers (Lally et al., 2017). This brain region is also recognized for its involvement in anxiety, as demonstrated in a study of healthy volunteers with contextual fear (Alvarez, Chen, Bodurka, Kaplan, & Grillon, 2011; Hasler et al., 2007) and it has been linked to depression in both preclinical (Drevets, Savitz, & Trimble, 2008; Ramirez-Mahaluf, Perramon, Otal, Villoslada, & Compte, 2018) and depression treatment studies (Mayberg et al., 2005).

Appetitive Pavlovian control may also be affected. In both clinical and subclinical depression samples, there have been reports of blunted reward responses (Bylsma, Morris, & Rottenberg, 2008; Eshel & Roiser, 2010; Halahakoon et al., 2020; Pizzagalli, Jahn, & O'Shea, 2005; Steele, Kumar, & Ebmeier, 2007), possibly due to reduced specificity (Huys et al., 2016; Nord et al., 2018). Serotonergic manipulations have also shown effects on appetitive Pavlovian processes in animals (Cohen, Amoroso, & Uchida, 2015) and healthy volunteers (Michely et al., 2020).

As such, Pavlovian control may be a candidate mediator of the effect of SSRIs on anxiety and depression. Here, we report a test of this hypothesis in the context of the PANDA RCT (Lewis et al., 2019). This trial compared sertraline to placebo for the treatment of depression in primary care in the UK (Duffy et al., 2019; Lewis et al., 2019; Salaminios et al., 2017). PANDA found no evidence that sertraline reduced depressive symptoms to a clinically meaningful extent at 6 weeks, with only a weak effect at 12 weeks. However, they found evidence that sertraline reduced anxiety at 6 and 12 weeks. We measured Pavlovian inhibition and a number of other reinforcement learning processes during this trial using computational modeling of the affective Go/NoGo task (Guitart-Masip et al., 2012). This is a well-established learning paradigm in which computational analyses allow appetitive and aversive Pavlovian processes to be measured (Guitart-Masip et al., 2012).

We pre-registered an analysis plan investigating five main hypotheses (osf.io/7q8v2). The primary analyses aimed to test whether treatment with the SSRI sertraline alters aversive Pavlovian control and whether aversive Pavlovian control is related to anxiety, i.e. whether Pavlovian inhibition might mediate the effect of sertraline on anxiety. We also examined the relationship between appetitive Pavlovian biases and depressive symptomatology. Overall, task compliance was poor, and the primary hypotheses were not supported. However, exploratory analyses did reveal that higher changes in aversive Pavlovian bias early on were linked to more severe depression after 12 weeks. Additionally, there was an effect of SSRI treatment on the aversive learning rate at week two and an association between learning from losses and anxiety.

Methods

Ethics

The National Research Ethics Service Committee, East of England – Cambridge South approved the study (ref: 13/EE/0418). The Medicines and Healthcare Products Regulatory Agency gave clinical trial authorization. Written informed consent was obtained from each participant before the study.

Participants

We present secondary analyses of data acquired in the context of the PANDA trial. PANDA was a randomized, double-blind, placebo-controlled pragmatic study investigating the clinical effectiveness of sertraline on depressive symptoms as the primary outcome.

Patients (aged 18–74 years) were recruited from 179 primary care surgeries in four UK sites (Bristol, Liverpool, London, York). The critical entry criterion was that general practitioners (GPs) and/or patients were uncertain about the potential benefits of an antidepressant. No lower or higher thresholds were set on depression severity or duration. The study aimed for a diverse participant pool by including doctors with varied decision-making approaches, promoting clinical equipoise, and capturing the spectrum of depressive symptom severity. The exclusion criteria were: unable to understand or complete study questionnaires in English; antidepressant treatment in the past eight weeks; comorbid psychosis, schizophrenia, mania, hypomania, bipolar disorder, dementia, eating disorder, or major alcohol or substance abuse; and medical contraindications for sertraline.

Patients were randomized to sertraline or placebo, stratified by severity, duration, and site, and followed up after 2, 6, and 12 weeks. For the first week, patients received one capsule (50 mg sertraline or placebo) a day. From week two onwards, they took two capsules per day, either containing 100 mg of sertraline or placebo, for up to 11 weeks. Medication could be increased to 150 mg in consultation with the local principal investigator in cases of non-response after six weeks. The study was double-blind: study patients, care providers, and all members of the research team were blinded to the study treatment allocation (Salaminios et al., 2017).

Measurements

The Go/NoGo task (Fig. 1a) was designed to study Pavlovian appetitive and aversive influence on choice by crossing action (go v. nogo) and valence (rewards v. losses; Guitart-Masip et al., 2012). Participants were verbally instructed that each fractal would lead to a more favorable outcome with either go or nogo, but that outcomes were probabilistic (cf. Fig. 1 for detailed task description). Each task administration employed a different fractal set. Fractal sets were randomized across participants and assessment timepoints. The Go/NoGo task was assessed at baseline, at two weeks (follow-up 1), and at six weeks (follow-up 2), but it was not part of the 12 weeks assessment (follow-up 3). The Generalized Anxiety Disorder Assessment (GAD-7; Spitzer, Kroenke, Williams, & Löwe, 2006), the Patient Health Questionnaire-9 (PHQ-9; Kroenke, Spitzer, & Williams, 2001), and the Beck Depression Inventory (BDI; Beck, Steer, & Brown, 1996) were completed at baseline and every follow-up. Several baseline variables were acquired (cf. Table 1).

Figure 1. Task and performance. (a) The Go/NoGo task consisted of four different conditions. On each trial one of four possible fractal images was shown. Actions were required in response to a circle that followed the fractal image after a variable delay. After a brief delay, the outcome was presented: a green upward arrow for a win, a red downward arrow for a loss, or a horizontal bar for a neutral outcome. In the go-to-win condition, pressing the key (‘go’) led to a reward with 80% and a neutral outcome with 20% probability, vice versa if they did not press the key (‘nogo’). In the go-to-avoid condition, pressing the key (‘go’) led to a neutral outcome with 80% and a loss with 20% probability. In the nogo-to-win, not pressing the key (‘nogo’) led to a reward with 80% and a neutral outcome with 20% probability. In the nogo-to-avoid condition, not pressing the key (‘nogo’ response) led to a neutral outcome with 80% and a loss with 20% probability. Each task administration consisted of 96 trials, with 24 trials per condition. (b) Mean percentage of correct responses in each of the four conditions. Black dots depict participants and black error bars depict standard deviation of the mean (s.d.). Dashed lines depict chance level. Post hoc comparisons were implemented by means of repeated measures t tests showing a significant difference in accuracy between Pavlovian congruent (got to win and nogo to avoid) and incongruent conditions (go to avoid and nogo to win). Significance ∗ ≤ 0.05, ∗∗ ≤ 0.01, ∗∗∗ ≤ 0.001, ∗∗∗∗ ≤ 0.0001.

Table 1. Baseline characteristics for participants providing informative Go/NoGo task data (N = 435)

Data are reported in N(%) or mean(s.d.). There was no evidence for differences in baseline characteristics between the treatment groups shown by the p values (≤0.05). PHQ-9 = Patient Health Questionnaire, 9-item version total score (possible range 0–27). GAD-7 = Generalized Anxiety Disorder Assessment, 7-item version total score (possible range 0–21). BDI = Beck Depression Inventory, 21-item version total score (possible range 0–63). CIS-R = Clinical Interview Schedule-Revised measuring depression severity score (possible range 0–21).

Computational models

Previously published computational models for this task (Guitart-Masip et al., 2012; Mkrtchian et al., 2017; Moutoussis et al., 2018; Scholz et al., 2022) provide formal, quantitative descriptions of the evolution of decisions over the course of learning during the task. The core parameters of interest in the models are the Pavlovian parameters. These capture appetitive Pavlovian influences through the extent to which participants automatically emit ‘go’ responses when faced with reward stimuli, and aversive Pavlovian inhibition through the extent to which they automatically emit ‘nogo’ responses when faced with loss stimuli. The Pavlovian processes are separate from instrumental learning processes, which emit ‘go’ and ‘nogo’ according to which of the two actions is more likely to lead to the better outcome. Other parameters include reward and loss sensitivity, learning rates, irreducible noise, and an overall ‘go’ bias.

Data validation

To evaluate whether the existing data was in principle sufficient to assess the key hypotheses, and to provide an informative a-priori estimate of power, two authors (J. M. and Q. J. M. H) were provided with blinded access to the behavioral task data only, but without access to group allocation, demographics, or measures of symptoms. These authors fitted different reinforcement learning (RL) models (for a list of the models, see Supplementary Materials B.1 RL Models) as described previously in the literature (cf. Huys et al. (2011a) and Supplementary Materials B.2 Model Fitting Procedure & B.3 Model Comparison). All datasets of the study were combined, disregarding within-subject information (i.e. treating repeated sessions as independent task assessments). In the supplements, we report the recoverability and reliability of the parameters (cf. Supplementary Materials Fig. B.3 and B.4).

Models were fitted separately to the data and compared using the integrated Bayesian Information Criterion (iBIC; Figure 2a) at the group level, where the individual likelihoods were first integrated over the individual parameters using a sampling procedure and then summed over all individuals. The most parsimonious model included learning rates, outcome sensitivities, and Pavlovian biases, all separated into rewarding and punishing contexts. Figure 2c shows that simulated data captured the empirical data qualitatively. Hence, standard models of the task are able to parametrically capture the variability of behavioral performance in the task across individuals and sessions on a trial-by-trial level.

Figure 2. Computational modeling of the Go/NoGo task. (a) shows the differences in integrated Bayesian Information Criterion (iBIC) scores for all models tested compared to the most parsimonious model (red star), where a smaller iBIC score indicates a more parsimonious model. All models are modified Q-learning models (Rescorla Wagner – RW) with two pairs of action-values (‘go’ and ‘nogo’) for each stimulus. The y-axis shows the number of free parameters for each model. The most parsimonious model includes separate learning rates for rewards and punishments, win and loss sensitivities, appetitive and aversive Pavlovian biases, irreducible noise, and a constant bias factor added to the action-value for ‘go’. (b) shows the histogram of the difference between the integrated loglikelihood (iLL) of the most parsimonious model and the iLL of the random baseline model. Datasets were declared as informative if the data was more than three times more likely to have occurred under the most parsimonious model (vertical red dashed line). (c) The four subplots show the average learning curves in blue (averaged over participants; solid line) for each condition separately. Each row of the raster images shows the choices of each participant. ‘Go’ responses are depicted in white, and ‘nogo’ responses are depicted in grey. Additionally, the average ‘go’ probability was separated into included datasets (orange) and excluded datasets (green). The solid line refers to empirical data and the dashed line to simulated data from the most parsimonious model. Informative datasets (orange) show that participants, on average, seem to learn over trials, which can be captured qualitatively well by the most parsimonious model. In contrast, the average ‘go’ probability of non-informative/excluded datasets (green) appears to have no temporal relation, hence showing no learning over trials. Further, it is well captured by the random baseline model.

In the Go/NoGo task, non-informative responses (e.g. always emitting the same response) cannot provide information about Pavlovian or other cognitive processes and therefore do not inform parameter estimates. Whether the data of a particular task run are meaningful can be evaluated formally by examining whether a model encompassing the core processes provides a more parsimonious account of the behavioral data than a random baseline model. In other words, to examine whether the observed behavioral data meaningfully constrained the model parameter estimates, we compared the integrated likelihood of the most parsimonious model to the integrated likelihood of a random baseline model for each dataset from each individual at each session. The integrated likelihood integrated over an individual parameters refers to the likelihood of the data given the group-level hyperparameters. A task run was deemed as missing if the integrated likelihood of the random baseline model was more than three times higher than that of the most parsimonious model at the group level (Fig. 2b). Note, the model selection process conducted only on the informative task runs yielded consistent results with those obtained on the complete dataset (cf. Supplementary Materials Fig. B.2).

The parameters for each informative task run were extracted from the most parsimonious model to test the hypotheses.

Preregistration

The key hypotheses and analyses were pre-registered on OSF (osf.io/7q8v2; cf. Supplementary Materials Table D.4).

Statistical analyses

Predictors of missing and non-informative data at baseline were identified using a univariate logistic regression. Significantly related baseline variables were used as covariates in all further analyses.

To investigate drug effects, we employed a mixed-effects linear regression (1) using group allocation as the independent variable and the parameter estimate (e.g. aversive Pavlovian bias as the dependent variable) controlling for stratification variables (baseline CIS-R total score in three categories, duration of depressive episode in two categories, and site) and including random intercepts. We reported mean differences (MD), 95% confidence intervals (CI), and the corresponding p values (p).

Next, we examined whether parameter estimates relate to depressive or anxiety symptoms using a mixed-effects multiple linear regression (2) with the parameter estimate as independent variable and log-transformed symptom scores (e.g. GAD-7 total score) as dependent variable. Random slopes and intercepts per individual were included. We controlled for group allocation and stratification variables. We reported regression coefficients (β), 95% confidence intervals (CI), and the corresponding p values (p).

For both analyses, we performed separate mixed-effects models for baseline and week two, baseline and week six and over all three time-points. To investigate a potential drug time interaction, we additionally performed a regression including a group-time interaction. The group variable in the mixed-effects models was coded [0,1,1] for a patients allocated to sertraline and [0, 0, 0] for a patients allocated to placebo. Both groups have a 0 at baseline because they were unmedicated at that time.

To investigate whether a baseline parameter estimate predict treatment outcome, we performed a simple linear regression predicting symptoms core at the at week 12 controlling for symptoms at baseline, group allocation and stratification variables.

As an exploratory analysis we examined whether early change in aversive Pavlovian bias (week 2 – baseline) relates to log-transformed BDI total score at week 12 using a simple linear regression including an interaction effect between group-allocation and Pavlovian bias.

Exploratory analyses repeated the analysis type 1 above for each individual parameter and used Bonferroni-correction to correct for testing multiple parameters (p ≤ (0.05/8) ≤ 0.00625).

Additionally, we conducted simple linear regression examining group differences in parameter slopes (early change = week two – baseline; late change = week six – week two). We also repeated analysis type 2 for each of the parameter estimates and the three psychological measures (GAD-7, PHQ-9, BDI) and used Bonferroni-correction to correct for testing multiple parameters (p ≤ (0.05/8 × 3) ≤ 0.002).

Finally, to assess test–retest reliability we calculated Pearson correlation of individuals' parameters between the different time points and employed intra-class correlation coefficients (ICCs; McGraw & Wong, 1996) using the informative data (cf. Supplementary Materials B.6 Test-Retest Reliability).

Results

A total of 655 patients were recruited and randomly assigned to sertraline (326, 50%) and placebo (329, 50%). Two patients in the sertraline group did not complete a substantial proportion of the baseline assessment and were excluded. Additionally, 25 patients (9 from the sertraline group and 16 from placebo) did not complete the Go/NoGo task at any time-point. This left 628 participants (315 sertraline and 313 placebo) for analyses (cf. Fig. A.1 in Supplementary Materials). Task data for seven patients at baseline, 99 patients at 2 weeks, and 145 patients at 6 weeks were missing. Missing follow-up data were more common in participants who had higher baseline depressive and anxiety symptoms, financial difficulties, were from ethnic minorities and were recruited from London (cf. Supplementary Materials Table E.5). Missing data did not differ statistically by treatment allocation.

Basic task characteristics

Examination of the average percent correct response per condition showed the typical interaction pattern characteristic of Pavlovian inference found in previous studies (Guitart-Masip et al., 2012; Mkrtchian et al., 2017; Moutoussis et al., 2018; Scholz et al., 2022) at all measurement points (Fig. 1b). Performance was better in Pavlovian congruent (go to win and nogo to avoid) than incongruent (go to avoid and nogo to win) conditions (|t| ∈ [4.65, 16.70], p < 0.001). There were no differences in average performance between patients allocated to sertraline and patients allocated to placebo (|MD| ∈ [0.00, 0.03], p > 0.05).

Computational modeling results

Overall, 747 (46%) task runs did not contain interpretable and informative behavioral data. Variables associated with non-informative behavior were higher age, lower education, and past antidepressant use. At week 2 non-informative task runs (N = 230, 43%) were more likely in patients who were allocated to the sertraline group (57%, X2 = 7.06, p = 0.008). In addition, baseline anxiety score, depression severity, and employment status were predictive of non-informative behavior at week 6 (cf. Supplementary Materials Table E.6). For all further analyses we focused on the 886 informative task runs from 435 patients (66% of those originally randomized) and adjusted for significant predictors of non-informative data as covariates. Characteristics of the remaining sample according to the study arm are shown in Table 1. Baseline characteristics of the sample were not statistically distinguishable between treatment groups.

The effect of sertraline on anxiety remained significant in the smaller included sample (week 6: MD = −0.1, CI[ − 0.17 to  − 0.03], p = 0.005; week 12: MD = −0.12, CI[ − 0.17 to  − 0.06], p ≤ 0.001; over time: MD = −0.08, CI[ − 0.12 to  − 0.04], p ≤ 0.001). This is important since our preregistered hypotheses were developed under consideration of this effect.

Preregistered hypotheses

The preregistered hypotheses were not supported (Table 2): there was no evidence that the aversive Pavlovian inhibition was affected by sertraline (Fig. 3a and b); that aversive Pavlovian inhibition was related to anxiety symptoms; that the baseline aversive Pavlovian bias was predictive of treatment response; that the appetitive Pavlovian bias was associated with depression or that the reward sensitivity was related to anhedonia.

Table 2. Mixed-effect linear models testing our pre-registered hypotheses (only informative data N _{task runs} = 886; N _patients = 435, 66% of those randomized)

We tested whether sertraline alters aversive Pavlovian control (Hypothesis 1; H1) and whether aversive Pavlovian control is related to anxiety (Hypothesis 2; H2). Hypothesis 3 regarding the aversive Pavlovian bias as a mediator for the effect of sertraline on anxiety was not investigated as there was no evidence for H1 and H2. Hypothesis 4 (H4) tested whether aversive Pavlovian bias at baseline before starting SSRI treatment predicted treatment outcome. Hypothesis 5 (H5) examined the relationship between the appetitive Pavlovian bias and depressive symptoms. Hypothesis 6 (H6), tested for a relationship between reward sensitivity and anhedonia. We controlled for stratification variables and variables associated with missing data in all analyses.

Figure 3. Effects of sertraline on RL parameters. (a) Shows the aversive Pavlovian bias at baseline and at the follow-ups separated into drug groups (blue, left = placebo; red, right = sertraline). (b) Shows the change in aversive Pavlovian bias between sessions separately for the drug groups. (c) Early changes in the aversive Pavlovian bias predict treatment outcome. This figure shows the relation between the change from baseline to week two in the aversive Pavlovian bias and log-transformed BDI total score (only of participants who had an informative task run at baseline and week 2). In blue the placebo group and in red the sertraline group. An interaction effect was observed between group and early change in the aversive Pavlovian predicting depression at 12 weeks driven by a significant association between the early change and log-transformed BDI total score at 12 weeks. (blue, left = placebo; red, right = sertraline). (d) Shows the loss learning rate at baseline and at the follow-ups separated into drug groups (blue, left = placebo; red, right = sertraline). (e) Shows the change in loss learning rate between sessions separately for the drug groups. Significance *  ≤  0.05, **  ≤  0.01, ***  ≤  0.001, ****  ≤  0.0001.

Exploratory analyses

Exploratory analyses of a subsample with better test–retest correlation, and of a subsample with low symptoms did not support the pre-registered hypotheses (cf. Supplementary Materials F Subsample Analyses).

Two sets of results in the exploratory analyses are noteworthy. The first relates to early change in the aversive Pavlovian bias. The slope of the aversive Pavlovian bias between baseline and week two was positively related to depressive symptoms at week 12 (log-transformed PHQ9 total score: β = 0.06, CI[0.0 − 0.11], p = 0.044; log-transformed BDI total score: β  =  0.07, CI[0.01 − 0.13], p = 0.016). A larger increase in aversive Pavlovian bias was associated with more severe subsequent depressive symptoms. Furthermore, the BDI model revealed an interaction between group allocation and early change in the aversive Pavlovian bias (β = 0.14, CI [0.02–0.26], p = 0.024; Figure 3c). That is, early change in aversive Pavlovian bias was more strongly related to BDI scores at week 12 in the sertraline group (β = 0.14, CI[0.05 − 0.23]), than in the placebo group (β = 0.02, CI[ − 0.03 to 0.07]). However, note that sertraline had no effect on the early change in aversive Pavlovian bias (MD = −0.08, CI[ − 0.32 to 0.15], p = 0.49). The second set of findings relates to the speed at which participants adapted behavior following losses (the loss learning rate). There was an effect of sertraline on the loss learning rate at week 2 (MD = 0.6, CI[0.22 − 0.97], p = 0.002; Figure 3d). The sertraline group learned faster from losses at week 2 than the placebo group. Early change in loss learning rate (week two – baseline) was higher in the sertraline group (MD = 0.75, CI[0.18 − 1.3], p = 0.009; Figure 3e), whereas later change (week six minus week two) was lower in the sertraline group (MD = −0.72, CI[ − 1.27 to  − 0.17], p = 0.011; Figure 3e). In the sertraline group, the early change was different from zero (t = 2.74, p = 0.007), whereas the later change was not (t = −0.32, p = 0.75). In contrast, in the placebo group, the early change did not differ from zero (t = −0.70, p = 0.483), but the late change did (t = 3.44, p < 0.001). Hence, the group difference in the late change was due to an increase in loss learning rate from baseline to week 6 in the placebo group. The aversive learning rate is strongly driven by switching after losses in the early part of the learning curve. Indeed, there was an elevated switching probability after losses during the first eight trials in the sertraline group (MD = 0.21, CI [0.0–0.41], p = 0.048; averaged across the go-to-avoid and nogo-to-avoid conditions). Finally, the loss learning rate was also positively associated with the anxiety scores (at week 2: β = 0.01, CI[0.0 − 0.02], p = 0.047; at week 6: β = 0.02, CI[0.0 − 0.03], p = 0.016; across all sessions: β = 0.02, CI[0.01 − 0.03], p = 0.001). However, there was no evidence for an association between anxiety symptoms and either the loss learning rate at baseline (β = 0.01, CI [−0.0 to 0.02], p = 0.24) or the early change in loss learning rate (week 2 – baseline; β = −0.02, CI[ − 0.04 to 0.01], p = 0.164). Additionally, this effect could not be shown based on the early switch probability described above (β = 0.0, CI[ − 0.02 to 0.02], p = 0.73).

Repeating these analyses on the complete sample including all task runs resulted in a broadly consistent pattern of effects (c.f. Supplementary Materials C Findings in the Whole Sample).

In post hoc analyses, we adjusted for the use of other antidepressants and/or psychotherapy, adherence score, and the number of tablets, yielding consistent results as detailed in the Supplementary Materials G Post-hoc Analyses.

Task reliability

Parameters showed poor to moderate reliability (ICC(3,1) ranging from 0 to 0.53; cf. Supplementary Materials B.6 Test-Retest Reliability). The aversive Pavlovian bias was the most reliable parameter (ICC(3, 1) = 0.53, CI[0.41 − 0.64], p < 0.001). The Pavlovian parameters and the go bias also significantly changed over time. The Pavlovian biases decreased (aversive: β = −0.1, CI[ − 0.16 to  − 0.04], p = 0.001; appetitive: β = −0.08, CI[ − 0.13 to  − 0.04], p < 0.001) and the go bias increased (β = 0.13, CI[0.05 − 0.21], p < 0.001) over sessions which likely led to an increase in task accuracy (β = 0.02, CI[0.01 − 0.03], p < 0.001). We note that age reduced accuracy (β = −0.03, CI[ − 0.04 to  − 0.02], p < 0.001), most likely due to increasing Pavlovian biases (aversive: β = 0.19, CI[0.11 − 0.26], p < 0.001; appetitive: β = 0.16, CI[0.11 − 0.21], p < 0.001) and reducing go bias with age (β = −0.4, CI [−0.49 to−0.32], p < 0.001).

Discussion

We investigated the effects of the SSRI sertraline on reinforcement learning mechanisms in the PANDA trial, a pragmatic multicenter, double-blind, placebo-controlled, randomized clinical trial. SSRIs are first-line pharmacological treatments for depression and anxiety, but the mechanism of SSRI action is still unknown. A better understanding of how SSRIs work could lead to improved response predictions and new, refined treatments. Our goal was to identify clinically relevant mechanisms to link receptor action to cognition and affective processing. Reinforcement learning enables such links and hence is a promising framework for investigating the mechanisms of SSRI action. The PANDA trial was the largest individual placebo-controlled trial not funded by the pharmaceutical industry. The sample was recruited in primary care based on clinical equipoise, and depressive symptoms ranged from mild to severe. Findings might therefore be of relevance to the broader primary care population. As sertraline acts through similar mechanisms as other SSRIs (Cipriani et al., 2018), the findings may also be relevant for other SSRIs.

Due to the poor task performance, almost half of the performed task runs were excluded. Early on (at week two) non-informative data was more prevalent in the sertraline group, suggesting that patients in the active group may have responded more randomly. Such randomness can be a signature of low overall motivation to perform the task. One possibility is that such a broad motivational reduction could be a signature of SSRI-induced affective blunting (Barnhart, Makela, & Latocha, 2004; Marazziti et al., 2019; McCabe, Cowen, & Harmer, 2009; Price, Cole, & Goodwin, 2009). However, there were no discernible differences in symptoms between patients who provided informative and non-informative data at week two, and sertraline had a positive impact on learning at week two in the included sample. These findings speak against a broad blunting effect.

The primary goal of this study was to test whether aversive Pavlovian bias mediates the effect of sertraline on anxiety. We found no evidence supporting an influence of sertraline on aversive Pavlovian bias. This result contrasts with previous research suggesting that Pavlovian inhibition is sensitive to serotonin (Crockett et al., 2009, 2012; Geurts et al., 2013b; Hebart & Gläscher, 2015). There are several possible reasons for this discrepancy. First, it may be that serotonin manipulations have different effects on Pavlovian inhibition in samples with and without depression and/or anxiety. While the current study was performed in a clinical population, previous studies primarily examined healthy volunteers. Second, previous research focused on acute changes via tryptophan depletion (Crockett et al., 2009, 2012; Geurts et al., 2013a; Hebart & Gläscher, 2015) or a single administration of an SSRI citalopram (Guitart-Masip et al., 2014) rather than the chronic administration examined here. It has long been posited that acute and chronic SSRI administration have opposite effects (e.g. Harmer, Cowen, & Goodwin, 2011; Harmer et al., 2009b). Third, we cannot rule out that some of the Pavlovian inhibition signal is conflated with the loss learning signal as there are non-negligible correlations between parameters (cf. Supplementary Materials Fig. B.6). This is likely compounded by broader issues with data quality, which in turn reduce the ability of models to distinguish aversive Pavlovian inhibition and learning from losses. Subsample analyses attempting to identify either test–retest or symptom loads as reasons for the null results did not yield clear results.

Exploratory analyses identified relationships between sertraline, aversive processing, and symptoms. First, sertraline affected learning from losses but not from rewards. This finding is in keeping with well-supported empirical evidence demonstrating that serotonin modulation impacts learning (Bari et al., 2010; Brigman et al., 2010; Michely et al., 2020; Scholl et al., 2017), and specifically punishment learning (Chamberlain et al., 2006; Cools, Roberts, & Robbins, 2008; Tanaka et al., 2007, 2009). Prolonged serotonin alterations have downstream effects including augmented learning and plasticity (Dayer, 2014; Kraus, Castrén, Kasper, & Lanzenberger, 2017). In the current dataset, the learning rate from losses increased over the first two weeks of sertraline treatment relative to placebo. The placebo group then ‘caught up’, removing the group differences in loss learning rate at six weeks. Changes in the performance of learning tasks are frequently observed and thought to represent a type of meta-learning, i.e. learning more broadly about the strategy of performing a task rather than learning within the task itself (Botvinick et al., 2019; Doya, 2002; Langdon et al., 2022; Vanschoren, 2019). As such, the late change in performance in the placebo group compared to the early change in the sertraline group suggests that sertraline may have increased the speed at which this meta-learning may have occurred and may have done so by specifically altering behavioral adaptation after losses within the task. One complication is that, at two weeks, there was already some evidence for changes in anxiety symptoms, and an inverse causal path (with anxiety mediating the effect of sertraline) cannot be excluded.

The loss learning rate was correlated with anxiety symptoms at both follow-up time points and over all measurement points. This is, in principle, in line with previous research outlined in a recent meta-analysis reporting higher punishment learning rates and slightly lower reward learning rates in patients (Pike & Robinson, 2022). Yet, this is difficult to reconcile with, first, the SSRI-induced increase in learning from punishment, and second the fact that both anxiety and depression are treated by SSRIs and are linked to heightened punishment learning themselves. Interestingly, a similar conundrum was present in the literature on learned helplessness, which was associated with increased levels of serotonin (Petty, Kramer, & Moeller, 1994), but could also be reversed as a response to SSRIs (Hajszan et al., 2010; Kirby, 2006; Malberg & Duman, 2003; Maudhuit et al., 1997). Hence, coupling increases in serotonin levels with a simple account of serotonin levels on behavior is unlikely to be able to explain SSRI effects. Indeed, the serotonin system is known to be exquisitely complex, with many different serotonin receptors distinctively distributed (Hansen et al., 2022). A possible explanation could be that SSRIs facilitate learning faster in a punishing environment, thus leading to less negative and more positive (or neutral) feedback. It is interesting to consider how this bias towards learning from losses might be linked to mood. Self-reports of happiness are linked to positive prediction errors (Rutledge, Skandali, Dayan, & Dolan, 2014), suggesting that negative prediction errors might similarly influence negative affective states. In other words, SSRIs might gradually improve mood by enhancing negative expectations through faster loss learning, thereby giving rise to less disappointing and more rewarding experiences.

Finally, improvements in depressive symptoms in the sertraline group were preceded by an early decrease in the aversive Pavlovian bias. In other words, patients on sertraline showed a higher increase in their tendency to withhold an action when facing a loss between baseline and the 2-week follow-up, the higher their depressive symptoms were after 12 weeks.

Overall, the findings draw a complex picture involving aversive processing, sertraline, and symptoms, possibly reflecting the known complexity of the serotonin system. Despite the methodological limitations and the failure to support the preregistered hypotheses, the exploratory data suggest alterations in the processing of losses. A tentative possibility is that SSRIs alter the speed of learning from losses early on, inducing a shift from Pavlovian to instrumental learning when confronted with losses. The alteration in aversive Pavlovian bias was not directly linked to sertraline. However, sertraline appeared to modulate the association between Pavlovian inhibition and future treatment outcomes. Reducing aversive Pavlovian control might hence promote approach responses in a punishing environment, facilitating unexpected rewarding experiences and thus helping to alleviate depressive symptoms.

Limitations

Inclusion in the trial was based on clinical equipoise, i.e. inclusion was based on an uncertainty whether medication could clinically be helpful for a particular person. This may have decreased the power to detect differences from placebo. For mechanistic studies such as the current one, it could be better to study a cohort of typical responders, i.e. patients who are prescribed medication with clinical confidence.

Extensive validation analyses showed that task performance was frequently objectively poor resulting in a large fraction of the task runs being non-informative. Non-informative task runs had to be excluded from analyses because formally they cannot provide information about cognitive mechanisms. We attempted to address this by correcting for baseline variables that were significantly associated with non-informative task runs. The sertraline and the placebo group in the final informative sample continued to be matched on baseline characteristics. Nevertheless, the exclusion of data has severely curtailed the power in the study. Furthermore, because noninformative data was more common in the drug than the non-drug arm, a causal interpretation is no longer warranted.

The poor task performance has important implications for future mechanistic research in this domain. Although the task has been extensively used in laboratory studies (Guitart-Masip et al., 2012), combined with neuroimaging (Guitart-Masip et al., 2011), pharmacological (Guitart-Masip et al., 2014) and other interventions and adapted (Millner, Gershman, Nock, & den Ouden, 2018; Moutoussis et al., 2018; Swart et al., 2017), it did not prove effective in a longitudinal clinical trial. This reinforces the paramount importance of acceptability and effectiveness testing of cognitive measurements for translational research and calls for an involvement of stakeholders in the design of research tasks.

The relationships between cognitive mechanisms and symptoms were weak. This probably reflects more general findings in the field (Eisenberg et al., 2019), but also the specific limitations around data quality mentioned above which limit the strength of possible associations (Spearman, 1904). We also note that our computational modeling approach was very conservative in that all parameters were allowed to change freely between participants and sessions, with no constraints for within-participant data.

Exploratory results were presented based on passing a conservative significance threshold and their relevance to the preregistered hypotheses. Nevertheless, they should be treated with caution prior to replication.

Conclusion

This study represents a significant exploration of specific reinforcement learning processes in a pragmatic RCT for depression. Specific reinforcement learning mechanisms did show a relationship to aspects of depression and anxiety and its treatment with SSRIs, but this was weak and not as hypothesized a priori. Sertraline influenced aversive processing in the first two treatment weeks by altering how participants learn to execute a passive or active action to avoid loss. Moreover, symptoms were associated with aversive processing but how this relationship relates to SSRI appears complex. The fact that almost half of the data was non-informative emphasizes the importance of developing patient-acceptable task probes.