Deposit? Yes, please! The effect of different modes of assigning reward- and deposit-based financial incentives on effort

Abstract

The effectiveness and uptake of financial incentives can differ substantially between reward- and deposit-based incentives. Therefore, it is unclear to whom and how different incentives should be assigned. In this study, the effect of different modes of assigning reward- and deposit-based financial incentives on effort is explored in a two-session experiment. First, students’ (n = 228, recruited online) discounting, loss aversion and willingness to pay a deposit were elicited. Second, an incentivized real-effort task was completed (n = 171, 25% drop-out). Two modes of assigning reward- or deposit-based financial incentives were compared: random assignment and ‘nudged’ assignment – assignment based on respondent characteristics allowing opting out. Our results show that respondents receiving nudged assignment earned more and persisted longer on the real-effort task than respondents randomly assigned to incentives. We find no differences in effectiveness between reward-based or deposit-based incentives. Overall, 39% of respondents in the nudged assignment mode followed-up the advice to take deposit-based incentives. The effect of deposit-based incentives was larger for the respondents who followed-up the advice than for respondents that randomly received deposit-based incentives. Overall, these findings suggest that nudged assignment may increase incentives’ effect on effort. Future work should extend this approach to other contexts (e.g., behaviour change).

Introduction

Motivation is a key component for engaging in behaviour and behaviour change with a significant impact on health or wellbeing, e.g., smoking cessation or physical exercise. Often these behaviours involve costs and/or effort in the short term with potential benefits only occurring later. Some individuals may complete these tasks exclusively on basis of their intrinsic motivation (our natural tendency to seek challenge, novelty and learning opportunities), while for others extrinsic motivation is also needed (Deci and Ryan, 2008). One potential strategy to increase motivation is the use of financial incentives. Some worry that this strategy would reduce intrinsic motivation (Frey and Oberholzer-Gee, 1997; Lacetera and Macis, 2010; Gneezy et al., 2011), as incentives would reduce the need for intrinsic motivation and their presence may signal the incentivized behaviour is unattractive (Bénabou and Tirole, 2006). Yet, the use of financial incentives for promoting behaviour change is widespread. For example, financial incentives have been found to be an effective tool to increase motivation for health behaviour change, at least in the short term (for systematic reviews, see Mitchell et al., 2013; Strohacker et al., 2013; Giles et al., 2014; Mantzari et al., 2015; Notley et al., 2019). Interestingly, recent evidence suggests that (at least in the context of vaccination) incentives are unlikely to harm intrinsic motivation or have other unintended negative consequences (Schneider et al., 2023).

In practice, many different incentive schemes can be used. Adams et al. (2014) distinguish between incentives based on (inter alia) the size, timing and direction of payment. Perhaps obviously, the size of a financial reward is of importance. Generally, it is assumed that higher rewards yield stronger motivation, although evidence suggests there are diminishing returns to increasing incentive size (Finkelstein et al., 2007; Augurzky et al., 2012). The timing of payments should also be carefully considered. A large body of economics literature suggests that future rewards are discounted (Frederick et al., 2002), and that many individuals overweigh any cost or benefit experienced today (Laibson, 1997). Hence, postponing payments into the future (heavily) decreases their value today. Finally, the direction of financial incentives is important, defined by Adams et al. (2014) as the ‘sign’ of the incentive used: is the incentive perceived as a reward to be gained through performing a task or as a loss (e.g., a fine) imposed when tasks are not completed? The difference between gain and loss incentives is crucial when individuals display loss aversion, i.e., the tendency that losses receive more weight than gains of the same size (Tversky and Kahneman, 1992).

Earlier work has used and/or compared financial incentives for behaviour change with different sizes, timing and direction (e.g., Halpern et al., 2011, 2015; Haisley et al., 2012; Patel et al., 2016). Several studies show that incentives involving losses are effective compared to a no incentive control (Giné et al., 2010; Cawley and Price, 2013; Royer et al., 2015). Some studies also present evidence suggesting that loss incentives are more effective than incentives based on gains (Patel et al., 2016; Erev et al., 2022), although other studies find no evidence for differences in effectiveness (Donlin Washington et al., 2016; Halpern et al., 2018), or evidence in the opposite direction (Halpern et al., 2015). Interpreting the existing literature is complicated by the various operationalisations of loss incentives. A common strategy used to operationalise incentives involving losses is to ask respondents to commit some of their own money into a deposit contract, to which some amount is added as an incentive. The total deposit is only returned to respondents after they attain an agreed-upon goal. Such matched deposit contracts have been used by e.g., John et al. (2011). Other authors (Giné et al., 2010; Royer et al., 2015), on the other hand, used completely self-funded deposit contracts, i.e., respondents put their own money at stake without matching. Note that for simplicity, we will refer to all incentives in which respondents risk losing some amount of their own money as deposit-based incentives.

In addition to the incentive scheme, an additional choice to be made is the mode of assignment of financial incentives. That is, should individuals be free to choose their incentive scheme, or can it be beneficial to provide incentives without individuals being consulted on their preferences beforehand? This is an important question, as the effectiveness of some types of incentive schemes may seem promising, but it appears voluntary take-up is low. For example, stated preference studies typically find large hypothetical take-up of deposit-based incentives (Sykes-Muskett et al., 2017; Lipman, 2020; Adjerid et al., 2021), in practice far fewer people are willing to actually deposit their own money (Halpern et al., 2015), especially when deposits are completely self-funded (Ashraf et al., 2006; Giné et al., 2010; Royer et al., 2015), as summarised in Carrera et al. (2019). Encouraging respondents to take up deposit contracts may require deposits to be small and/or opt-out designs, as shown by Erev et al. (2022). Although offering individuals a choice of incentive schemes may potentially increase the effectiveness of incentives due to increased autonomy (Boderie et al., 2020; Adjerid et al., 2021; Dizon-Ross and Zucker, 2021; Woerner et al., 2021), take-up of commitment devices such as deposit-based incentives is typically considered dependent on individuals’ being sophisticated about their preferences (Laibson, 1997). As also explored by Halpern et al. (2015) and Adjerid et al. (2021), deposit-based incentives may seem more effective than incentives based on gains, but perhaps this is driven primarily by the self-selection of particularly eager respondents into deposit-based incentive. As such, there may be a benefit of exogenously assigning or nudging less eager individuals towards supposedly beneficial incentive schemes (Adjerid et al., 2021). However, so far only a handful of studies have investigated variation in effectiveness of incentive schemes according to their mode of assignment (Adjerid et al., 2021; Dizon-Ross and Zucker, 2021; Woerner et al., 2021; Incekara-Hafalir et al., 2023), with conflicting evidence.

In this paper, we conduct an online experiment with a real-effort task among 228 students. The setting intends to mimic a health behaviour intervention which requires effort now and yields benefits in the future (in our case, payments one week later). Note that we implemented financial incentives with different sizes. That is, respondents could earn either 8 (low incentive), 12 (medium incentive) or 20 (high incentive) euro in our experiment. Our experimental design enables studying both the assignment mode of incentives, as well as the incentive scheme on effort for incentives of different sizes. In the 2 × 2 experiment, our first basic contrast is a comparison of a treatment arm where individuals are randomised into incentive schemes vs a treatment arm where individuals receive informed advice about which incentive to take. That is, in the latter treatment arm, respondents are not only offered a choice between deposit- and reward-based incentives, but we additionally provide them with informed advice based on personal characteristics, e.g., loss aversion. The informed advice is implemented as a default selection of one of the two incentives schemes, with the opportunity to opt-out. In other settings, implementing or changing the default with the goal of helping respondents is often referred to as nudging (Thaler and Sunstein, 2009). Hence, we will refer to this mode of assignment as nudged assignment. This contrast, therefore, sheds light on whether a nudged, yet voluntary choice enhances effort over simple random assignment to financial incentives. Our second basic contrast involves studying the effect of a deposit-based incentive scheme vs reward-based incentive scheme among the ones who were randomly assigned an incentive scheme. This second contrast sheds light on whether incentive schemes based on losses yield more effort than incentive schemes based on gains. Third, by comparing the effects of the deposit-based incentive scheme across the nudged (with free choice) and the random assignment groups, and by studying the characteristics of those that choose a deposit, we learn more about self-selection into deposit-based incentive schemes and which individuals are most likely to take up deposit-based incentive schemes and benefit from them.

In line with the three contrasts in our experiment, our contribution to the literature is threefold. First, our work extends the literature on modes of assigning financial incentives by implementing nudged assignment. Earlier work in contract theory (Larkin and Leider, 2012; Chaudhry and Klinowski, 2016; Bernard et al., 2019) has suggested that free choice among different types of incentives enables individuals to ‘sort’ into incentives that fit their preferences. That is, individuals sort into contracts that they expect will maximise earnings, but these expectations (e.g., in case of overconfidence) may be inaccurate (Larkin and Leider, 2012). Choice also offers opportunities for sophisticated individuals to commit themselves to future actions (Dizon-Ross and Zucker, 2021; Incekara-Hafalir et al., 2023), e.g., by allowing individuals to self-select into more challenging incentive schemes or to help them persevere. Other studies, however, have found that, on balance, those that had the opportunity to choose their own incentive scheme perform the same or worse than those randomly assigned to incentives (Chaudhry and Klinowski, 2016; Adjerid et al., 2021; Woerner et al., 2021). Our work follows up on the suggestion for future research put forward by Adjerid et al. (2021), who suggest to nudge individuals towards potentially beneficial incentive schemes. That is, we default (a selection of) respondents into deposit-based incentives, which may help promote their take-up when free choice exists. Indeed, other related studies have found default settings to affect uptake in commercially available deposit-based incentives, i.e., on www.stickk.com (Goldhaber-Fiebert et al., 2010; Bhattacharya et al., 2015). Opt-out deposit-based incentive designs were also used in Erev et al. (2022), who found they were effective and taken up by up to 55% of their sample.

Second, by randomly assigning deposit-based incentives in one treatment arm, our study enables to estimate the effect of deposit-based incentives over reward-based incentives without having to worry about the self-selection of individuals into deposit-based incentives. In addition, we randomly varied the size of the reward. As such, our work contributes to an existing literature that has compared the effectiveness of reward- and deposit-based incentives as well as incentives of different sizes. Typically, diminishing or no effects of increasing financial incentive size are found (e.g., Jeffery et al., 1983; Finkelstein et al., 2007; Augurzky et al., 2012). Studies on health behaviour change that randomise respondents to deposit-based incentives (i.e., punishment) over reward-based incentives have found mixed effects (Halpern et al., 2015, 2018; Donlin Washington et al., 2016; Patel et al., 2016), and our study could help interpret mixed results in earlier work.

Third, by studying the characteristics of individuals opting-in and opting-out of deposit-based incentives in the treatment arm that combined a nudge with voluntary choice, we contribute to the understanding of the self-selection of individuals into deposit-based incentives. So far, the available evidence suggests that take-up of deposit-based incentives is higher among men and individuals with higher income (Halpern et al., 2016). Furthermore, some studies suggest that preference for immediate rewards (i.e., present bias) is associated with take-up of deposit-based incentives (Ashraf et al., 2006; Augenblick et al., 2015). Lipman (2020), however, found no such evidence, and furthermore, hypothetical take-up of deposit-based incentives was not associated with loss aversion. This is perhaps surprising, as one may also expect that take-up and/or effectiveness of deposit-based incentives is associated with loss aversion. In particular, motivation or effort may increase when loss averse individuals work towards preventing the loss of a deposit (compared to realising a gain of the same size), and if individuals anticipate this they may take up deposits. In the present study, preference for deposit-based incentives and the personal characteristics on which the informed advice was based, e.g., loss aversion, were elicited at baseline irrespective of the experimental condition, we can compare the effectiveness of the deposit-based incentive among different types of individuals. This enhances the understanding of which individuals benefit most from this type of incentive scheme. Hence, our work will provide further insight into the often supposed (but rarely studied) link between loss aversion and take-up and/or effectiveness of deposit-based incentives (Halpern et al., 2015).

Our findings suggest that respondents who are able to choose an incentive scheme after receiving advice allocate more effort and earn more than respondents randomly allocated to an incentive scheme. In respondents randomly assigned to incentive schemes, no effects of deposit-based incentives were observed compared with reward-based incentives. Interestingly, our results show that those who follow the advice to take up deposit-based incentives earn more and allocate more effort compared to those who were assigned to deposit-based incentives randomly, but no such effects are found for incentives based on rewards. The only predictor of take-up of deposit-based incentives was demand for commitment, which may suggest that the effect of deposit-based incentives could partially be due to sophisticated individuals self-selecting into deposits, and that sophisticated individuals are not easily identified through measures of present bias and loss aversion.

Materials and methods

Approval for this online experiment was provided by Erasmus School of Economics’ (ESE) internal review board, section Experiments (reference: Application 2021-09). Furthermore, we prepared a demo version of the experiment for review (https://erasmusuniversity.eu.qualtrics.com/jfe/form/SV_1TRtbxbrDFuNVvE).

Sample and recruitment

A sample of n = 228 respondents was recruited through the ESE Econlab panel, a system designed to recruit students for research participation. Panel members are typically (former) ESE students, i.e., students enrolled in economics, business or management programmes. Respondents were recruited through an email message inviting them to take part in a two-session study on the effect of different payments on effort in which they could earn up to 20 euro. The only inclusion criterion was having a Dutch bank account, as this facilitated digital payments. Note that the sample size for this study was determined to be in line with the budget available for the study (rather than being informed by a priori power analysis), but is generally in line with or larger than that of previous studies on (effects of incentives on) slider tasks (de Araujo et al., 2015; Gill and Prowse, 2019).

Experimental design

Figure 1 shows the timeline of this experiment, while Figure 2 shows the design of this experiment. We provided individuals with financial incentives for completing a tedious task. Two modes of assignment were compared, which were operationalised as between-subjects conditions:

• • random assignment arm: assignment to incentives (i.e., deposit-based vs reward-based lump sum) occurs randomly.¹

• • nudged assignment arm: a mode of assignment in which respondents have free choice between incentives, but receive informed advice on which to take in the form of a default from which respondents can opt-out.

Figure 1. Experimental timeline.

Figure 2. Study design and experimental flow (including drop-out).

We also randomised respondents into one of three payment conditions, which determined the maximum amount respondents could earn, €8, €12 or €20, respectively. We will henceforth referred to these conditions as low (€8)/medium (€12)/high (€20) payment. Respondents were informed about their payment condition simultaneously with their randomisation status. The online experiment consisted of three time points, T0, T1 and T2. Note that, as can be seen from Figure 1, the experiment mimics the effort-reward trade-off that typically occurs for behaviour change, i.e., effort allocation at T1 leads to a delayed reward at T2.

T0: baseline session

At T0, 228 respondents were recruited to take part in the study (after providing informed consent), completed a set of baselines measures (used for nudged assignment), practiced the tedious task and were informed about the incentives used in the study as well as the condition they were assigned to. The informed consent and first pages of the experiment also included details about the delayed payments this study used and the chance of potentially not earning anything if the experiment was not completed. Median completion time for session T0 was 18 min.

Baseline measures

Respondents completed a series of questions collecting demographics (age, sex, income and educational attainment), as well as measures of delay discounting, demand for commitment and loss aversion.

Delay discounting, in line with Boderie et al. (2020), was measured with the 27-item monetary choice questionnaire (MCQ) developed by Kirby and Maraković (1995), a standard and widely used measure of delay discounting (Kaplan et al., 2016). In the MCQ, respondents are asked to make a series of decisions between a smaller monetary amount paid out sooner (e.g., €54 today) or a larger amount paid out later (e.g., €55 in 117 days). Smaller-sooner amounts would be paid out today and ranged between €14 and €80, while larger-later rewards ranged between €25 and €85 and would be paid out between 7 and 186 days. Respondents with a stronger tendency to choose the immediate reward would display stronger delay discounting. The MCQ was developed to estimate discount rate k in a hyperbolic discount function (Mazur, 1987). That is, the present value of a delayed reward V can be expressed as V =  A/1 + kD, where A is the amount and D is the delay. In this study, we used the automated scoring developed by Kaplan et al. (2016) to estimate k. Furthermore, a non-parametric measure of discounting was used, i.e., the proportion of larger-later responses, which is derived as X/27 where X is the number of larger-later responses. Note that all rewards and delays were hypothetical.

Demand for commitment was measured by presenting respondents with a multiple-choice question. The following question was used: ‘Imagine you have made plans to invest some amount of effort on a task you would normally not enjoy much, but has benefits in the future, for example: exercising, doing taxes, going to the doctor/dentist. To make sure you actually stick to your plan next week, you are offered to pay a small deposit. That is, you can pay €5 that you will receive back in full if you indeed stick to your plan (i.e., go exercise, do the taxes, visit the doctor), but is lost if you forget or postpone. Would you pay this deposit?’. Respondents could answer: (1) Yes, absolutely, (2) Yes, probably, (3) I'm not sure, (4) No, probably not and (5) No, absolutely not. The first two answers are interpreted as having demand for commitment.

Loss aversion was, in line with Lipman (2020), measured with the non-parametric method (Abdellaoui et al., 2016). The method involves eliciting three chained indifferences between monetary gambles, enabling estimation of a loss aversion coefficient λ as defined by Köbberling and Wakker (2005). Loss aversion is defined with λ > 1 (λ = 1, λ < 1) indicating loss aversion (loss neutrality, gain seeking). More details on the implementation of the non-parametric method can be found in Supplementary Appendix A.

Tedious task and incentives for effort

After completing these measures, respondents were reminded that they had to complete a second online session (i.e., at T1), in which they would complete a set of tedious tasks. The tedious task was modelled after the slider task developed by Gill and Prowse (2019). In this task, respondents are asked to move adjustable slider bars to a specific point. Respondents were explained they could complete as little or many sliders as they wanted (between 0 and 400), and they would earn a reward for each task completed. After this explanation, respondents completed a practice task (i.e., one page with 20 sliders), such that they could judge the type of effort provision required of them. Finally, respondents were informed that they had earned a €4 show-up fee and received information about the incentives provided for their effort on the slider task in T1. The show-up fee would be paid when they completed the second session (T1), for which automated invitations were sent out exactly 1 week later. With the slider tasks they would earn a reward that would be paid out at T2, i.e., one week after completing the slider task. Effort-contingent payments were delayed to T2 to be able to draw parallels between the slider tasks and health behaviour change (e.g., exercise), as in these cases immediate effort is often traded off against a reward in the future.

The following parametrisation and incentives were used. Respondents completed P number of pages (P  ≤ 20) and each page consisted of 20 sliders that were set to 0. As such the total number of sliders respondents could complete before exiting the experiment was S = 20P, S ≤ 400. Each slider had to be moved to exactly 25. For each slider moved to 25, respondents would earn a reward r (i.e., paid out at T2). Two types of incentives were used: reward- and deposit-based incentives. Reward-based incentives entailed that respondents would earn $\euro ( S\ast r)$ at T2, as well as their show-up fee being paid out after completing the online experiment (these were paid within 4 h of completing the experiment). Note that respondents could also return at T1, complete 0 sliders and exit the experiment with their €4 show-up fee. Deposit-based incentives were operationalised by informing respondents that their show-up fee was added to the per-slider fee. That is, rather than receiving the show-up fee at T1, the per-slider reward r was increased to $\;r_d = r + ( \euro 4/S)$. In other words, respondents deposited their show-up fee and earned it back by completing sliders. The full deposit would only be earned back by completing 400 sliders. Hence, from a rational perspective, the deposit-based incentive is dominated by the lump-sum offered with reward-based incentives. However, the reward for effort is higher in the deposit-based incentive scheme, such that some respondents may use this incentive scheme as a commitment device to exert more effort while completing the task.

In designing our parametrisation of incentives, we were unsure how exactly respondents would respond to deposit-based incentives of different sizes. Hence, we decided to include incentives with different sizes, with the size of the deposit remaining constant. That is, payment conditions were operationalised by taking for the low (€8) condition: $r = \euro 0.01$, for the medium (€12) condition $r = \euro 0.02$ (medium) and for the high (€20) condition $r = \euro 0.04$ (high), respectively. With a €4 show-up fee, this implies for the low (€8) condition $r_d = \euro 0.02$, for the medium (€12) condition $r_d = \euro 0.03$ (medium) and for the high (€20) condition $r_d = \euro 0.05$ (high), respectively. As such, compared to reward-based incentives the per-slider reward would be increased by 100, 50 or 25%, respectively, when the show-up fee was deposited. Table 1 shows example pay-offs for respondents completing 0, 200 and 400 sliders.

Table 1. Example pay-offs in different conditions for respondents completing 0, 200 and 400 sliders

Assignment to incentives

In both treatment arms (i.e., random and nudged assignments), respondents received information on both incentive schemes, referred to as the basic and deposit scheme (see Supplementary material). Respondents in the random assignment arms (n = 115) were informed of the incentive they were randomly assigned to. Respondents in the nudged assignment arm (n = 113) received the following instruction: ‘In both the basic and deposit scheme you have the opportunity to earn the same amount of money for persistent effort on a tedious task. The deposit scheme may help to keep you motivated, however, because you are not only earning “extra” money, but also earning back money you made earlier. If you don't complete all tasks, however, you might lose money. You have the opportunity to choose between the two schemes, and you are free to choose whichever you prefer. However, based on the monetary choices and questionnaires you filled in earlier, we have preselected the scheme we think will help best to motivate you. Whether you follow our recommendation or not is your choice.’ In line with Boderie et al. (2020), the advice was based on pragmatic thresholds, as follows: whenever respondents (1) demanded commitment, (2) chose the larger-later reward in fewer than 14 out of 27 questions (i.e., proportion of larger, later rewards <50% indicating preference for earlier rewards) or (3) had λ > 2, they were recommended to take-up deposit-based incentives. If none of these three conditions was met, respondents were recommended to take-up reward-based incentives. The first two decision rules were modelled after Boderie et al. (2020), whereas the decision rule for loss aversion was based on the observation that, on average, λ typically is around 2 (Tversky and Kahneman, 1992; Lipman et al., 2019a).

T1: effort provision

One week after completing the T0 experimental session, respondents received an email invitation for session T1 (which had a median completion time of 37 min). One hundred and seventy-one respondents out of 228 showed up at T1, see Figure 1. In this session, the slider tasks were completed, but respondents first completed the MCQ, demand for commitment question and the non-parametric method again. This repeated measurement was included to test the robustness of the default selection in nudged assignment arm (i.e., would people receive the same advice?) and allows us to identify the test–retest reliability of the measures used in nudged assignment. After completing these measures, respondents were asked if they wanted to start on the sliders or finish the experiment and earn their show-up fee (if they had not deposited it). If they decided to start on the slider tasks, they could complete a page of 20 sliders. After each page respondents’ current earning (to be paid out at T2) was updated and they were again asked if they wanted to continue completing sliders or exit the task. Once respondents completed the final page with sliders or decided to quit, they were instructed to prepare up to two digital payment requests. The first involved the show-up fee (if not deposited) and the second involved slider earnings.

T2: payment

Payment requests were automatically sent to an inbox, with delivery of the payment request for slider earnings being delayed by exactly 1 week. The inbox was monitored on a daily basis, with payments being made as soon as possible (within ~4 h).

Data analysis

Descriptive statistics are provided for the sample of respondents at each time point and Chi-squared tests are used to assess differences between these samples. The main outcome measures are (1) effort and (2) earnings. Effort is operationalised as persistence (we will use these terms interchangeably) and is operationalised as the number of sliders a respondent completed at T1. We also report the number of pages (rather than the number of sliders), because the experiment was set-up such that after each page respondents were asked if they wanted to complete another page of sliders, and respondents would generally finish the whole page or quit the experiment. Earning reflects the total monetary reward earned. Following the study design, we first contrast the random and nudged assignment arm, followed by contrasting the reward- and deposit-based incentives within the random arm, and finally contrast the reward- and deposit-based incentive schemes within the nudged arm. Each contrast is explored descriptively, followed by regression models explaining effort or earnings based on each contrast. Finally, we performed linear regression models investigating the impact of each contrast on persistence and earnings corrected for payment condition. Supplementary Appendix B also contains a set of additional results, e.g., Kaplan-Meier survival curves for ‘survival’ in the experiment (i.e., continuing the slider tasks) and a set of regression analyses where persistence and earnings are modelled while controlling for demographics.

Results

Demographics

A total of 228 students participated in the first session, of whom the majority were master students with an income below €15,000 annually and were aged 18–24. Gender was roughly equally distributed with a slight majority of females. Of these respondents, 171 returned for the second session one week later (drop-out rate 25%). No evidence was found for selective drop-out according to the characteristics listed in Table 2, except by education (p = 0.031). First year students had a higher tendency to drop out. Although more respondents dropped out in the nudged assignment (see Table 2), this difference was not significant. The propensity to drop out was also not associated with assignment arm or payment condition.²

Table 2. Demographic characteristics of the study sample at both sessions with p-values for Chi-squared tests of independence comparing proportion returning for T1 vs drop-out.

Baseline and repeated measures

Table 3 shows the economic preferences elicited at T0 for the full sample, the sample that showed up for session T1 and the repeated measure elicitation. Drop-out is not related to economic preferences: we find no evidence for differences in demand for commitment (Chi-squared test, p = 0.99), loss aversion or delay discounting (t-tests, all p's > 0.16) between the full sample or the sample after drop-out.

Table 3. Baseline and repeated measures used for nudged assignment

LL, larger delayed reward.

Our data showed considerable hypothetical demand for commitment, as seen in Table 3. That is, 65% of the sample would demand commitment at T0 (66% among those that showed up for T1). In the repeated measure completed at T1, this was somewhat lower, with 56% demanding commitment. Chi-squared tests were suggestive of slightly lower demand for commitment in T1 and the repeated measure but the test missed significance (p = 0.063). Demand for commitment at T0 and T1 were significantly and strongly correlated (Pearson's r = 0.73, p < 0.001).

In session T0, we found considerable loss aversion. That is, 212 out of 228 respondents were loss averse (93%), and the same was true for 160 out of the 171 respondents returning for session T1. The proportion of loss averse respondents was similar for the repeated measurement in session 2 (92%). Comparing TO and T1 (3.69 vs 6.05, respectively), no statistically significantly difference was found (t(169) = 1.49, p = 0.112). The loss aversion parameters estimated at T0 and T1 were significantly but only small to moderately correlated (Pearson's r = 0.21, p = 0.004).

Table 3, furthermore, shows that across all sessions a slight majority of respondents preferred the larger delayed rewards in most items. The k-parameter indicates the degree of sensitivity to delay, where the small numbers in Table 3 indicate that respondents were generally not strongly discounting delays. Furthermore, the test–retest reliability of the MCQ appears reasonable, as differences between measurements appeared small. The k-parameters elicited at T0 and T1 were significantly and strongly correlated (k-parameter: Pearson's r = 0.65, p < 0.001). Finally, we checked if (in)stability of economic preferences would affect the recommended incentive scheme. For 89.5% of our respondents, the recommended incentive scheme would have been the same in both sessions.

Payment conditions

Figure 3 shows the mean earnings per payment condition, separated by assignment arms and incentive scheme. Perhaps unsurprisingly, the proportions of respondents completing the slider task differed between payment condition, such that it was highest for the high payment condition and lowest for the low condition. When looking at the effect of payment condition on persistence, we find some evidence for differences in persistence between payment condition. That is, respondents in the low (€8), medium (€12) and high (€20) payment condition completed 235, 273, 315 out of 400 sliders, earning €6.10, €8.83 and €16.01, respectively (see Supplementary Appendix C for a figure that shows the amount of sliders completed across all subgroups in the experiment). Given that the total monetary amount available might influence persistence of respondents, we included payment condition as a control variable in a set of regression analyses investigating differences in persistence and earnings in different subsets of the sample. Table 4 also shows the effect of payment condition across different specifications. It appears that the high (€20) payment condition generally yielded higher effort and earnings, whereas the significance of medium (€12) payment condition depended on model specification.

Figure 3. Mean earnings (in cents) by assignment arm, payment condition and incentive scheme.

Table 4. Linear regression analysis results with persistence and total earnings as independent variables

Note:

Model 1: Effect of choice on persistence and earnings.

Model 2: Effect of deposit-based incentives on persistence and earnings when it is not a choice.

Model 3: Effect of deposit-based incentives on persistence and earnings when it is a choice.

Model 4: Effect of choice on persistence and earnings among those who have a deposit-based incentive scheme.

Model 5: Effect of choice on persistence and earnings among those who have a reward-based incentive scheme.

Contrast 1: the effect of (nudged) choice

With 115 respondents in the random assignment arm and 113 in the nudged assignment, arm distribution was spread evenly at T0 (as expected given treatment arms were randomly assigned). Drop-out in the nudged arm (32 out of 113, 23%) compared to the random arm (25 out of 115, 22%, see also Figure 1) was not statistically significant different (Chi-squared test, p = 0.320). Visual inspection of respondent persistence (see Figure 4) suggests that a difference in starting point and a difference in slope exists between the arms. The starting point reflects the percentage of respondents that started slider tasks, and very few respondents completed the experiment without allocating any effort. A similar pattern can be observed for the total earnings between the random and nudged arm, where the earnings in the nudged arm are higher, but the statistical test missed significance (t-test, p = 0.081). Table 4 model 1 shows the regression equivalent of contrast 1 taking payment condition into account. Having a choice (i.e., the nudged assignment arm) was positively associated with both persistence (β = 48.4, p = 0.044) and earnings (β = 189.6, p = 0.011). Hence, conditional on payment condition, nudged assignment with an opt-out significantly increased both effort and earnings compared to randomly assigned incentives.

Figure 4. Slider pages completed (i.e., persistence) and earnings (in cents) for each contrast our design enables.

Contrast 2: the effect of deposit-based incentives (for random assignment)

Within the random assignment arm, respondents were evenly distributed among reward- and deposit-based incentives schemes. Drop-out was not statistically different (Chi-squared test, p = 0.250) between those randomly assigned deposit-based incentives (15 out of 55, 27%) and those assigned to reward-based incentives (10 out of 60, 17%). Figure 4 panel 2 shows that persistence and earnings were similar between the two incentive schemes. In Table 4, Model 2 regression results showed no significant differences between reward- and deposit-based incentive schemes corrected for payment condition for both persistence and earnings. Those randomised to the deposit-based scheme seemed to earn less, but this effect missed statistical significance (β = −214.0, p = 0.068). This is intuitive: if persistence is not higher among those who take up for deposit-based incentives, then total earnings will be lower in this group since they have given up their show-up fee.

Contrast 3: the effect of deposit-based incentives (for nudged arm)

Contrast 3a: who chooses deposit-based incentives?

Most respondents in the nudged assignment arm were advised to take up deposit-based incentives (i.e., 100 out of 113 respondents). This result is explained by the high hypothetical demand for commitment as well as considerable loss aversion.³ Interestingly, when respondents were recommended to take reward-based incentives, 100% of the respondents adhered to this advice. In contrast, only 44% adhered to the advice to choose a deposit-based scheme.⁴ Next, we explored if the demographics were associated with adherence to the informed advice implemented in the nudged assignment arm, as well as investigating if the baseline measures that determine the advice are related to adherence. In a set of univariate tests (i.e., t-tests or Chi-squared tests), we found no evidence in favour of associations between demographics and adherence to informed advice. As such, respondents’ propensity to take the informed advice was not dependent on their demographics. Similar, respondents' propensity to take the informed advice was neither dependent on economic preferences, i.e., delay discounting, demand for commitment and loss aversion. This result was robust to excluding those receiving the advice to take reward-based incentives, of whom everyone took the advice, and no one deviated. Further multivariate analyses supported the previous findings. The only evidence in favour of an association between advice adherence and economic preferences was observed when restricting the data to only those who showed up for both sessions, i.e., ignoring drop-out. In this model, demand for commitment was associated with advice adherence (β = −0.30, p = 0.010), suggesting that those who demanded commitment were more inclined to adhere to their nudged assignment. Note that we also explored if persistence and earning depended on demand for commitment, delay discounting and loss aversion in multivariate regressions but found no evidence for associations between for both effort and earnings (all p-values >0.341, see Supplementary material).

Contrast 3b: the effect of deposit-based incentives in those who chose them

Within the nudged assignment arm, respondents were free to choose the incentive scheme of their preference, hence the distribution was not equal between the two schemes. Sixty-nine out of 113 (61%) respondents chose reward-based incentives while 44 (39%) chose the deposit-based incentive scheme. Since this assignment was not random, any possible difference between the two groups reflects a combination of effects of the incentive scheme and selection bias deriving from a self-selection of respondents into the incentive scheme of their choice. Drop-out was nearly identical between the schemes at T1. The persistence and earnings within the deposit-based incentive group were not statistically significantly different (see Table 4: Model 3). Finally, we looked at the effect of assignment in those who received deposit-based incentives (Model 4) or reward-based incentives (Model 5), while taking into account the payment condition. As seen from Model 4, having a choice did have a significant effect on earnings for respondents who chose a deposit.

Discussion

In this study, we studied the effect of both the assignment mode of incentives, as well as the incentive scheme in an incentivized real-effort task for incentives of different sizes. Two modes of assigning respondents to either reward- or deposit-based financial incentives are compared: random assignment and ‘nudged’ assignment. In the nudged assignment condition, assignment was based on respondent characteristics (discounting, loss aversion and demand for commitment) allowing opting out. This design allowed us to expand the literature in several directions, discussed below.

Random vs nudged assignment to incentive designs

Earlier work has shown that those who have the opportunity to choose their own incentive scheme perform the same or worse than those randomly assigned to incentives (Chaudhry and Klinowski, 2016; Adjerid et al., 2021; Woerner et al., 2021). The findings of our study are in contrast to that literature, as it appears that respondents allocated more effort and earned slightly more when offered a choice among incentives (after controlling for payment condition). Note that this is just one interpretation of our findings, our results may also be explained by respondents being randomised to incentives performing worse (perhaps because of disappointment with their assignment). A key difference between our study and earlier work is the main dependent variable: in our work, effort on a tedious task and, in other studies, different forms of health behaviour. Hence, whether or not choice among incentive schemes is beneficial may be related to how difficult it is to anticipate what type of incentives will help them stay motivated. In particular, Woerner et al. (2021), in the context of meditation, find that given the choice between incentive schemes respondents sort into incentives that would theoretically be optimal given their anticipated meditation benefits. Their findings, however, suggest that choice has a negative effect on meditation frequency, but only for respondents that did not meditate before the study. This may suggest that choice among incentive schemes is not beneficial when individuals lack the experience needed to anticipate the effort needed and benefits associated with some behaviour. In our case, respondents practice the real-effort task and due to the simple nature of the experiment, the effort needed and benefits associated with completing slider tasks should have been clear. Potentially, choice between incentive schemes is beneficial tasks for which the respondent can easily anticipate effort needed and benefits accrued (i.e., slider tasks) and (potentially) detrimental for complex behaviours (e.g., health behaviour change). Recent evidence (Incekara-Hafalir et al., 2023) suggests that choice between incentives may be beneficial for student selecting into different grading schemes for a series of small exercises as part of a course. This may be interpreted as being in line our results. Arguably, students are able to anticipate the effort needed and benefits accrued from completing coursework on time, which would explain why as in our study, Incekara-Hafalir et al. (2023) find beneficial effects of choice.

Reward- vs deposit-based incentives of different sizes

The second key contrast in this study was between reward- and deposit-based incentives of different sizes. In particular, we compared effectiveness of these incentive schemes in those randomly assigned to them, as this avoids self-selection. We find that respondents were slightly less likely to show up for the next session of our experiment if they were assigned to deposit-based incentives. This appears to be in line with the low attractiveness of deposit-based incentives observed in earlier work, in which voluntary take-up was typically low (Giné et al., 2010; Halpern et al., 2015; Royer et al., 2015), however this effect was not statistically significant. Importantly, effort provision was not affected by the type of incentive scheme respondents were assigned to, and as a result respondents earned less when they were randomly assigned to deposit-based incentives. This finding is in contrast to work by Patel et al. (2016) who found larger effectiveness for deposit-based incentives (compared to reward-based incentives) for physical exercise. On the other hand, our null-result is in line with findings by Halpern et al. (2018) for smoking cessation. Hence, it appears the use of deposit-based incentives is beneficial compared to not using incentives at all (Giné et al., 2010; John et al., 2011; Royer et al., 2015), but it remains unclear if deposit-based incentives outperform incentives that do not involve losses. Future work may compare reward- and deposit-based incentives in a design that also includes a control condition without incentives to further explore this issue. Furthermore, our study included incentives of different sizes. Increasing reward size by 50 and 150% significantly increased persistence by ~16 and 34%, respectively, which seems to suggest some degree of diminishing sensitivity to increasing reward size. This finding is in contrast to de Araujo et al. (2015), who find that persistence on slider task is largely insensitive to reward sizes. We do find that deposit-based incentives combined with low payments seem to yield a higher chance of drop-out (see Supplementary material), which may suggest take-up of deposit-based incentives take-up can be increased by increasing total reward size.

Take-up of deposit-based incentives

Using a set of pragmatic decision rules (modelled after Boderie et al. (2020)), we advised respondents that demanded commitment and/or displayed considerable delay discounting/loss aversion to take-up deposit-based incentives (other respondents were advised to take-up reward-based incentives). This advice was implemented as a pre-selection of the advised incentive scheme, i.e., as a default. We find that many individuals display demand for commitment, as well as strong discounting and considerable loss aversion using existing methodology (Kirby and Maraković, 1995; Lipman, 2020). Our estimates for delay discounting as well as their test–retest reliability appear to be in line with previous work (Matousek et al., 2022). Our estimates for loss aversion appear to be slightly larger than previous work (Abdellaoui et al., 2016; Brown et al., 2021), and the low test–retest reliability we observe adds to a literature on the (lack of) stability of loss aversion (Yechiam and Hochman, 2013; Lipman et al., 2019a). Given the low test–retest reliability, it may seem inadvisable to use loss aversion estimates for assigning incentives, yet in our study, the scheme respondents were defaulted to, using a pragmatic approach similar to Boderie et al. (2020), would have been the same across test and retest for a large majority of respondents. Take-up of these default schemes was high, 100% for reward-based incentive schemes and a considerable 44% for deposit-based incentive schemes. In other words, nearly half of our respondents elected to voluntarily deposit part of the money they earned in order to commit themselves to complete more sliders, even though no matching was applied. Take-up of deposit-based incentives in our study is larger than in some published studies using deposit-based incentives without matching (Ashraf et al., 2006; Giné et al., 2010; Royer et al., 2015), in line with the high hypothetical take-up typically found in lab-based studies (Lipman, 2020; Adjerid et al., 2021). Carrera et al. (2019), however, estimated that the average take-up rate of deposit contracts for earned money, as in our study, was 47%, i.e., quite close to the take-up in this study. This would imply that the use of defaults appears to add little to take-up of deposit-based incentives, in contrast to what was suggested in work done with commercially available deposit-based incentives, i.e., on www.stickk.com (Goldhaber-Fiebert et al., 2010; Bhattacharya et al., 2015). Yet, another treatment arm with voluntary choice without a default would be needed to draw this conclusion.

Predicting take-up of deposit-based incentives

Although the advice to take-up deposit-based incentives was based on discounting, loss aversion and demand for commitment, only the latter was associated with take-up (when ignoring drop-out). These results are in accordance with work by Lipman (2020), who also found no association between discounting and loss aversion and uptake of deposit-based incentives. Yet, this null-result for both discounting and loss aversion remains puzzling. In a study on the use of deposit-based incentives to promote savings, Ashraf et al. (2006) found evidence that take-up was associated with discounting, and loss aversion is often discussed as a reason for low take-up of deposit-based incentives (Halpern et al., 2015). The association between demand for commitment observed in this study suggests that it is particularly ‘sophisticated’ individuals who take-up deposit-based incentives. That is, individuals who realise they need commitment to perform tedious tasks realise that without such commitment in place they would not perform the behaviour even if they planned to do it. Sophisticated individuals realise they have such time-inconsistent preferences (Laibson, 1997), and look for ways to constrain their future choices. In our study, this meant taking-up deposit-based incentives, and, as such, voluntary taking-up an incentive scheme in which previously earned money is only returned in full if respondents complete all slider tasks. The association observed between demand for commitment and take-up of deposit-based incentives, therefore, shows that individuals who believe they want commitment in a hypothetical context actually restrict themselves.

Furthermore, our results suggest that among respondents receiving deposit-based incentives, those that chose them (by taking-up our advice) completed more tasks and earned more than those we randomly assigned to deposit-based incentives. Interestingly, this analysis showed that payment condition was no longer a significant predictor of the amount of slider tasks. A potential explanation is that when respondents get a choice in selecting their own incentive scheme, more motivated respondents self-select into a deposit-based incentive scheme, explaining the higher levels of effort among those with deposit-based incentive schemes in the nudged assignment. We find no such effect of choice for reward-based incentives. Collectively, these results seem in line with Dizon-Ross and Zucker (2021) who find that respondents use free choice among incentives to commit themselves to future actions. That is, those that expect to need commitment and are willing to pay a deposit self-select into deposit-based incentives and as a consequence earn more. This result, therefore, may also caution against widespread use of (nudged) choice, as sophistication is not the only predictor of demand for commitment. Willingness to enter into incentive schemes with deposit-based incentives may also be associated with ability to pay, i.e., income or socio-economic status.

External validity

The real-effort task used in this experiment differs in many structural ways from health behaviour in the field: e.g. (i) the pay-off structure is completely transparent and predictable as opposed to field behaviour where benefits may be uncertain, (ii) we use a one-time tedious task, whereas most field behaviour requires persistent effort and (iii) field behaviour may involve additional benefits beyond just monetary incentives (e.g., better health). This (non-exhaustive) list of differences implies that we cannot rule out limited external validity for informing related work in modes of assigning incentives for health behaviour (Adjerid et al., 2021; Dizon-Ross and Zucker, 2021; Woerner et al., 2021). Nevertheless, in designing our study, we deliberately designed effort-contingent rewards such that they bear similarities with health behaviour. For example, respondents completed an effortful task (sliders) at T1 to receive payment a week later at T2, which may be taken to resemble trade-offs between effort provision now and (health) benefits later that are present in, e.g., smoking cessation and when going to the gym. The findings of this study suggest that, at least for simple, tedious tasks completed in one session, choice between incentives (including advice) may be beneficial. Hence, our findings may apply more closely to field behaviour that is also simple and tedious (e.g., hand hygiene, see Talbot et al. (2013)) or for provision of incentives for one-shot decisions (e.g., vaccination, see Campos-Mercade et al. (2021)). Regardless, future work should extend our design to field behaviour to further explore the external validity or our findings.

Limitations

Besides taking into account potentially low external validity, the findings of this study should be interpreted in light of a set of limitations. First, the informed advice we provided respondents with was, in line with Boderie et al. (2020), based on a set of pragmatic cut-offs determined a priori, which raises several issues. For example, under the current specification, a large majority of respondents was given the advice to take-up deposit-based incentives. This may be considered problematic, as in fact, in our study reward-based deposits dominate deposit-based incentives, as respondents can only be equally good off and only if they complete all tasks (in all other cases they earn less for the same effort). In our study, we felt we needed to clarify why respondents would consider deposits if there was no matching. One may argue that these added explanations that are only included to deposit-based incentives and in the nudged assignment may be considered a form of framing. We are unable to disentangle framing from effects of mode of assignment or incentive schemes, which requires a different type of design (e.g., de Buisonjé et al., 2022). As such, future research should avoid such strong framing, and preferably include a theoretical model of incentive design (e.g., Gonzalez-Jimenez, 2022; Incekara-Hafalir et al., 2023), which may allow designing optimal incentives for respondents given their time and risk preferences. Such work could also explore alternative modes of assignment, e.g., if respondents’ characteristics can be measured beforehand for them to be assigned optimal incentive schemes (without opt-out). Second, our study used a relatively small sample, which suggests that both the test power for the contrasts included may be questioned.⁵ Third, the economic preferences (i.e., delay discounting and loss aversion) were elicited for hypothetical rewards. Typically, it is preferred for risk and time preferences to be elicited with incentive-compatible procedures (Galizzi and Wiesen, 2018), i.e., with procedures that translate to real payments. Fourth, in this study, economic preferences, including demand for commitment, were measured before offering respondents choice of reward- or deposit-based incentives. As such, respondents may have felt a need to act consistent with their hypothetical demand for commitment, which would not have been observed if this question was not asked. Hence, it is possible the current order biases take-up of deposit-based incentives, and explains why demand for commitment was associated with take-up of deposit-based incentives. Future work should consider randomising the order of the demand for commitment question (and other economic preferences), to be able to estimate the size of this bias. Fifth, the measures applied for estimating discounting and loss aversion also suffer from limitations, which thus also apply here. For example, the MCQ used to measure discounting uses relatively small monetary amounts, and it is well-known that discounting is larger for smaller amounts (Attema, 2012). Furthermore, the MCQ is not able to estimate discount rates for highly impatient individuals (Towe et al., 2015). The non-parametric method used to estimate loss aversion, on the other hand, involves chained indifferences, meaning that it is not incentive-compatible and may be sensitive to error propagation (Abdellaoui et al., 2016; Lipman et al., 2019b). Finally, in the design that was used for this study, it was impossible to disentangle the effect of having the opportunity to choose incentives from receiving informed advice. Disentangling these effects would require comparing nudged assignment with a condition in which respondents choose incentives without receiving advice.

Conclusion

To conclude, our study provides evidence that offering respondents free choice of incentives, including advice on which to take, may be beneficial in enhancing effort. That is, respondents that could self-select into incentives earned more and allocated more effort than those randomly allocated to an incentive scheme. We find that this may be driven by self-selection into deposit-based incentives by sophisticated individuals, which suggests that offering choice among different incentives scheme could only be beneficial for the subgroup that is sophisticated about requiring a deposit scheme to maximise their long-run utility.