Eliciting risk preferences in an artefactual field experiment via replication and an alternative approach

Abstract

This paper investigates risk preferences using an artefactual field experiment conducted with a non-standard subject pool of farmers in Ghana. I introduce an alternative methodology for studying preferences following replication of a seminal risk elicitation procedure by Binswanger (Am J Agric Econ 62(3):395407, 1980). An important feature of both approaches is that they are easy to understand and, hence, are particularly suitable for eliciting preferences among subjects with low levels of formal education. I successfully replicate Binswanger's study, documenting how his original result of the moderate level of risk aversion for an average farmer can be generalized to a different country. However, using my alternative approach, whereby lotteries are presented in the loss domain, I find that half of my experimental subjects violated expected utility theory. This approach is of relevance to the current literature on studying risk preferences among subjects with poor literacy skills.

1 Introduction

In this paper, I introduce an alternative methodology to measure subjects’ risk preferences. My elicitation procedure builds on Binswanger's seminal work (1980). The original risk elicitation procedure was applied by him to study attitudes toward risk among Indian farmers. Importantly, both Binswanger's and my procedure are easy to understand by experimental subjects, which is crucial for precisely studying preferences among subjects with low levels of literacy. I first use Binswanger's methodology as a replication of the case of Ghanaian farmers,¹ followed by my alternative procedure, where the same decision-makers choose among identical options except that the options are presented in the loss domain. This approach enables the identification of subjects who violate expected utility theory, and, hence, is of relevance to the recent literature focusing on the behavioral explanations for decisions made by subjects from populations characterized by low rates of literacy.

The importance of studying the risk preferences of decision-makers has long been emphasized by economists (Holt & Laury, 2014). This also has led to the need to introduce methods to measure preferences toward risk with the aim of inducing decision-makers to reveal their preferences (Harrison & Rutstrom, 2009). Binswanger's work (1980) has been extremely influential in this field, as he proposed the first elicitation procedure that enabled the precise measurement of risk parameters from expected utility theory.

Binswanger's original procedure consists of subjects choosing one incentivized lottery from an ordered set, where the first lottery is the risk-free option and subsequent ones are characterized by increasing expected payoffs and increasing variance around these payoffs. Harrison and Rutstrom (2008) classify this design as an ordered lottery selection procedure. The seminal work by Binswanger has been applied to numerous contexts and has also included a range of modifications. Most notably, Eckel and Grossman (2002) used the procedure with a smaller set of lotteries and also used a standard subject pool of university students.² Given that the subjects in Binswanger's study were not university students and were presented with abstract lotteries, the experiment falls into the category of artefactual field experiments (Harrison & List, 2004). Binswanger's procedure was also applied in other artefactual field experiments, with the non-standard subject pool comprising not farmers but village households (Barr et al., 2012). The procedure was also used in studies with subjects from developing countries, with decisions being hypothetical and, hence, without monetary incentives (Ashraf et al., 2009; Bryan, 2019; Giné & Yang, 2009).

Aside from the ordered lottery selection procedure introduced by Binswanger (1980), the most common risk elicitation procedure is the multiple price lists popularized by Holt and Laury (2002). The key feature of this methodology is that subjects are presented with an ordered set of binary lottery options and are required to make choices in all lotteries at once (Harrison & Rutstrom, 2008). Experiments were conducted in a range of studies to compare subjects’ behavior across the procedures of ordered lottery selection and multiple price lists. While the latter approach enables one to identify a more precise set of risk parameters based on the assumptions of the functional form of utility, its relatively high level of complexity implies that it is typically not well understood by subjects with low levels of mathematical skills (Dave et al., 2010). Given this drawback, the simpler procedure introduced by Binswanger is more suited to eliciting the risk preferences of low numeracy populations (Charness et al., 2013).

This study contributes to the literature on studying risk preferences by introducing an alternative methodological approach for eliciting attitudes toward risk. My procedure has two main advantages: It enables the identification of subjects in violation of expected utility theory, and it can easily be understood by the experimental subjects, which is particularly important for studying risk preferences among subjects with low levels of formal education.

My first finding is that I successfully replicate Binswanger's (1980) experiment using a similar non-standard subject pool of farmers from another country. Similar to Binswanger, I find that the average experimental subject exhibits a moderate level of risk aversion. My second finding is that the decisions of half of the subjects cannot be explained by expected utility theory. Specifically, when using the alternative methodology, whereby identical abstract lotteries are framed in the loss domain, 51% of subjects choose lotteries that were different from their preferred lottery in the gain frame.

The remainder of this paper is structured as follows: Sect. 2 presents the experimental design. Experimental results are presented in Sect. 3. The findings are discussed in Sect. 4. Section 5 concludes the paper.

2 Experimental design and sample description

This paper investigates farmers’ attitudes toward risk using abstract lotteries. The experiment was undertaken in villages in the cocoa producing Ashanti region in Ghana. The villages were randomly selected from the list of villages used in the Centre for the Study of African Economies (the University of Oxford) survey (collected biannually since 2002). The dataset consists of 350 farmers, invited at random to participate in the experiment by means of accessing the lists of cocoa farmers of all licensed buying companies operating in a given village. All subjects who were invited participated in the experiment. Sessions were conducted in the local Twi language by trained experimenters who always kept the same roles in all experimental sessions.

There were 20 experimental sessions in total, which were conducted in local schools, with an average number of approximately 18 subjects per session. A typical session lasted approximately 40 minutes, and the average experimental winnings (paid in GHC, the local currency New Ghanaian Cedis), were approximately GHC 7.5, including a show-up fee of GHC 2. These winnings were equivalent to approximately two daily wages for hired labor on farm in the local area at the time when the experiment was conducted.

Great care and efforts were made to ensure the privacy of the decision-making among the subjects, as well as the subjects’ understanding of the experiment and of the probability distribution associated with the choices presented to subjects. At the beginning of each session, subjects were randomly allocated their seat in the location of the experiment. After describing the structure of the experiment, subjects were asked questions to test their understanding of the instructions. Furthermore, a number of visualization tools and demonstration draws were used. Each subject was also asked to answer questions measuring understanding of the experiment.

2.1 Elicitation of risk preferences

The experiment elicited risk preferences by means of two incentivized ordered lottery selection procedures presented to the subjects in random order.³ The first procedure mimics the methodology introduced by Binswanger (1980). The second procedure is an alternative methodological approach whereby the identical abstract lotteries used in the first procedure are presented in the loss domain. Given that individuals make decisions in both procedures (i.e., a within-subject experimental design was used), the dataset enables me to investigate whether the decision-making of the experimental subjects was consistent with expected utility theory. Importantly, both procedures are easy to understand, and therefore particularly suitable for studying risk preferences among subjects with low level of formal education (Charness et al., 2013).

Figure 1 shows the first elicitation procedure. Farmers were presented with a set of six lotteries (A, B, C, D, E, and F) and were asked to pick the most preferred option. Each lottery consisted of two equally likely outcomes. The outcome of the lottery was determined by a draw of one of the tokens (a low outcome if a black token was drawn and a high outcome if an orange token was drawn). For instance, a subject who chose lottery A would win 0 if a black token was drawn or win 10 if an orange token was drawn. The mean and variance were highest in lottery A and gradually decreased in subsequent lotteries, with lottery F offering the lowest mean and no variance across low and high outcomes (i.e., winnings of 3 with certainty).

Fig. 1 Presentation of Binswanger Game (gain domain)

The benchmark used for analyzing my data is expected utility theory and I consider the utility function of the following form:

(1) $U\left(x\right)=\left(\begin{array}{cc}\frac{x^{1-\sigma }}{1-\sigma }& for\sigma \ne 1\\ \text{ln}\left(x\right)& for\sigma \ne 1\end{array}\right)$

where x is the lottery prize and $\sigma$ is the risk aversion parameter. With the specification of the constant relative risk aversion (CRRA), individuals with $\sigma <0$ are risk loving, individuals with $\sigma =0$ are risk neutral, and individuals with $\sigma >0$ are risk averse.^{4 , 5} Under expected utility theory, a decision-maker evaluates each outcome $kϵ(1,\cdots ,N)$ using objective probabilities p(k), and, hence, the expected utility from choice $iϵ(1,\cdots ,6)$ is equal to:

${\text{EU}}_i=\sum _1^{N}p\left(k\right)U\left(k\right)$

Table 1 shows the mean and variance of each of the six lotteries. The last column of the table shows the range of the coefficients of risk aversion $\sigma$ for which a given lottery choice is optimal. Since lottery A offered the highest mean and the highest variance, this lottery should have been chosen by the least risk averse individuals ( $\sigma$ ranging from − ∞ to 0.17). Subsequent lotteries were characterized by decreasing mean and variance, and should be chosen by the individuals with increasing value of $\sigma$ . Lottery F offered the lowest mean and zero variance, and should thus be chosen by the most risk averse subjects ( $\sigma$ ranging from 1.42 to + ∞).

Table 1 The lottery payoffs, variance, and corresponding range of risk aversion (gain domain)

Gain domain (GHC 0 at start)	50% (low outcome)	50% (high outcome)	Mean	Variance	CRRA range
A	0	10	5	25	(− ∞) – 0.17
B	0.5	9	4.75	18.06	0.18 – 0.45
C	1	7.5	4.25	10.56	0.46 – 0.64
D	1.5	6	3.75	5.06	0.65 – 1.00
E	2	4.5	3.25	1.56	1.01 – 1.41
F	3	3	3	0	1.42 – (+ ∞)

The second risk elicitation procedure was similar to the first approach except that the corresponding lotteries were presented in the loss domain. The only difference was that in the second procedure subjects’ real payoffs would be determined by subtracting possible real outcomes from a hypothetical endowment of GHC 10. For example, a subject choosing lottery A would lose GHC 10 (implying a real payoff of GHC 0) if a black token was drawn or lose GHC 0 (implying a real payoff of GHC 10) if an orange token was drawn.

The risk elicitation procedures in the gain and loss domain are equivalent in terms of expected utility. Therefore, the ranges of the CRRA in the loss domain (henceforth referred to as ${\sigma }_{\text{L}}$ ) are identical to the CRRA in the gain domain (henceforth referred to as ${\sigma }_{\text{G}}$ ) described in Table 1. This implies that an expected utility maximizer should make identical choices in the gain and loss domain. Conversely, the choice of different lotteries in the two risk elicitation games would violate the predictions of expected utility theory.

2.2 Sample description

Table 2 displays the key descriptive statistics obtained in the short post-experiment questionnaire from 350 experimental subjects. The subjects are small-scale cocoa farmers with an average farm size of 3.4 ha. In my sample there is a slightly higher proportion of males, and the average age is equal to 56 years. The majority of the participants are married and household heads. A crucial characteristic of my sample is a low level of formal education. This is a very important feature as subjects with little education may have difficulties in understanding complex risk elicitation procedures (Dave et al., 2010).

Table 2 Descriptive statistics

	Mean	Std. Dev
Farm size (ha)	3.396	3.139
If male	0.642	0.480
Age	56.286	13.495
Education level	2.284	1.441
Household size	6.823	4.081
If household head	0.769	0.422
If married	0.754	0.431

Sample size is equal to 350. Education level takes integer values between 0 and 5 corresponding to the highest completed educational stage

3 Experimental results

3.1 Results from eliciting risk preferences in the gain domain

In order to obtain the average value of the CRRA in my sample, I follow Binswanger (1980) and convert each of the six ranges of CRRA coefficients (corresponding to an individual's choice among six available lotteries) in Table 1 into a single number. Specifically, I take the midpoint of the range of the CRRA coefficient corresponding to a choice of a particular lottery (e.g., choosing lottery C implies a CRRA range from 0.46 to 0.64 in Table 1 and, hence, a midpoint equal to 0.55).⁶

The dataset enabled me to obtain the average value of the CRRA parameter in my sample. This is calculated by multiplying the proportions of lottery choices reported in Table 3 by the CRRA midpoints corresponding to each lottery. The estimated value of the average CRRA is equal to 0.68, which is a moderate degree of risk aversion. This value is very similar to the mean CRRA of 0.71 among Indian farmers reported by Binswanger (1980).

Table 3 Percentage distributions of lottery choices (gain and loss domain)

Lottery	${\sigma }_{\text{G}}$ range	Midpoint	Lotteries in the gain domain	Lotteries in the loss domain
A	(− ∞) – 0.17	0.085	32%	24%
B	0.18 – 0.45	0.315	19%	14%
C	0.46 – 0.64	0.55	13%	11%
D	0.65 – 1.00	0.825	7%	15%
E	1.01 – 1.41	1.21	6%	7%
F	1.42 – (+ ∞)	1.71	23%	29%
			100%	100%

It is noteworthy that the estimated value of the CRRA for each of the subjects’ preferred lotteries depends on how the CRRA ranges are actually calculated, particularly the ranges for lotteries A (due to its lower bound of − ∞) and F (due to its upper bound of + ∞). To address this issue, I follow Binswanger (1980) and investigate whether the result of average risk aversion is confirmed using alternative calculations of the values of the CRRA ranges. Specifically, I also obtain these values using geometric means.⁷ I also censor at different values the lower bound of lottery A and the upper bound of lottery F. While these changes result in different values of the average CRRA, the results do not vary significantly.⁸ Most importantly, using these alternative approaches I also obtain the main result of the moderate degree of risk aversion.

Column (4) in Table 3 presents the proportions of subjects choosing particular lotteries in the gain domain, namely, when the risk elicitation procedure follows Binswanger's (1980) approach. Subjects preferring the riskiest Lottery A are classified as either the most risk loving or the least risk averse. 33% of subjects chose this lottery. Lottery F was risk-free, and it was chosen by extremely risk averse subjects. It was chosen by 23% of the subjects. A choice of lottery B, C, D or E was made by individuals with moderate degrees of risk aversion. In total, 45% of subjects chose one of these lotteries.

3.2 Results from eliciting risk preferences in the loss domain

My alternative risk elicitation procedure consists of presenting subjects with the above six lotteries in the loss domain. The lottery choices made by the subjects using this method are reported in column (5) in Table 3. 46% of subjects chose lotteries B, C, D or E (i.e., lotteries with moderate expected payoff and variance), which is a similar proportion to the corresponding proportion in the gain domain. However, preferences for lotteries A and F differed markedly across the two domains. Only 24% of subjects chose lottery A in the loss domain, compared to 32% in the gain domain. In contrast, a significantly larger proportion of subjects chose lottery F in the loss domain than in the gain domain (29% and 23%, respectively).

While the average CRRA obtained from the choices in the loss domain equals 0.83, and, hence, also indicates a moderate degree of risk aversion, the differences in the proportions of the choices of lotteries provide some evidence that many subjects did not choose identically across these two games.⁹ I investigate this possibility further by performing tests for equality of means, which are reported in Table 4. The null hypothesis of mean equality is strongly rejected in the paired sample t test (p = 0.001). I also test this hypothesis by the bootstrap test for mean equality. I obtain the bootstrap statistic using bootstrap samples drawn with replacement from the original sample. I reject the null hypothesis at the 0.01 level using the bootstrap test, which provides further evidence that the mean difference is statistically significant.

Table 4 Tests of equality of means (gain and loss domain)

	Paired sample t test	Bootstrap test
Test statistic	t = − 4.44	z = − 4.55
p value	0.001	0.001

Bootstrap test uses 9999 bootstrap samples

In order to better understand the choices made by the experimental subjects, I investigate risk preferences at individual level by combining choices of each subject both in the gain and loss domain. Table 5 displays the pairs of choices made by all 350 subjects in both domains. Six lotteries in the gain domain are displayed along the first column, while six lotteries in the loss domain are displayed along the first row.

To further explore the patterns documented in Table 5, I divide the sample into three groups. The first group consists of subjects who made identical choices in the gain and in the loss domain (diagonal elements of matrix in Table 4). Due to the fact that the two risk elicitation procedures were identical in expected utility terms, the CRRA parameter in the gain domain (i.e., ${\sigma }_{\text{G}}$ ) is the same as the CRRA parameter in the loss domain (i.e., ${\sigma }_{\text{L}}$ ) for the subjects in this group.

Grouping the data into the other two categories helps to distinguish between those whose risk aversion coefficient in the gain domain was greater than that in the loss domain and vice versa. The second group consists of subjects who chose more risky lotteries in the gain domain than in the loss domain (all north-east off-diagonal elements of matrix in Table 5, implying ${\sigma }_{\text{G}}<{\sigma }_{\text{L}})$ . Subjects in the third group chose less risky lotteries in the gain domain than in the loss domain (all south-west off-diagonal elements of matrix in Table 5, implying ${\sigma }_{\text{G}}>{\sigma }_{\text{L}}$ ).¹⁰

Table 5 Pairs of choices by individual (gain and loss domain)

Gain\loss	A	B	C	D	E	F
A	61	14	8	9	2	19
B	12	19	10	8	8	9
C	3	7	13	10	3	10
D	1	4	3	10	1	4
E	3	5	2	3	8	1
F	5	0	3	10	3	59

Table 6 shows the proportions of individuals placed in each of the three groups. Importantly, the table reveals an interesting pattern of heterogeneity. Only approximately half of the individuals made identical choices in the gain and in the loss domain, and there are significant proportions of individuals in each of the other groups. Specifically, 33% of the subjects are more risk averse when choices are presented as losses rather than as gains ( ${\sigma }_{\text{G}}<{\sigma }_{\text{L}}$ ). This is higher than 18% of the subjects who are less risk averse when choices are presented as losses rather than as gains ( ${\sigma }_{\text{G}}>{\sigma }_{\text{L}}$ ).¹¹

Table 6 Pairs of choices by individual (gain and loss domain)

	${\sigma }_{\text{G}}={\sigma }_{\text{L}}$	${\sigma }_{\text{G}}<{\sigma }_{\text{L}}$	${\sigma }_{\text{G}}>{\sigma }_{\text{L}}$	Total
Observations	170	116	64	350
Proportion (%)	49%	33%	18%	100%

3.3 Results from regressions of personal characteristics on risk aversion

I proceed by investigating whether any farmer-specific characteristics have a statistically significant impact on lottery choices in the gain domain (Table 7) or in the loss domain (Table 8). I consider three different empirical specifications depending on how the dependent variable is defined. Specifically, I consider models whereby the CRRA ranges are constructed using either arithmetic means or geometric means, and a model whereby the dependent variable takes integer values of 0, 1, 2, 3, 4, and 5 for lotteries A, B, C, D, E, and F, respectively.

In the three models with lottery choices in the gain domain (Table 7), none of the controls are statistically significant except for the gender variable. Specifically, in these regression models males are found to be less risk averse, and the result is statistically significant at 10%. The coefficients on all of the household characteristics are not statistically significant in the three models with lottery choices in the loss domain (Table 8). Finally, I also run a regression whereby the dependent variable is a dummy variable taking the value of 1 if a subject made the same choices in the gain and in the loss domain (i.e., a subject chose lotteries in accordance with expected utility theory). Table 9 shows that none of the controls are statistically significant in this model.

4 Discussion

The overall pattern observed in my data indicates that there may be heterogeneity across subjects in their decision-making. This heterogeneity relates to the general idea emphasized by Harrison and Rutstrom (2009), namely, one single model of decision-making may not capture well the behavior of all subjects. Expected utility theory may not be the accurate theory explaining the behavior of all subjects participating in a given experiment. While it might accurately explain the choice by a substantial proportion of subjects, another substantial proportion of subjects may not act according to expected utility theory.

As noted by Harrison and Rutstrom (2009), different types of decision-makers may be characterized more accurately using distinctive theoretical frameworks, and also within each type there may be additional form of heterogeneity, such as a different degree of risk aversion (for which I also find evidence in my dataset). Half of my experimental subjects (51%) perceive lotteries in the gain frame and in the loss frame differently, which could not be explained by expected utility theory.^{12 , 13} Understanding the preferences of decision-makers is important in policy evaluation (Duflo et al., 2007), such as in the context of promoting the adoption of more profitable but riskier agricultural technologies (Gollin et al., 2021). As a range of available policy interventions has come available in recent decades, gaining a better understanding of risk preferences is becoming essential for maximizing policy impacts and cost-effectiveness, and for identifying subjects who can benefit most from a given intervention (Muralidharan & Niehaus, 2017).

5 Conclusion

In this paper, I use an artefactual field experiment to study the risk preferences of Ghanaian farmers. I begin by replicating the study by Binswanger (1980) using a non-standard subject pool of farmers from a different country. I subsequently consider an alternative methodological approach whereby an identically ordered lottery selection procedure in the gain domain is presented to experimental subjects in the loss domain. I find that the average subject in my sample is moderately risk averse. However, using my alternative methodological approach, I also find that the decisions of half (51%) of the experimental subjects were not consistent with expected utility theory. These experimental findings combining the highly influential methodology developed by Binswanger with an alternative methodology are in line with the growing trend of research focusing on behavioral explanations of actual decisions made by economic agents.