Target versus budget reverse auctions: an online experiment using the strategy method

Abstract

Reverse auctions, also known as procurement auctions, are used in various fields by public or corporate buyers to purchase goods and services from multiple sellers at the best price. Unlike in selling auctions, in reverse auctions a budget constraint rather than a target quantity is often announced by the auctioneer. However, in auction theory no equilibrium bidding strategy has yet been found in the case when a budget constraint is announced. Here we compare the two auction formats in an online experiment with 329 participants. We use the strategy method to obtain participants’ bidding strategies from which we run exhaustive simulations of auction outcomes to define equivalent target and budget constraints. This original methodology allows to overcome the issue of randomness of the auction outcome related to bidders’ values and to compare the two formats in a rigorous way. When each bidder has a single unit to sell, from the buyer’s perspective, we find that, on average, the budget-constrained auction format outperforms the target-constrained auction format.

1. Introduction

In market design, auctions are often promoted as an alternative to fixed payments to reduce the asymmetry of information between an auctioneer and the bidders. Indeed, the level of payment in auctions is determined through competition among bidders, who must make a trade-off between increasing their margin and increasing their probability of winning the auction.

Contrary to standard selling auctions, in reverse or procurement auctions, the auctioneer is the buyer, and the bidders are the sellers. In a multi-unit reverse auction, the buyer can announce a target to the bidders, that is, the quantity or number of units to be purchased. In such target-constrained (hereafter Target) reverse auctions, the buyer accepts the lowest bids until the target is reached. Alternatively, in budget-constrained (hereafter Budget) reverse auctions, the buyer announces a budget constraint, that is, his objective is to buy the maximum quantity or number of units without exceeding the announced budget. Target reverse auctions are implemented to address a range of environmental issues by buying back items such as water use licenses Cummings et al. (2004), Janmaat (2011), fisheries quotas DePiper et al. (2013) and fishing vessels in order to decrease fishing capacities Squires (2010). They are also used to allocate payments for environmental services, as in the US Conservation Reserve Program Hellerstein 2017. However, in such auctions for environmental services, also known as conservation auctions, it is typically a budget constraint that is used, as shown in the review by Bingham et al. (2021), Table 2. The former Australian ecoTender program Stoneham et al. 2012 is another example of the application of Budget conservation auctions, along with numerous other initiatives Rolfe et al. 2017. It should be noted that in real-world situations, target or budget constraints are often supplemented with bid caps or reserve prices Stoneham et al. 2012, Hellerstein 2017, above which no bid is accepted.

Table 2 Main results from the experiment

Notes: Standard deviations in parenthesis.

* Three subjects in Target and two in Budget were removed randomly to get multiples of four in both treatments. Results are robust across various sets of group combinations.

In this study, we compare the performance of the two reverse auction formats, Target and Budget, to determine which constraint would enable the auctioneer, on average, to purchase the maximum quantity for the minimum cost. The preference for a Target or Budget auction depends on the buyer’s demand function. If marginal utility becomes zero above a given target, and/or if it is essential to reach a threshold to obtain an environmental benefit, then the target auction is the obvious choice. On the other hand, if the buyer has a fixed budget envelope (it is impossible to spend more, and the envelope is lost for everyone if it is not spent), then a Budget auction is called for. If not, the question arises. The comparison Target versus Budget is meaningful when the buyer has no strong constraint (such as reaching a specific threshold or complying with a fixed budget). Surprisingly, the question of how the announced constraint affects the efficiency of a reverse auction outcome remains understudied.

Indeed, auction theory has widely been built on the study of selling auctions. Consequently, in the literature, it is always the quantity or number of units that is announced in share auctions or multi-unit auctions Milgrom 2004, Klemperer (2004), Krishna 2009. Budget-constrained auctions have received very little attention in the auction literature, yet announcing a budget is relatively common in reverse auctions. In this paper, we attempt to shed some light on this question. We consider the simple case in which each bidder has only one unit to sell and thus competes on price only. All units are identical for the buyer, but the sellers each have independent and private costs to produce their unit. We consider a discriminatory multi-unit reverse auction, that is, a sealed bid auction where winning bidders are paid their own bid. In a multi-unit target (selling) auction, an equilibrium bidding strategy exists that is based on maximizing the bidders’ expected surplus Harris and Raviv (1981), Cox et al. (1984). Hailu et al. 2005 and Liu (2021) extend this result to the reverse auction case in which a target constraint is announced. However, to our knowledge, no equilibrium bidding strategy has yet been identified for an auction in which a budget has been announced. Therefore, we cannot determine theoretically which format is the most efficient in a reverse auction.

To make a relevant comparison, we need to define equivalent constraints. Given a fixed number of bidders, the lower the number of units requested by the buyer (in Target) or the lower the budget announced by the buyer (in Budget), the greater the degree of competition. However, we can hardly predict a priori the level of each constraint that equalizes the level of competition between Target and Budget. To overcome this issue, we propose an original method using simulations based on experimental bidding strategies. Concretely, to ensure some kind of equivalence, we set the target constraint exogenously and then define the budget constraint endogenously according to the results obtained in the Target treatment Schilizzi and Latacz-Lohmann (2007). One of our contributions is the way we calculate the average outcome of the auctions to define equivalent constraints. We use the strategy method to obtain the bidding strategy for each subject. The strategy method is a well-established method in experimental economics Selten 1967, Mitzkewitz and Nagel (1993), Brandts and Charness (2011) which has been used in few auction experiments Rapoport and Fuller (1995), Selten and Buchta (1999), Güth et al. (2002), Güth et al. (2003), Kirchkamp et al. (2009), Katuščák et al. 2015, Mill and Morgan 2022, but to our knowledge never in the case of reverse auctions. In experimental auctions, the strategy method consists in asking subjects for their entire bidding strategy (for all cost levels) in a single round. Then, we simulate from the subjects’ bidding strategies what we call the “average budget”, which is used to set the constraint in the Budget treatment. We also simulate the average number of units purchased in the Budget treatment. Finally, if the average number of units purchased in Budget is significantly higher than the number of units announced in Target, Budget has higher budgetary efficiency than Target and vice versa.

To the best of our knowledge, only Schilizzi and Latacz-Lohmann (2007) and Boxall et al. 2017 have compared Target and Budget. They use laboratory experiments in the context of conservation auctions in which student subjects play the role of farmers (bidders) whose opportunity costs are set randomly. Subjects take part in repeated auctions with three and fifteen periods, respectively. Random costs are reshuffled each period in Schilizzi and Latacz-Lohmann (2007) and every five periods in Boxall et al. 2017. Both experiments are multi-unit auctions, but bidders have one unit to sell in Schilizzi and Latacz-Lohmann (2007) and possibly several in Boxall et al. 2017. Both experiments find that the Target format outperforms the Budget format in the first round but that after several repetitions the auction performance evens out.

Our experiment differs from these two previous experiments in three main ways. First, we propose a totally decontextualized experiment for better control and to allow the results to be extended to any reverse auction. Second, we use the strategy method to obtain subjects’ complete bidding strategies from which we simulate exhaustively the possible auction outcomes. Third, we use the above-mentioned exhaustive simulations to compute equivalent target and budget constraints so as to compare both formats in a more rigorous way.

In Section 2, we present some theoretical background underlying our experiment, as well as the equilibrium bidding strategy in the Target format. Then, in Section 3, we introduce our experimental design, based on the strategy method, and its online implementation. Section 3 also details the way we simulate auction outcomes to set equivalent constraints and measure efficiency in Target and Budget. A description of subjects and control variables is conducted in Section 4, while the results are presented in Section 5. Finally, Section 6 gives some discussions and Section 7 concludes.

2. Theory of reverse auctions

In this section, we present a theoretical framework which is derived from auction theory (2.1), the two types of announced constraints (2.2) and the symmetric equilibrium bidding strategy in the Target case (2.3).

2.1. Auction game

We consider a multi-unit procurement or reverse auction with an auctioneer who is the buyer and with N risk-neutral symmetric bidders who are each sellers of a single unit. Thus, each bidder i proposes a single bid b_i which is the selling price for his unit. All the units are homogeneous and perfectly divisible for the auctioneer. Each bidder i produces his unit at a private cost c_i. It is common knowledge that costs are identically and independently drawn from the same distribution with a density function $f(.)$ and a cumulative function $F(.)$ on the interval $[\underline{c},\overline{c}]$. Let $b_i(c)$ be the bidder i’s bidding function (or bidding strategy), which is assumed to be increasing and differentiable. When a N-uple of costs, $(c_1,..., c_N)$, are assigned to bidders, bids b_i are ranked by the auctioneer in ascending order of price with rank (r), $r=1, ... ,N$.

\begin{equation*}b_{(1)} \leq b_{(2)}\leq ... \leq b_{(N)} \end{equation*}

The lowest bids are selected until the announced constraint is reached (see Section 2.2). The auctioneer can split the last selected unit to meet his constraint or in case of ties.

We consider a discriminatory (or first price) reverse auction, thus the price paid to each winning bidder is defined by the bidder’s own submitted bid.

2.2. Announced constraint

We distinguish two auction formats: the target-constrained auction (Target) and the budget-constrained auction (Budget). These formats do not impact the payment rule but define differently the limit of the selection rule in a reverse multi-unit auction.

In Target, before the bidders submit their bid, the auctioneer announces the quantity he will buy. Let M^T (with $0 \lt M^T \lt N$) be the targeted number of units. Then, to minimize his expenses, the auctioneer selects the lowest bids until the desired quantity is reached. Thus, he buys the M^T least expensive units. Formally, M^T may be any real positive number, not only an integer.

In Budget, the auctioneer announces B, the maximum amount of money he will spend to buy the highest possible quantity. Thus, units are purchased in ascending order of price until the available budget is reached or all the N available units for sale are purchased. In the budget format, it is likely that the announced budget will not fit exactly with the purchase of an integer number of units. This is not a problem, since we consider units to be perfectly divisible. In addition, note that the budget constraint may not be reached, and a balance may remain when the sum of all the bids is lower than the budget announced, that is, $B \gt \sum_{i=1}^{N} b_i$.

Announcing one or the other constraint leads to two different auction formats for the bidders which are also based on two different objectives for the auctioneer. In Target his aim is to minimize his expenditure while buying exactly the right number of units (the target). In Budget, his goal is to buy as many units as possible without exceeding his budget.

The choice of the constraint type may be driven by a real constraint or a preferred objective. Of course, in the end, the trade-off between price and quantity can only be solved by setting the buyer’s demand function. Nevertheless, to keep the comparison exercise as general as possible, we do not impose a given demand function. Rather, we assume that the buyer does not have any strict constraints: his available budget is unlimited and his marginal utility for each unit until the N ^th is strictly positive. Thus, we assume that the buyer’s objective is to purchase the maximum quantity for the minimum budget.

2.3. Equilibrium bidding function

In Target, when M^T is a positive integer, bidders seek to maximize their expected gain, expressed as:

(1)\begin{equation} E(b_i,c_i) = (b_i - c_i).Prob(b_i \lt b_{(M^T+1)}), \end{equation}

with $b_{(M^T+1)}$ the first rejected bid Müller and Weikard (2002).

Considering the symmetric equilibrium of the auction game introduced in Section 2.1, Hailu et al. 2005 and Liu (2021) demonstrate that the unique equilibrium bidding function $b^*(c)$ in Target is:

(2)\begin{equation} b^*(c) = \frac{\int_{c}^{\overline{c}} u F(u)^{M^T-1}(1-F(u))^{N-M^T-1} f(u) du}{\int_{c}^{\overline{c}} F(u)^{M^T-1}(1 - F(u))^{N-M^T-1} f(u) du}. \end{equation}

In Budget, the quantity purchased is unknown to bidders, because it depends on other bids. Therefore, there is no simple equilibrium bidding function Müller and Weikard (2002), as strategic interactions can hardly be modelled.¹

Without any theoretical result, we compare the two auction formats in a decontextualized online lab experiment where subjects play the auction game described in Section 2.1.

3. Online experiment and auction outcome simulations

In the current section we describe the strategy method and the reasons we have adopted it in our online lab experiment (3.1). Next, we present our experimental design (3.2) and explain how we compute experimental outcomes based on participants’ complete bidding strategies (3.3). Finally, we show how the online experiment has been implemented (3.4).

3.1. Strategy method

In induced value auction experiments, costs are usually assumed to be uniformly distributed across a given interval $[\underline{c},\overline{c}]$. In practice, only a discrete sample of costs is necessarily considered. Let J be the number of possible values within this interval. In most experiments, one cost per bidder is drawn in order to perform the auction, and several periods are conducted with different sets of costs to generate more data. In these repeated auctions, results may depend on cost draws, even if the same set of costs is kept across treatments. A learning process can occur over periods, potentially introducing bias into the results Güth et al. (2003), Lusk and Shogren 2007. Finally, there may be a wealth effect when several auctions are played and paid for successively. To overcome these issues, we use the strategy method (or cold strategy) to get the entire bidding strategy of every subject in a single round.

In practice, subjects have to fill in a decision table containing the J possible cost values with J corresponding bids. By asking subjects to bid for all possible costs at once, rather than conducting successive auctions one after the other with different costs, we give them the opportunity to better refine their strategy according to their cost.

The strategy method facilitates the implementation of an online experiment, as subjects do not need to be connected at the same time. In our experiment, subjects had as much time as they wanted to respond to the 21 possible choices. No time limit or pressure was imposed on them, contrary to what is generally practiced in the laboratory to reduce the waiting time of the fastest subjects, especially when multiple periods are announced.

In each treatment, groups of N bidders are randomly formed ex post, and the N-uple of cost k used to define subjects’ earnings among the $K=J^N$ possible cost arrangements is also randomly drawn ex post.

3.2. Experimental design

We choose a between-subjects design where subjects are randomly assigned to a single treatment to prevent any order effect.²

Comparing the two auction treatments requires setting equivalent constraints. As stated in the introduction and illustrated in Figure 1, we first run the Target treatment with an exogenous constraint set to M^T units. In the following section, we will detail how we compute the average budget (B) from the bidding functions obtained in Target to buy M^T units. This average budget B is then used as the announced constraint in the Budget treatment. Although the budget constraint is endogenously defined in our experiment, it is an exogenous constraint for the participants in the Budget treatment. Next, we compute the average number of units purchased M^B from the bidding functions obtained in Budget. Finally, if M^B is higher than M^T then Budget outperforms Target and vice versa.

Fig. 1 Overview of the experimental design

In the following section (3.3), we explain how we compute exact group-level values of outcomes (budget spent or quantity purchased) that sum up all possible cost arrangements k. We aim to obtain one representative group-level value of outcomes over the K possible cost arrangements to eliminate the uncertainty associated with random cost draws. Each treatment is conducted on several groups of N bidders, so in order to get independent data, groups need to be independent. Therefore, to compute the average auction outcome at the treatment level, each subject is randomly assigned to a single group of N bidders. There are G^T and G^B independent groups, respectively, in Target and Budget.

3.3. Simulation of auction outcomes

The advantage of having the bidding strategies of all subjects is to be able to simulate the auction outcome for any group g of N subjects and for any cost arrangement k. These simulations generate a very rich data set which allows us to eliminate the randomness related to the drawing of costs. Indeed, simulations can be run on all the $K=J^N$ possible cost arrangements.

We define an auction as a group g of N bidders associated to a given N-uple of costs k (corresponding to the k ^th N-uple of costs). As described in Section 2.1, in each auction the buyer ranks the bids b_i in ascending order of price: $b_{(1)} \leq b_{(2)}\leq ... \leq b_{(N)} $.³

Calculation of the average budget B spent in Target to set equivalent constraints:

Here we detail how we compute from subjects’ bidding strategies the average budget spent in the Target treatment, which is used as the endogenous constraint in the Budget treatment.

In Target, only the M^T cheapest units are selected. Therefore, the budget spent in group g with cost arrangement k is $B_{gk} = \sum_{r=1}^{M^T} b_{(r)}$. The exact mean of the budget spent B_g is computed within each group g ( $g=1,...,G^T$) on all the possible cost arrangements k ( $k=1,..., K$) such as:

(3)\begin{equation} B_{g} = \frac{\sum_{k=1}^{K} B_{gk}}{K} \end{equation}

Finally, the average budget B is the mean of the G^T group-level average budgets B_g

(4)\begin{equation} B = \frac{\sum_{g=1}^{G^T} B_{g}}{G^T}. \end{equation}

Calculation of the average number of units auctioned in Budget to compare auction performance:

Cost-effectiveness is measured by comparing the average number of units auctioned in Budget (M^B) with the number of units announced in Target (M^T) for an equivalent average budget B (see Figure 1). Similarly to the computation of B in Target, the number of units auctioned $M^B_{gk}$ is first defined at the auction level.

Define t as a positive integer such as $0 \lt t \lt N $, then

(5)\begin{equation} M^B_{gk}= \begin{cases} t + \frac{B-\sum_{r=1}^{t} b_{(r)}}{b_{(t+1)}} & \text{if}\ \sum_{r=1}^{t} b_{(r)} \leq B \lt \sum_{r=1}^{t+ 1} b_{(r)} \\ N & \text{if}\ B \geq \sum_{r=1}^{N} b_{(r)} \\ \end{cases} \end{equation}

Note: there may be a non-null residual budget if the budget B is great enough to purchase all the units. We assume that any residual budget is lost for both the auctioneer and the bidders. The exact group-level means $M^B_{g}$ — used as observations — are computed over all cost arrangements k such as

(6)\begin{equation} M^B_{g} = \frac{\sum_{k=1}^{K} M^B_{gk}}{K} \end{equation}

Finally, the average number of units purchased in Budget M^B is the mean of the G^B group-level values.

(7)\begin{equation} M^B = \frac{\sum_{g=1}^{G^B} M^B_{g}}{G^B} \end{equation}

Calculation of the average allocative efficiency in both formats:

In addition to the number of units purchased from equivalent budget and target constraints, we check the allocative efficiency of each auction. In each auction, bids are ranked by increasing price, not by production cost. Therefore, the winning bids may not always correspond to the lowest production costs. Allocative efficiency is at the maximum level when all purchased units are from the bidders with the lowest costs. Let $c_{(r)}$ be the cost corresponding to the r ^th unit when arranged in ascending order of bids and $c_{[i]}$ be the cost corresponding to the i ^thunit in ascending order of cost.

\begin{equation*}c_{[1]} \leq c_{[2]} \leq ... \leq c_{[N]} \end{equation*}

To measure the allocative efficiency of an auction, whether it is in Target or Budget, we use the ratio of the sum of the lowest costs ( $C^{min}_{gk}$) to the sum of the actual costs corresponding to winning bids ( $C^{act}_{gk}$), such as

(8)\begin{equation} AE_{gk} = \frac{C^{min}_{gk}}{C^{act}_{gk}} \end{equation}

with $C^{min}_{gk} = \sum_{i=1}^{M_{gk}} c_{[i]}$ and $C^{act}_{gk} = \sum_{r=1}^{M_{gk}} c_{(r)}$.⁴ Note that if the sum of the cost of the winning bids is zero, then the numerator is necessarily zero. So, in that case, we consider that the allocative efficiency is equal to 1. As for the two prior outcomes, we compute the exact group means AE_g, which we use as observations (one observation per group), and finally AE, which is the treatment-level average of the G^T or G^B observations.

3.4. Online implementation of the experiment

The experiment was programmed with o-Tree software Chen et al. 2016 and implemented online with an instructional video.

In the instructional video, subjects are told they are participating in an experiment in which they are anonymous sellers and that they can earn money depending on their decisions and those of other participants. Indeed, they are randomly assigned to groups of four participants without being able to identify the three other members of their group. The relatively small number of bidders (N = 4) was chosen to increase the number of independent observations, that is, the number of groups. Subjects are given the possibility to sell a unit of a good to a single buyer (the experimenter). To participate in the experimental game, subjects must complete a decision table (Fig. 2) containing 21 possible production costs for their unit. The distribution of private costs c_i is uniform between € 0 and € 100 and includes J = 21 possible cost values corresponding to multiples of € 5. We define the exogenous Target $M^T=N/2=2$.⁵ For each cost, subjects must choose a selling price above or equal to the corresponding cost and rounded up to the nearest euro.

Fig. 2 Decision table

We explain to the subjects that at the end of the experiment, in order to determine earnings, a production cost will be drawn randomly for each participant. Then, for each subject, the bid associated to his/her randomly drawn cost will be collected from his/her decision table. Finally, in each group of four participants, the cheapest units will be bought until the announced constraint is exhausted (according to the auction treatment).

Subjects’ gains are defined as follows. If they do not succeed in selling their unit, they gain nothing. If they do succeed in selling their unit, they receive a payment equal to the difference between their selling price and their production cost. Full instructions for the Target treatment⁶ are available in the Online Appendix A.1.

In such an online environment, subjects cannot ask questions; therefore, they must have a perfect understanding of the instructions. To this end, after the video they are required to answer a comprehension questionnaire consisting of True/False questions (see Online Appendix A.2). After responding to each question, the correct answer appears on the participant’s screen. At any time during the experiment, subjects can access a text version of the instructions.

After completing the decision table, subjects answer a short questionnaire (see Online Appendix A.3). First, we elicit risk aversion with a self-assessment question, as in Dohmen et al. (2011). As this behavioral characteristic may have an impact on the way subjects bid, it is necessary to ensure that our two treatment groups are balanced with regard to this variable. Second, we assess the difficulty respondents had in proposing selling prices. We speculate that it is more difficult for subjects to bid in Budget than in Target. Finally, we ask a few socio-demographic questions.

After each treatment, we randomly made groups of four participants, drew a production cost for each bidder and determined the result of the auctions (and thus each participant’s payoff) according to their bid. All the subjects received their participation fee just after they validated their participation. The auction payoff was paid to the winning bidders a few days after their participation in the experiment.

4. Data

Our experiment was conducted online in June 2021 and involved 329 subjects from the general French population who were registered on the FouleFactory platform.⁷ Participants received a standard fee for a 15-minute survey (€ 2) and a potential extra gain from auction earnings (€ 2.56 per subject, on average).⁸ This extra gain may have boosted participants’ engagement, as they usually get paid only the fixed amount to respond to surveys. Participants knew that they would be assigned ex post to a randomly constituted group of four bidders.

The average age was 41 and 50.8% of the subjects were women. (Std. Dev. = 13). Some socio-demographic categorical variables are shown in Table 1, in which we see that 48.9% of our subjects had at least a bachelor’s degree and (at least) 43.2% earned € 1900 or more per month. On the three comprehension questions, 47.4% of the respondents made no mistakes, 42.9% made only one mistake, 9.7% made two mistakes and none made three.

Table 1 Sample description (n = 329)

We had 131 participants in Target and 198 in Budget, which allowed us to constitute $G^T = 32$ and $G^B = 49$ groups of four bidders. Three subjects in Target and two subjects in Budget were removed randomly so that the number of subjects was a multiple of four. Although subjects were randomly assigned to the two treatments, we observed (see Online Appendix B.1) that samples were not balanced on the Income and Profession variables. However, robustness checks presented in the Online Appendix B.2 show that this does not impact the validity of our results.

5. Results

Our main results on the performance comparison between Target and Budget treatments is presented in 5.1. We then analyze the variability of outcomes in each auction format across cost draws (5.2), which is possible because our simulations are exhaustive on the cost arrangements within the groups. Finally, we explain our results by emphasizing both the role of subjects’ bidding behavior and the role of the auction formats themselves (5.3).

5.1. Main results

The Target treatment results in an average budget B of € 72.32 (see Table 2),⁹ which is the amount the auctioneer needs, on average, to purchase $M^T=2$ units, given the bidding strategies of 128 subjects assigned to $G^T =$ 32 groups. This amount was rounded down to € 72 to be used as the budget constraint in the Budget treatment.¹⁰

As reported in Table 2, the Budget treatment results in an average purchase of $M^B=2.135$ units.¹¹ Note that the Budget treatment also benefits from a positive average balance of E = € 0.09. We consider this excess budget to be lost. Overall, the Budget format allows subjects to purchase significantly more units (7% more) on average than the Target format with the same average budget (Wilcoxon signed-rank test, p-value $\mathrel{ \lt } 0.001$).¹² Here, rounding the budget constraint to the nearest lower integer and having a non-null residual budget are conservative assumptions that support our conclusion that the Budget format outperforms the Target format.

Regarding the allocative efficiency, a slight difference is found between the two treatments (+0.006 in Budget). This difference is significant at the 10% level (Wilcoxon rank-sum test, p-value=0.053). In addition, the results presented in Table 2 show that allocative efficiency is relatively high (AE is close to one) in both auction formats.¹³

Finally, contrary to our speculation, it was not more difficult for our respondents to bid in Budget than in Target. According to the self-assessment variable Ease to bid, with values comprised between zero (absolutely no difficulty to bid) and 10 (very difficult to bid), the average values per treatment (6.81 for Target and 6.54 for Budget) are not significantly different (Wilcoxon rank-sum test, p-value = 0.555).

5.2. Variability of auction outcomes across cost draws

Figures 3 and 4 illustrate the variability of outcomes within each group by providing (independent) exact group means (B_g and $M^B_g$, respectively), with intervals corresponding to their standard deviations. The average group-level standard deviation is € 38.95 in Target (which represents 53.86% of the average budget computed at the treatment level) and 0.632 units in Budget (29.59% of the average number of units purchased). Therefore, from the auctioneer’s point of view, we observe, on average, a lower uncertainty due to the randomness of costs in Budget than in Target. This result could be another advantage of announcing to bidders a budget rather than a target.

Fig. 3 Exact means and standard deviations of the budget spent per group in Target (B_g)

Fig. 4 Exact means and standard deviations of the number of units purchased per group in Budget ( $M^B_g$)

We see from Figure 3 that the group exact means of budget spent in Target vary according to group by a relatively small amount: around € 72 (the average budget denoted by the horizontal dashed line). Indeed, the standard deviation of group exact means is only € 6.56 (see Table 2). We also observe a low variability in the exact mean of the groups in Budget (Fig. 4). Here, the standard deviation is only 0.099 units. Furthermore, the exact mean of units purchased by group is less than two units (the exogenous target constraint denoted by the horizontal dashed line) in only four groups out of 49, which is consistent with the results presented in Table 2. In this paper, however, independent observations are computed on the basis of exhaustive simulations regarding costs. This ensures that the comparison of the performance of the two treatments is not related to random cost draws.

5.3. Why does Budget outperform Target?

Budget can outperform Target because the auction constraint format is different, but also because bidders bid differently in each treatment. Indeed, an explanation for the higher budgetary efficiency in Budget is that the offers made by the subjects tend to be lower in Budget than in Target. Figure 5 displays the average bid by cost level.¹⁴

Fig. 5 Average bids in Target and Budget

At this stage, average bidding strategies appear similar in the two treatments (with slightly lower price offers in Budget, which could explain its superiority). Note that experimental bids are very different from the equilibrium bidding strategy in Target, which is consistent with results found by Liu (2021). To break down the total treatment effect, we consider the average outcome of a fictive treatment, which consists in simulating a Budget-constrained auction using subjects’ bids from the Target treatment. As illustrated in Figure 6, the comparison of this fictive treatment with the Target treatment allows us to isolate the effect of the constraint format (format effect = 2.086 – 2 = 0.086 units) since we keep the same bidding strategies (Target bids). We find this effect to be significant at the 1% level (Wilcoxon signed-rank test, p-value $\mathrel{ \lt } 0.001$). Our main hypothesis to explain this format effect in favor of the Budget auction is its flexibility in the number of units purchased. Contrary to the Target auction, the Budget auction allows buying a higher (lower) number of units when bids are relatively low (high) and therefore, on average, more units are purchased in Budget than in Target. By comparing the fictive treatment with the budget treatment, we isolate the effect of subjects’ bidding strategies (bids effect = 2.135 – 2.086 = 0.049 units) since we keep the same auction constraint format (Budget constraint of € 72). This effect is significant at the 5% level (Wilcoxon rank-sum test, p-value = 0.034). All these comparisons of units purchased are made with a constant average budget (€ 72). The total treatment effect is the sum of the two effects (0.086 + 0.049 = 0.135 units). In Online Appendix C, we demonstrate through examples with simulated data that the format effect is not specific to the bidding strategies from this experiment.

Fig. 6 Breakdown of the treatment effect

6. Discussion

We found that the Budget format provides both higher budgetary and allocative efficiency than the Target format. This contrasts with results found by Schilizzi and Latacz-Lohmann (2007) and Boxall et al. 2017 in the context of conservation auctions. These experimental studies suggest that Target presents greater cost-effectiveness than Budget in the first auction period, but that the performance of Target erodes faster with repetitions. Both papers attribute this relative decrease in Target performance to faster learning by participants of the cut-off price in Target, which we cannot test with our current experimental protocol. Compared to prior research conducted by Schilizzi and Latacz-Lohmann (2007) and Boxall et al. 2017, our experiment has a smaller auction size. However, we assume that the units are perfectly divisible by the buyer, and we consider results without any rounding at the auction and at the group levels, and up to three decimal places at the treatment level. This allows detecting small differences in our main variable of interest, even with N = 4 and $M^T=2$. We acknowledge that the number of bidders and the level of competition are likely to affect auctions’ performance, but these issues are not within the scope of our study.

Our online experimental results rely on the use of an innovative methodological framework based on the strategy method which is a “cold” procedure where subjects must bid for several possible costs at the same time Brandts and Charness (2011). This is a key difference from Schilizzi and Latacz-Lohmann (2007) and Boxall et al. 2017, who used “hot” or direct-response procedures where one individual cost is drawn per period and subjects must bid directly according to the cost drawn. The strategy method has already been used in some previous studies, but only in the context of selling auctions Rapoport and Fuller (1995), Selten and Buchta (1999), Güth et al. (2002), Güth et al. (2003), Kirchkamp et al. (2009), Katuščák et al. 2015, Mill and Morgan 2022. An extensive literature compares direct-response and strategy methods in various behavioral games Fischbacher et al. (2012), Columbus and Böhm (2021). In some cases, a difference was found between the two methods Casari and Cason (2009), which may be explained by a hypothetical bias related to the strategy method. However, most of the time no or mixed evidence has been found Brandts and Charness (2011), Fischbacher et al. (2012), Columbus and Böhm (2021), in particular in auction experiments Rapoport and Fuller (1995), Armantier and Treich (2009). Moreover, these differences may be explained by learning or wealth effects, which are not present with the strategy method. Note that we may have naive subjects who may have no experience of auctions. A potential drawback of our experimental protocol, in contrast to more standard auction experiments in which subjects play multiple periods of auctions, is that there is no learning effect from repetitions. This is because the decision table has to be filled in all at once, without any feedback. This does not allow subjects to learn how to bid through practice. However, to ensure that our results are robust to subjects’ understanding, we provide robustness checks in Online Appendix D. The main result remains unchanged on subsamples of subjects who devoted the most time to complete the decision table (D.1), correctly answered the comprehension question about competition (D.2) and found it easier to bid than others (D.3).

We have combined the strategy method with ex post numerical simulations to generate a large number of auctions. With the parameter values in our experiment, it is even possible to simulate exhaustively all possible auction outcomes. A benefit of the strategy method is that our experiment could be conducted online with a large number of subjects without requiring them to be connected at the same time. In addition, we used as independent observations the exact mean of each group computed over all possible induced cost arrangements. This ensures that the comparison of the two treatments is not biased by the randomness of cost draws. Indeed, outcomes may vary considerably according to the cost arrangement considered in each group (see Section 5.2).

7. Conclusion

To summarize, the aim of this paper was to compare the relative performance of target-constrained and budget-constrained reverse auctions. To do so, equivalent constraints were set up by determining the budget constraint endogenously from the average budget spent in the Target treatment. We used the strategy method to obtain subjects’ complete bidding strategies, which allowed us to make ex post simulations and avoid any potential bias generated by randomly induced costs. We found that the Budget format provides, on average, a greater amount of units for an equivalent budget than the Target format. But, although the difference is significant, it is relatively small in this experiment. Furthermore, we find that the allocative efficiency is slightly higher in Budget than in Target. In other words, the Budget format outperforms the Target format on both criteria. Yet, when considering a practical situation, it is important to note that the previous result is only valid when the buyer does not have a strong target constraint, for example., to reach a specific threshold in order to benefit from environmental outcomes.

This paper fills a gap in the literature, as this is the first decontextualized study to deal with relative performance of Target and Budget reverse auctions. Currently, there is a lack of theoretical research on procurement auctions where the buyer announces a budget instead of a target. Our experiment provides insights on how bidders bid in such auctions, but further theoretical work is needed. Specifically, it is worth considering what budget should be announced to achieve, on average, a given target. The current paper produces and analyzes auction experimental data, thanks to the combination of the strategy method and a simulation exercise. This innovative methodology may be useful for future experiments on auctions. We acknowledge that in our study many aspects have not been explored, such as the impact of the auction group size N on auction performance, or alternatively, whether one format would better foster sellers’ participation than the other (here N is fixed and exogenous). Indeed, participation and risk of collusion are important issues in auctions. The way the reverse auction is framed and designed may have an important impact on both of these issues, which would nevertheless depend on the context in practice. The uniform distribution of costs we used for practical considerations could also lead to an overestimation of the variability of outcomes compared with a normal distribution. Finally, the performance of both formats with repetitions remains beyond the scope of this study. These issues are to be explored in further work, as is the combination of the two constraints in a single auction treatment.