Hostname: page-component-745bb68f8f-b6zl4 Total loading time: 0 Render date: 2025-01-08T04:56:54.306Z Has data issue: false hasContentIssue false
  • English
  • Français

Lab-like findings from online experiments

Published online by Cambridge University Press:  01 January 2025

Irene Maria Buso ,
Daniela Di Cagno ,
Lorenzo Ferrari ,
Vittorio Larocca ,
Luisa Lorè ,
Francesca Marazzi ,
Luca Panaccione  and
Lorenzo Spadoni Open the ORCID record for Lorenzo Spadoni [Opens in a new window]
Irene Maria Buso*
Affiliation:
Department of Economics, Ca’ Foscari University of Venice, Venice, Italy
Daniela Di Cagno*
Affiliation:
Department of Economics and Finance, Luiss University, Rome, Italy
Lorenzo Ferrari*
Affiliation:
Department of Economics and Finance, Luiss University, Rome, Italy
Vittorio Larocca*
Affiliation:
Center for Experimental Studies of Internet, Entertainment and Gambling (CESIEG), Luiss University, Rome, Italy
Luisa Lorè*
Affiliation:
Department of Economics, University of Innsbruck, Innsbruck, Austria
Francesca Marazzi*
Affiliation:
Centre for Economic and International Studies (CEIS), University of Rome Tor Vergata, Rome, Italy
Luca Panaccione*
Affiliation:
Department of Economics and Law, Sapienza University of Rome, Rome, Italy
Lorenzo Spadoni*
Affiliation:
Department of Economics and Finance, Luiss University, Rome, Italy
*
irenemaria.buso@unive.it
ddicagno@luiss.it
lferrari@luiss.it
vlarocca@luiss.it
luisa.lore@uibk.ac.at
francesca.marazzi@uniroma2.eu
luca.panaccione@uniroma1.it
e-mail: lspadoni@luiss.it
Get access Rights & Permissions [Opens in a new window]

Abstract

Laboratory experiments have been often replaced by online experiments in the last decade. This trend has been reinforced when academic and research work based on physical interaction had to be suspended due to restrictions imposed to limit the spread of Covid-19. Therefore, data quality and results from web experiments have become an issue which is currently investigated. Are there significant differences between lab experiments and online findings? We contribute to this debate via an experiment aimed at comparing results from a novel online protocol with traditional laboratory settings, using the same pool of participants. We find that participants in our experiment behave in a similar way across settings and that there are at best weakly significant and quantitatively small differences in behavior observed using our online protocol and physical laboratory setting.

Keywords

MethodologyExperimentsLab-like dataCovid-19

JEL classification

C90: General C81: Methodology for Collecting, Estimating, and Organizing Microeconomic Data • Data Access
Type
Original Paper
Information
Journal of the Economic Science Association , Volume 7 , Issue 2: Special Issue: Experiments in the time of COVID 19 – Challenges and Insights (Pages 103-209) , December 2021 , pp. 184 - 193
Copyright
Copyright © 2021 The Author(s), under exclusive licence to Economic Science Association

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

We thank the Editors, Maria Bigoni and Dirk Engelmann, and two anonymous referees for useful comments. We also thank Sofia De Caprariis for her assistance during this project.

References

Amir, O., Rand, D., Kobi, Y. (2012). Economic games on the internet: The effect of $1 stakes. PLoS One, 10.1371/journal.pone.0031461CrossRefGoogle Scholar
Arechar, A., Gätcher, S., Molleman, L. (2018). Conducting interactive experiments online. Experimental Economics, 21(1), 99–131. 10.1007/s10683-017-9527-2CrossRefGoogle ScholarPubMed
Buso, I. M., De Caprariis, S., Di Cagno, D., Ferrari, L., Larocca, V., Marazzi, F., Panaccione, L., & Spadoni, L. (2020). The effects of covid-19 lockdown on fairness and cooperation: Evidence from a lablike experiment. Economics Letters, 196(C), S0165176520303487. https://EconPapers.repec.org/RePEc:eee:ecolet:v:196:y:2020:i:c:s0165176520303487.CrossRefGoogle ScholarPubMed
Chandler, J., Mueller, P., Paolacci, G. (2014). Nonnaiveté among Amazon Mechanical Turk workers: Consequences and solutions for behavioral researchers. Behavior Research Methods, 46, 112–130. 10.3758/s13428-013-0365-7CrossRefGoogle ScholarPubMed
Chen, D., Schonger, M., & Wickens, C. (2016). oTree-an open-source platform for laboratory, online, and field experiments. Journal of Behavioral and Experimental Finance, 9(C), 8–97. https://EconPapers.repec.org/RePEc:eee:beexfi:v:9:y:2016:i:c:p:88-97.CrossRefGoogle Scholar
Crump, M., McDonnell, J., Gureckis, T. (2013). Evaluating Amazon’s Mechanical Turk as a tool for experimental behavioral research. PLoS One, 8(3),e57410. 10.1371/journal.pone.0057410CrossRefGoogle Scholar
Dohmen, T., Falk, A., Huffman, D., Sunde, U., Schupp, J., Wagner, G. G. (2011). Individual risk attitudes: Measurement, determinants, and behavioral consequences. Journal of the European Economic Association, 9(3), 522–550. 10.1111/j.1542-4774.2011.01015.xCrossRefGoogle Scholar
Duch, M. L., Grossmann, M. R. P., Lauer, T. (2020). z-Tree unleashed: A novel client-integrating architecture for conducting z-Tree experiments over the internet. Journal of Behavioral and Experimental Finance, 28(3), 10.1016/j.jbef.2020.100400CrossRefGoogle Scholar
Giamattei, M., Yahosseini, K. S., Gächter, S., & Molleman, L. (2020). LIONESS Lab: A free web-based platform for conducting interactive experiments online. Journal of the Economic Science Association, 6(1), 95–111. https://doi.org/10.1007/s40881-020-00087-. https://ideas.repec.org/a/spr/jesaex/v6y2020i1d10.1007_s40881-020-00087-0.html.CrossRefGoogle Scholar
Greiner, B. (2015). Subject pool recruitment procedures: Organizing experiments with ORSEE. Journal of the Economic Science Association, 1(1), 114–125. 10.1007/s40881-015-0004-4CrossRefGoogle Scholar
Gupta, N., Rigotti, L., & Wilson, A. (2021). The experimenters’ dilemma: Inferential preferences over populations. arXiv: 2107.05064Google Scholar
Horton, J., Rand, D., Zeckhauser, R. (2013). The online laboratory: Conducting experiments in a real labor market. Experimental Economics, 14, 399–425. 10.1007/s10683-011-9273-9CrossRefGoogle Scholar
Litman, L., Moss, A., Rosenzweig, C., & Robinson, J. (2021). Reply to MTurk, Prolific or panels? Choosing the right audience for online research. https://ssrn.com/abstract=3775075 or https://doi.org/10.2139/ssrn.3775075.CrossRefGoogle Scholar
Peer, E., Rothschild, D., Gordon, A., Evernden, Z., Damer, E. (2021). Data quality of platforms and panels for online behavioral research. Behavior Research Methods, 10.3758/s13428-021-01694-3CrossRefGoogle Scholar
Weimann, J., Brosig-Koch, J. (2019). Methods in Experimental Economics, Springer 10.1007/978-3-319-93363-4CrossRefGoogle Scholar
Supplementary material: File

Buso et al. supplementary material

Buso et al. supplementary material
Download Buso et al. supplementary material(File)
File 502.7 KB