The Cambridge Handbook of Research Methods and Statistics for the Social and Behavioral Sciences Volume 2: Performing Research
Book contents
- The Cambridge Handbook of Research Methods and Statistics for the Social and Behavioral Sciences
- Cambridge Handbooks in Psychology
- The Cambridge Handbook of Research Methods and Statistics for the Social and Behavioral Sciences
- Copyright page
- Dedication
- Contents
- Figures
- Tables
- Contributors
- Preface
- Part I Quantitative Data Collection Sources
- 1 Student Samples in Research
- 2 Mechanical Turk: A Versatile Tool in the Behavioral Scientist’s Toolkit
- 3 Social Media Research
- 4 Prolific: Crowdsourcing Academic Online Research
- 5 Field Research
- 6 Organizational Research
- 7 Integrating Culture in Research
- 8 Mixed Methods and Multimethod Research
- Part II Important Methodological Considerations
- Part III Self-Report Measures
- Part IV Behavioral Measures
- Part V Physiological Measures
- Part VI Qualitative Data Collection Sources
- Index
- References
4 - Prolific: Crowdsourcing Academic Online Research
from Part I - Quantitative Data Collection Sources
Published online by Cambridge University Press: 12 December 2024
By
Book contents
- The Cambridge Handbook of Research Methods and Statistics for the Social and Behavioral Sciences
- Cambridge Handbooks in Psychology
- The Cambridge Handbook of Research Methods and Statistics for the Social and Behavioral Sciences
- Copyright page
- Dedication
- Contents
- Figures
- Tables
- Contributors
- Preface
- Part I Quantitative Data Collection Sources
- 1 Student Samples in Research
- 2 Mechanical Turk: A Versatile Tool in the Behavioral Scientist’s Toolkit
- 3 Social Media Research
- 4 Prolific: Crowdsourcing Academic Online Research
- 5 Field Research
- 6 Organizational Research
- 7 Integrating Culture in Research
- 8 Mixed Methods and Multimethod Research
- Part II Important Methodological Considerations
- Part III Self-Report Measures
- Part IV Behavioral Measures
- Part V Physiological Measures
- Part VI Qualitative Data Collection Sources
- Index
- References
Summary
Prolific is a website that offers researchers the ability to recruit and sample participants for online research. In contrast to earlier crowdsourcing platforms, such as Amazon Mechanical Turk (MTurk), it focuses primarily on academic and marketing research – typically done through online surveys and experiments. In this chapter, I aim to introduce this platform to researchers conducting online studies and to provide knowledge and practical advice on how to best use the platform for online research. The review includes explanations of how the site works, the composition of its pool of participants, the options available to researchers for sampling and recruiting participants online, how to achieve advanced abilities by connecting Prolific to research software (e.g., Qualtrics, Gorilla), and how to ensure high data quality when using Prolific. I then review the evidence on the current state of data quality on Prolific, suggesting that it can provide higher data quality than MTurk and also better than some commercial panels. I conclude with a summary of the advantages and disadvantages of using Prolific for online research and potential future developments in the platform that could promote more credible online research.
- Type
- Chapter
- Information
- The Cambridge Handbook of Research Methods and Statistics for the Social and Behavioral SciencesVolume 2: Performing Research, pp. 72 - 92Publisher: Cambridge University PressPrint publication year: 2024
References
Berinsky, A. J., Margolis, M. F., & Sances, M. W. (2014). Separating the shirkers from the workers? Making sure respondents pay attention on self‐administered surveys. American Journal of Political Science, 58(3), 739–753.CrossRefGoogle Scholar
Chandler, J. J., Mueller, P., & Paolacci, G. (2014). Nonnaïveté among Amazon Mechanical Turk workers: Consequences and solutions for behavioral researchers. Behavior Research Methods, 46(1), 112–130.CrossRefGoogle ScholarPubMed
Chandler, J. J., & Paolacci, G. (2017). Lie for a dime: When most prescreening responses are honest but most study participants are impostors. Social Psychological and Personality Science, 8(5), 500–508.CrossRefGoogle Scholar
Chandler, J. J., Paolacci, G., Peer, E., Mueller, P., & Ratliff, K. A. (2015). Using nonnaive participants can reduce effect sizes. Psychological Science, 26(7), 1131–1139.CrossRefGoogle ScholarPubMed
Chandler, J. J., Rosenzweig, C., Moss, A. J., Robinson, J., & Litman, L. (2019). Online panels in social science research: Expanding sampling methods beyond Mechanical Turk. Behavior Research Methods, 51(5), 2022–2038.CrossRefGoogle ScholarPubMed
Douglas, B. D., Ewell, P. J., & Brauer, M. (2023). Data quality in online human-subjects research: Comparisons between MTurk, Prolific, CloudResearch, Qualtrics, and SONA. PLOS ONE, 18(3), e0279720.CrossRefGoogle ScholarPubMed
Gerlach, P., Teodorescu, K., & Hertwig, R. (2019). The truth about lies: A meta-analysis on dishonest behavior. Psychological Bulletin, 145(1), 1–44.CrossRefGoogle Scholar
Goodman, J. K., Cryder, C. E., & Cheema, A. (2013). Data collection in a flat world: The strengths and weaknesses of Mechanical Turk samples. Journal of Behavioral Decision Making, 26(3), 213–224.CrossRefGoogle Scholar
Goodman, J. K., & Paolacci, G. (2017). Crowdsourcing consumer research. Journal of Consumer Research, 44(1), 196–210.CrossRefGoogle Scholar
Hauser, D. J., Moss, A. J., Rosenzweig, C., Jaffe, S., & Robinson, J. (2022). Evaluating CloudResearch’s approved group as a solution for problematic data quality on MTurk. Behavioral Research Methods, 55(8), 3953–3964. https://doi.org/10.31234/osf.io/48yxjCrossRefGoogle Scholar
Nichols, A. L., & Edlund, J. E. (2020). Why don’t we care more about carelessness? Understanding the causes and consequences of careless participants. International Journal of Social Research Methodology, 23(6), 625-638.CrossRefGoogle Scholar
Oppenheimer, D. M., Meyvis, T., & Davidenko, N. (2009). Instructional manipulation checks: Detecting satisficing to increase statistical power. Journal of Experimental Social Psychology, 45(4), 867–872.CrossRefGoogle Scholar
Paolacci, G., Chandler, J., & Ipeirotis, P. G. (2010). Running experiments on Amazon Mechanical Turk. Judgment and Decision making, 5(5), 411–419.CrossRefGoogle Scholar
Peer, E., Brandimarte, L., Samat, S., & Acquisti, A. (2017). Beyond the Turk: Alternative platforms for crowdsourcing behavioral research. Journal of Experimental Social Psychology, 70, 153–163.CrossRefGoogle Scholar
Peer, E., Rothschild, D., & Gordon, A. (2023). Behavioral Lab 3.0. Towards the next generation of online behavioral research [preprint]. PsyArXiv.Google Scholar
Peer, E., Rothschild, D., Gordon, A., Evernden, Z., & Damer, E. (2022). Data quality of platforms and panels for online behavioral research. Behavior Research Methods, 54, 1643–1662. https://doi.org/10.3758/s13428-021-01694-3CrossRefGoogle ScholarPubMed
Peer, E., Vosgerau, J., & Acquisti, A. (2014). Reputation as a sufficient condition for data quality on Amazon Mechanical Turk. Behavior Research Methods, 46(4), 1023–1031.CrossRefGoogle ScholarPubMed
Stewart, N., Ungemach, C., Harris, A. J., Bartels, D. M., Newell, B. R., Paolacci, G., & Chandler, J. (2015). The average laboratory samples a population of 7,300 Amazon Mechanical Turk workers. Judgment and Decision Making, 10(5), 479–491.CrossRefGoogle Scholar
Thomas, K. A., & Clifford, S. (2017). Validity and Mechanical Turk: An assessment of exclusion methods and interactive experiments. Computers in Human Behavior, 77, 184–197.CrossRefGoogle Scholar
Tversky, A., & Kahneman, D. (1981). The framing of decisions and the psychology of choice. Science, 211(4481), 453–458.CrossRefGoogle ScholarPubMed