Hostname: page-component-745bb68f8f-v2bm5 Total loading time: 0 Render date: 2025-01-08T16:07:03.589Z Has data issue: false hasContentIssue false
  • English
  • Français

Multilevel Multivariate Meta-analysis with Application to Choice Overload

Published online by Cambridge University Press:  01 January 2025

Blakeley B. McShane  and
Ulf Böckenholt
Blakeley B. McShane*
Affiliation:
Northwestern University
Ulf Böckenholt
Affiliation:
Northwestern University
*
Correspondence should be made toBlakeleyB.McShane,Kellogg School of Management, NorthwesternUniversity, Evanston, IL 60208, USA. Email: b-mcshane@kellogg.northwestern.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

We introduce multilevel multivariate meta-analysis methodology designed to account for the complexity of contemporary psychological research data. Our methodology directly models the observations from a set of studies in a manner that accounts for the variation and covariation induced by the facts that observations differ in their dependent measures and moderators and are nested within, for example, papers, studies, groups of subjects, and study conditions. Our methodology is motivated by data from papers and studies of the choice overload hypothesis. It more fully accounts for the complexity of choice overload data relative to two prior meta-analyses and thus provides richer insight. In particular, it shows that choice overload varies substantially as a function of the six dependent measures and four moderators examined in the domain and that there are potentially interesting and theoretically important interactions among them. It also shows that the various dependent measures have differing levels of variation and that levels up to and including the highest (i.e., the fifth, or paper, level) are necessary to capture the variation and covariation induced by the nesting structure. Our results have substantial implications for future studies of choice overload.

Keywords

multilevelmultivariatemeta-analysischoiceoverload
Type
Original Paper
Information
Psychometrika , Volume 83 , Issue 1 , March 2018 , pp. 255 - 271
Copyright
Copyright © 2017 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

The authors thank Alex Chernev for his encouragement and support of this article. Supplementary materials for this article are available online at http://www.blakemcshane.com/research.

References

Akaike, H., (1974). A new look at the statistical model identification, IEEE Transactions on Automatic Control, 19(6) 716–723.CrossRefGoogle Scholar
Becker, B. J., (2000). Multivariate meta-analysis. In Tinsley, H. E. A. & Brown, S. D. (Eds.), Handbook of applied multivariate statistics and mathematical modeling. San Diego: Academic Press.Google Scholar
Becker, B.J., Hedges, L.V., Pigott, T.D., Statistical analysis policy brief 2004 Oslo: The Campbell Collaboration.Google Scholar
Berkey, C SHoaglin, D C, Antczak-Bouckoms, A., Mosteller, F., Colditz, G A, (1998). Meta-analysis of multiple outcomes by regression with random effects, Statistics in Medicine, 17(22) 2537–2550 ISSN 1097-0258.3.0.CO;2-C>CrossRefGoogle ScholarPubMed
Chernev, A., (2003). When more is less and less is more: The role of ideal point availability and assortment in consumer choice, Journal of Consumer Research, 30(2) 170–183.CrossRefGoogle Scholar
Chernev, A., (2003). Product assortment and individual decision processes, Journal of Personality and Social Psychology, 85, 151–162.CrossRefGoogle ScholarPubMed
Chernev, A., (2005). Feature complementarity and assortment in choice, Journal of Consumer Research, 31(4) 748–759.CrossRefGoogle Scholar
Chernev, A., (2006). Decision focus and consumer choice among assortments, Journal of Consumer Research, 33(1) 50–59.CrossRefGoogle Scholar
Chernev, A., Hamilton, R., (2009). Assortment size and option attractiveness in consumer choice among retailers, Journal of Marketing Research, 46(3) 410–420.CrossRefGoogle Scholar
Chernev, A., Böckenholt, U., Goodman, J., (2015). Choice overload: A conceptual review and meta-analysis, Journal of Consumer Psychology, 25(2) 333–358.CrossRefGoogle Scholar
Cohen, J., (1992). A power primer, Psychological Bulletin, 112, 155–159.CrossRefGoogle ScholarPubMed
Diehl, K., Poynor, C., (2010). Great expectations?! Assortment size, expectations and satisfaction, Journal of Marketing Research, 47(2) 312–322.CrossRefGoogle Scholar
Fasolo, B., Carmeci, F.A., Misuraca, R., (2009). The effect of choice complexity on perception of time spent choosing: When choice takes longer but feels shorter, Psychology and Marketing, 26(3) 213–228.CrossRefGoogle Scholar
Goodman, J.K., Malkoc, S.A., (2012). Choosing here and now versus there and later: The moderating role of psychological distance on assortment size preferences, Journal of Consumer Research, 39(4) 751–768.CrossRefGoogle Scholar
Gourville, J TSoman, D., (2005). Overchoice and assortment type: When and why variety backfires., Marketing Science, 24(3) 382–395.CrossRefGoogle Scholar
Greifeneder, R., Scheibehenne, B., Kleber, N., (2010). Less may be more when choosing is difficult: Choice complexity and too much choice, Acta Psychologica, 133(1) 45–50.CrossRefGoogle ScholarPubMed
Harville, D.A., (1977). Maximum likelihood approaches to variance component estimation and to related problems, Journal of the American Statistical Association, 72(358) 320–338.CrossRefGoogle Scholar
Haynes, G.A., (2009). Testing the boundaries of the choice overload phenomenon: The effect of number of options and time pressure on decision difficulty and satisfaction, Psychology & Marketing, 26(3) 204–212.CrossRefGoogle Scholar
Inbar, Y., Botti, S., Hanko, K., (2011). Decision speed and choice regret: When haste feels like waste, Journal of Experimental Social Psychology, 47(3) 533–540.CrossRefGoogle Scholar
Ishak, K.J., Platt, R.W., Joseph, L., Hanley, J.A., (2008). Impact of approximating or ignoring within-study covariances in multivariate meta-analyses, Statistics in Medicine, 27, 670–686.CrossRefGoogle ScholarPubMed
Iyengar, S.S., Lepper, M.R., (2000). When choice is demotivating: Can one desire too much of a good thing?, Journal of Personality and Social Psychology, 79(6) 996–1006.CrossRefGoogle ScholarPubMed
Kalaian, H.A., Raudenbush, S.W., (1996). A multivariate mixed linear model for meta-analysis, Psychological Methods, 1(3) 227–235.CrossRefGoogle Scholar
Lin, C-HWu, P-H, (2006). The effect of variety on consumer preferences: The role of need for cognition and recommended alternatives, Social Behavior and Personality: An International Journal, 34(7) 865–875.CrossRefGoogle Scholar
McShane, B.B., Böckenholt, U., (2014). You cannot step into the same river twice: When power analyses are optimistic, Perspectives on Psychological Science, 9(6) 612–625.CrossRefGoogle ScholarPubMed
McShane, B.B., Böckenholt, U., (2017). Single paper meta-analysis: Benefits for study summary, theory-testing, and replicability, Journal of Consumer Research, 43, 1048–1063.CrossRefGoogle Scholar
McShane, B.B., Böckenholt, U., Hansen, K.T., (2016). Adjusting for publication bias in meta-analysis: An evaluation of selection methods and some cautionary notes, Perspectives on Psychological Science, 11(5) 730–749.CrossRefGoogle ScholarPubMed
Mogilner, C., Rudnick, T., Iyengar, S.S., (2008). The mere categorization effect: How the presence of categories increases choosersí perceptions of assortment variety and outcome satisfaction, Journal of Consumer Research, 35(2) 202–215.CrossRefGoogle Scholar
Morrin, M., Broniarczyk, S MInman, J J, (2012). Plan format and participation in 401 (k) plans: The moderating role of investor knowledge., Journal of Public Policy & Marketing, 31(2) 254–268.CrossRefGoogle Scholar
Oppewal, H., Koelemeijer, K., (2005). More choice is better: Effects of assortment size and composition on assortment evaluation, International Journal of Research in Marketing, 22(1) 45–60.CrossRefGoogle Scholar
Payne, J.W., Bettman, J.R., Johnson, E.J., The adaptive decision maker 1993 Cambridge: Cambridge University Press.CrossRefGoogle Scholar
Riley, R.D., (2009). Multivariate meta-analysis: The effect of ignoring within-study correlation, Journal of the Royal Statistical Society, A, 172, 789–811.CrossRefGoogle Scholar
Robinson, G.K., (1991). That blup is a good thing: The estimation of random effects, Statistical Science, 6(1) 15–32.Google Scholar
Scheibehenne, B., Greifeneder, R., Todd, P.M., (2009). What moderates the too-much-choice effect?, Psychology and Marketing, 26(3) 229–253.CrossRefGoogle Scholar
Scheibehenne, B., Greifeneder, R., Todd, P.M., (2010). Can there ever be too many options? A meta-analytic review of choice overload, Journal of Consumer Research, 37(3) 409–425.CrossRefGoogle Scholar
Schwartz, B., (2004). The paradox of choice: Why more is less, New York: Ecco.Google Scholar
Sela, A., Berger, J., Liu, W., (2009). Variety, vice, and virtue: How assortment size influences option choice, Journal of Consumer Research, 35(6) 941–951.CrossRefGoogle Scholar
Shah, A.M., Wolford, G., (2007). Buying behavior as a function of parametric variation of number of choices, Psychological Science, 18(5) 369–370.CrossRefGoogle ScholarPubMed
Townsend, C., Kahn, B.E., (2014). The ‘visual preference heuristic’: The influence of visual versus verbal depiction on assortment processing, perceived variety, and choice overload, Journal of Consumer Research, 40(5) 993–1015.CrossRefGoogle Scholar