Hostname: page-component-78c5997874-8bhkd Total loading time: 0 Render date: 2024-11-11T06:03:41.376Z Has data issue: false hasContentIssue false
  • English
  • Français

An Analysis of Wine Critic Consensus: A Study of Washington and California Wines*

Published online by Cambridge University Press:  16 April 2015

Eric T. Stuen Open the ORCID record for Eric T. Stuen [Opens in a new window] ,
Jon R. Miller  and
Robert W. Stone
Eric T. Stuen*
Affiliation:
Department of Business, University of Idaho, College of Business and Economics, 875 Perimeter Drive MS3161, Moscow, ID 83844-3161
Jon R. Miller
Affiliation:
Department of Business, University of Idaho, College of Business and Economics, 875 Perimeter Drive MS3161, Moscow, ID 83844-3161; e-mail: jrmecon@uidaho.edu.
Robert W. Stone
Affiliation:
Department of Accounting, University of Idaho, College of Business and Economics, 875 Perimeter Drive MS3161, Moscow, ID 83844-3161; e-mail: rstone@uidaho.edu.
*
e-mail: estuen@uidaho.edu (corresponding author).
Get access Rights & Permissions [Opens in a new window]

Abstract

We examine the degree of consensus in quality ratings of prominent U.S. wine publications. For the purposes of wine consumption and research, are ratings on the ubiquitous 100-point scale reliable measures of quality? The value of expert judgment has been called into question by a number of studies, especially in the context of wine competitions and tasting events. Using data on 853 wines, we find a moderately high level of consensus, measured by the correlation coefficient, between most pairs of publications, similar to the level found by Ashton (2013). Rank and intraclass correlations are similar. Consensus is not found to be related to the blinding policies (or lack thereof) of the critical publications. (JEL Classifications: C93, D46)

Keywords

critic consensusproduct qualitywine ratings
Type
Articles
Information
Journal of Wine Economics , Volume 10 , Issue 1 , May 2015 , pp. 47 - 61
Copyright
Copyright © American Association of Wine Economists 2015 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

*

We thank an anonymous referee for valuable comments. We also thank Dominic Cicchetti, Omer Gokcekus, Robin Goldstein, Robert Hodgson, Lynne Stokes, and other participants at the 2014 AAWE Annual Conference in Walla Walla, WA, for helpful comments and Stefan Richardson and Matthew Hill for excellent research assistance.

References

Ali, H.H., Lecocq, S., and Visser, M. (2008). The impact of gurus: Parker grades and en primeur wine prices. Economic Journal, 118(529), F158–F173.CrossRefGoogle Scholar
Ashenfelter, O. (2006). Tales from the crypt: Bruce Kaiser tells about the trials and tribulations of a wine judge. Journal of Wine Economics, 1(2), 173–175.CrossRefGoogle Scholar
Ashenfelter, O., and Quandt, R. (1999). Analyzing a wine tasting statistically. Chance, 12(3), 16–20.CrossRefGoogle Scholar
Ashenfelter, O., and Storchmann, K. (2012). Editorial: The Judgment of Princeton and other articles. Journal of Wine Economics, 7(2), 139–142.Google Scholar
Ashton, R.H. (2011). Improving experts’ wine quality judgments: Two heads are better than one. Journal of Wine Economics, 6(2), 160–178.Google Scholar
Ashton, R.H. (2012). Reliability and consensus of experienced wine judges: Expertise within and between? Journal of Wine Economics, 7(1), 70–87.CrossRefGoogle Scholar
Ashton, R.H. (2013). Is there consensus among wine quality ratings of prominent critics? An empirical analysis of red Bordeaux, 2004–2010. Journal of Wine Economics, 8(2), 225–234.Google Scholar
Ashton, R.H. (2014). Wine as an experience good: Price versus enjoyment in blind tastings of expensive and inexpensive wines. Journal of Wine Economics, 9(2), 171–182.Google Scholar
Bodington, J. (2012). 804 tastes: Evidence on preferences, randomness, and value from double-blind wine tastings. Journal of Wine Economics, 7(2), 181–191.CrossRefGoogle Scholar
Brien, C.J., May, P., and Mayo, O. (1987). Analysis of judge performance in wine-quality evaluations. Journal of Food Science, 52(5), 1273–1279.Google Scholar
Cao, J., and Stokes, L. (2010). Evaluation of wine judge performance through three characteristics: Bias, discrimination, and variation. Journal of Wine Economics, 5(1), 132–142.CrossRefGoogle Scholar
Cicchetti, D.V. (2004a). Who won the 1976 blind tasting of French Bordeaux and U.S. cabernets? Parametrics to the rescue. Journal of Wine Research, 15(3), 211–220.Google Scholar
Cicchetti, D.V. (2004b). On designing experiments and analyzing data to assess the reliability and accuracy of blind tastings. Journal of Wine Research, 15(3), 221–226.CrossRefGoogle Scholar
Cicchetti, D. (2006). The Paris 1976 wine tastings revisited once more: Comparison of ratings of consistent and inconsistent tasters. Journal of Wine Economics, 1(2), 125–140.CrossRefGoogle Scholar
Cliff, M.A., and King, M.C. (1997). Application of eggshell plots for evaluation of judges at wine competitions. Journal of Wine Research, 8, 75–80.CrossRefGoogle Scholar
Gawel, R., and Godden, P.W. (2008). Evaluation of the consistency of wine quality assessment from expert wine tasters. Australian Journal of Grape and Wine Research, 14(1), 1–8.CrossRefGoogle Scholar
Ginsburgh, V., and Zang, I. (2012). Shapley ranking of wines. Journal of Wine Economics, 7(2), 169–180.CrossRefGoogle Scholar
Gokcekus, O., and Nottebaum, D. (2011). The buyer's dilemma—Whose rating should a wine drinker pay attention to? AAWE Working Paper 91.Google Scholar
Goldstein, R., Almenberg, J., Dreber, A., Emerson, J.W., Herschkowitsch, A., and Katz, J. (2008). Do more expensive wines taste better? Evidence from a large sample of blind tastings. Journal of Wine Economics, 3(1), 1–9.CrossRefGoogle Scholar
Hodgson, R.T. (2008). An examination of judge reliability at a major U.S. wine competition. Journal of Wine Economics, 3(2), 105–113.CrossRefGoogle Scholar
Hodgson, R.T. (2009a). An analysis of the concordance among 13 U.S. wine competitions. Journal of Wine Economics, 4(1), 1–9.Google Scholar
Hodgson, R.T. (2009b). How expert are “expert” wine judges? Journal of Wine Economics, 4(2), 233–241.Google Scholar
Lawless, H., Liu, Y., and Goldwyn, C. (1997). Evaluation of wine quality using a small-panel hedonic scaling method. Journal of Sensory Studies, 12(4), 317–332.CrossRefGoogle Scholar
McGraw, K.O., and Wong, S.P. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1(1), 30–46.CrossRefGoogle Scholar
Miller, J., Genc, I., and Driscoll, A. (2007). Wine price and quality: in search of a signaling equilibrium in 2001 California cabernet sauvignon. Journal of Wine Research, 18(1), 35–46.Google Scholar
Olkin, I., Lou, Y., Stokes, L., and Cao, J. (2014). Analyses of Wine-Tasting Data: A Tutorial. Journal of Wine Economics, 10(1), doi:10.1017/jwe.2014.26.Google Scholar
Quandt, R.E. (2012). Comments on the Judgment of Princeton. Journal of Wine Economics, 7(2), 152–154.Google Scholar
Taber, G.M. (2012). The Judgment of Princeton. Journal of Wine Economics, 7(2), 143–151.Google Scholar
Ward, D.L. (2012). A graphical and statistical analysis of the Judgment of Princeton wine tasting. Journal of Wine Economics, 7(2), 155–168.Google Scholar
Weil, R.L. (2005). Analysis of reserve and regular bottlings: Why pay for a difference no critics claim to notice? Chance, 18(3), 9–15.Google Scholar