Hostname: page-component-745bb68f8f-l4dxg Total loading time: 0 Render date: 2025-01-12T14:45:28.323Z Has data issue: false hasContentIssue false
  • English
  • Français

Evaluation of Wine Judge Performance through Three Characteristics: Bias, Discrimination, and Variation*

Published online by Cambridge University Press:  08 June 2012

Jing Cao  and
Lynne Stokes
Jing Cao
Affiliation:
Department of Statistical Science, Southern Methodist University, Dallas, Texas, 75275, email: jcao@smu.edu
Lynne Stokes
Affiliation:
Department of Statistical Science, Southern Methodist University, Dallas, Texas, 75275, email: slstokes@smu. edu
Get access Rights & Permissions [Opens in a new window]

Abstract

Judge performance is a critical component of a wine competition's success. A number of studies have shown that wine judges may differ considerably in their opinions. In this paper, we have conducted an in-depth examination of wine judge performance at a U.S. wine competition. Three characteristics of judge's performance are examined: bias, discrimination ability, and variation. Based on the analysis, we can identify the judges who had discrepant scoring patterns and can gain insight into which of the three characteristics cause particular judges to disagree. The evaluation of wine judge performance through these three characteristics may provide useful information for training them to have consistent performance and in assisting competition coordinators in judge selection. (JEL Classification: C1, D8, Q13)

Type
Articles
Information
Journal of Wine Economics , Volume 5 , Issue 1 , Spring 2010 , pp. 132 - 142
Copyright
Copyright © American Association of Wine Economists 2010

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Ashenfelter, O. (2006). Tales from the crypt: Bruce Kaiser tells us about the trials and tribulations of a wine judge. Journal of Wine Economics, 1, 173–175.CrossRefGoogle Scholar
Cao, J., Stokes, L., and Zhang, S. (2010). A Bayesian approach to ranking and rater evaluation: an application to grant reviews. Journal of Educational and Behavioral Statistics, 35, 194–214.CrossRefGoogle Scholar
Carlin, B.P. and Louis, T.A. (2008). Bayesian Methods for Data Analysis. 3rd ed., Boca Raton: CRC Press.CrossRefGoogle Scholar
Cicchetti, D.V. (2004). Who won the 1976 wine tasting of French Bordeaux and U.S. cabernets? Parametrics to the rescue. Journal of Wine Research, 15, 211–220.CrossRefGoogle Scholar
Cicchetti, D.V. (2006). The Paris 1976 tastings revisited once more: comparing ratings of consistent and inconsistent tasters. Journal of Wine Economics, 1, 125–140.CrossRefGoogle Scholar
Cliff, M.A. and King, M.C. (1996). A proposed approach for evaluating expert judge performance using descriptive statistics. Journal of Wine Research, 7, 83–90.CrossRefGoogle Scholar
Cliff, M.A. and King, M.C. (1997). Application of eggshell plots for evaluation of judges at wine competition. Journal of Wine Research, 8, 75–80.CrossRefGoogle Scholar
Hodgson, R.T. (2008). An examination of judge reliability at a major U.S. wine competition. Journal of Wine Economics, 3, 105–113.CrossRefGoogle Scholar
Hodgson, R.T. (2009). An analysis of the concordance among 13 U.S. wine competitions. Journal of Wine Economics, 4, 1–9.CrossRefGoogle Scholar
Johnson, V.E. and Albert, J. A. (1999). Ordinal Data Modeling. New York: Springer.CrossRefGoogle Scholar
Mlodinow, L. (2009). A Hint of Hype, A Taste of Illusion. Wall Street Journal, Life & Style, November 20, 2009. online at: http://online.wsj.com/article/SB10001424052748703683804574533840282653628.htmlGoogle Scholar
Seaman, C.H., Dou, J., Cliff, M.A., Yuksel, D., and King, M.C. (2001). Evaluation of wine competition judge performance using principle component similarity analysis. Journal of Sensory Studies, 16, 287–300.Google Scholar
Young, H.P. (1988). Condorcet's theory of voting. The American Political Science Review, 82, 1231–1244.CrossRefGoogle Scholar