Hostname: page-component-78c5997874-dh8gc Total loading time: 0 Render date: 2024-11-11T06:15:48.443Z Has data issue: false hasContentIssue false
  • English
  • Français

Analyses of Wine-Tasting Data: A Tutorial*

Published online by Cambridge University Press:  04 November 2014

Ingram Olkin ,
Ying Lou ,
Lynne Stokes  and
Jing Cao
Ingram Olkin
Affiliation:
Ingram Olkin is Professor Emeritus, Department of Statistics, Stanford University; e-mail: olkin@stanford.edu.
Ying Lou
Affiliation:
Ying Lou is Manager Statistician, Lilly Suzhou Pharmaceutical Co, Shanghai Branch; e-mail: lou_ying@lilly.com.
Lynne Stokes
Affiliation:
Lynne Stokes is Professor, Department of Statistical Science, Southern Methodist University; e-mail: slstokes@smu.edu.
Jing Cao
Affiliation:
Jing Cao is Associate Professor, Department of Statistical Science, Southern Methodist University; e-mail: jcao@smu.edu.
Get access Rights & Permissions [Opens in a new window]

Abstract

The purpose of this paper is to provide a tutorial of data analysis methods for answering questions that arise in analyzing data from wine-tasting events: (i) measuring agreement of two judges and its extension to m judges; (ii) making comparisons of judges across years; (iii) comparing two wines; (iv) designing tasting procedures to reduce burden of multiple tastings; (v) ranking of judges; and (vi) assessing causes of disagreement. In each case we describe one or more analyses and make recommendations on the conditions of use for each. (JEL Classifications: C10, C12, C13, C59, C90)

Keywords

correlationdesign of experimentsjudge agreementmeasures of concordancemeta-analysismultiple judges
Type
Articles
Information
Journal of Wine Economics , Volume 10 , Issue 1 , May 2015 , pp. 4 - 30
Copyright
Copyright © American Association of Wine Economists 2014 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

*

The authors wish to thank the editor Karl Storchmann and the reviewer for their comments and suggestions. Jing Cao was partially supported by NSF grant IIS-1302564.

References

Amerine, M.A., and Roessler, E.B. (1976). Wines, Their Sensory Evaluation. San Francisco: W.H. Freeman.Google Scholar
Ashenfelter, O., and Quandt, R. (1999). Analyzing a wine tasting statistically. Chance, 12(3), 16–20.Google Scholar
Cao, J., and Stokes, L. (2010). Evaluation of wine judge performance through three characteristics: Bias, discrimination, and variation. Journal of Wine Economics, 5(1), 132–142.Google Scholar
Cochran, W.G., and Cox, G.M. (1957). Experimental Designs. New York: Wiley.Google Scholar
Cohen, J. (1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20(1), 37–46.Google Scholar
Cohen, J. (1968). Weighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit. Psychological Bulletin, 70(4), 213–220.CrossRefGoogle ScholarPubMed
Cooper, H., Hedges, L. V., and Valentine, J. C. (2009). The Handbook of Research Synthesis and Meta-Analysis, 2nd ed. New York: Russell Sage Foundation.Google Scholar
Dixon, W.J., and Massey, F.J. Jr. (1969). Introduction to Statistical Analysis, 3rd ed. New York: McGraw-Hill.Google Scholar
Dixon, W.J., and Mood, A.M. (1946). The statistical sign test. Journal of the American Statistical Association, 41(236), 557–566.Google Scholar
Fisher, R.A., and Yates, F. (1974). Statistical Tables for Biological, Agricultural and Medical Research, 2nd ed. Edinburgh: Oliver and Boyd.Google Scholar
Friedman, M. (1937). The use of ranks to avoid the assumption of normality implicit in the analysis of variance. Journal of the American Statistical Association, 32(200), 675–701.Google Scholar
Gawel, R., and Godden, P.W. (2008). Evaluation of the consistency of wine quality assessments from expert wine tasters. Australian Journal of Grape and Wine Research, 14(1), 1–9.Google Scholar
Ginsburgh, V., and Zang, I. (2012). Shapley ranking of wines. Journal of Wine Economics, 7(2), 169–180.Google Scholar
Hodgson, R.T. (2008). An examination of judge reliability at a major U.S. wine competition. Journal of Wine Economics, 3(2), 105–113.Google Scholar
Hodgson, R.T. (2009). An analysis of the concordance among 13 U.S. wine competitions. Journal of Wine Economics, 4(1), 1–9.Google Scholar
Kendall, M.G. (1938). A new measure of rank correlation. Biometrika, 30(1/2), 81–93.Google Scholar
Kendall, M.G. (1945). The treatment of ties in ranking problems. Biometrika, 33(3), 239–251.Google Scholar
Kendall, M.G. (1962). Rank Correlation Methods, 3rd ed. London: Charles Griffin.Google Scholar
Kendall, M.G., and Smith, B.B. (1939). The problem of m rankings. Annals of Mathematical Statistics, 10(3), 275–287.Google Scholar
Larntz, K., and Perlman, A. (1988). A simple test for equality of correlation matrices. In Statistical Decision Theory and Related Topics IV, Berger, J.O. and Gupta, S.S. (Eds.), 289–298.Google Scholar
Legendre, P. (2005). Species associations: The Kendall coefficient of concordance revisited. Journal of Agricultural, Biological, and Environmental Statistics, 10(2), 226–245.Google Scholar
McGraw, K.O., and Wong, S. (1996). Forming inferences about some intraclass correlation coefficients. Psychological Methods, 1(1), 30–46.Google Scholar
Miller, G.A. (1956). The magical number seven, plus or minus two some limits on our capacity for processing information. Psychological Review, 63(2), 81–97.Google Scholar
Quandt, R. (2006). Measurement and inference in wine tasting. Journal of Wine Economics, 1(1), 7–30.Google Scholar
Quandt, R. (2007). A Note on a test for the sum of ranksums. Journal of Wine Economics, 2(1), 98–102.CrossRefGoogle Scholar
Shrout, P.E., and Fleiss, J.L. (1979). Intraclass correlations: uses in assessing rater reliability. Psychological Bulletin, 86(2), 420–428.Google Scholar
Siegel, S. (1956). Nonparametric Statistics for the Behavioral Sciences, New York: McGraw-Hill.Google Scholar
Spearman, C. (1904). The proof and measurement of association between two things. American Journal of Psychology, 15(1), 72–101.Google Scholar
Supplementary material: File

Olkin Supplementary Material

Supplementary Material

Download Olkin Supplementary Material(File)
File 9.8 KB