Hostname: page-component-78c5997874-mlc7c Total loading time: 0 Render date: 2024-11-10T07:25:13.311Z Has data issue: false hasContentIssue false
  • English
  • Français

Judging reliability at wine and water competitions

Published online by Cambridge University Press:  25 November 2022

Elena C. Berg ,
Michael Mascha  and
Kevin W. Capehart Open the ORCID record for Kevin W. Capehart [Opens in a new window]
Elena C. Berg
Affiliation:
Department of Computer Science, Math, and Environmental Science, The American University of Paris, 5 Boulevard de la Tour-Maubourg, 75007 Paris, France
Michael Mascha
Affiliation:
FineWaters, Fine Water Academy
Kevin W. Capehart*
Affiliation:
Department of Economics, California State University, Fresno, 5245 N Backer Ave M/S PB20, Fresno, CA 93740
*
Corresponding author: Kevin W. Capehart, email: kcapehart@csufresno.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

Studies suggest the inter-rater reliability of judges at wine competitions is higher than what would be expected by random chance, but lower than what is observed when experts in other fields make judgments specific to their expertise. To further contextualize the (un-) reliability of wine judging while also extending the study of fine water, we examine the inter-rater reliability of judges at an annual international competition for bottled waters. We find that the inter-rater reliability of water judging is generally better than chance and, at best, about the same as the inter-rater reliability of wine judging at some wine competitions. These results suggest that perceptible differences between fine waters exist but are less pronounced than those between fine wines and, also, that aesthetic standards with respect to fine waters exist but are currently less established than those for fine wines.

Keywords

blind tastingexpert evaluationinter-rater reliabilitywaterwine

JEL classification

Q25: Water L66: Food • Beverages • Cosmetics • Tobacco • Wine and Spirits D83: Search • Learning • Information and Knowledge • Communication • Belief • Unawareness
Type
Article
Information
Journal of Wine Economics , Volume 17 , Issue 4 , November 2022 , pp. 311 - 328
Copyright
Copyright © The Author(s), 2022. Published by Cambridge University Press on behalf of American Association of Wine Economists

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Ashenfelter, O., and Quandt, R. (1999). Analyzing a wine tasting statistically. Chance, 12(3), 16–20.CrossRefGoogle Scholar
Ashton, R. H. (2011). Improving experts’ wine quality judgments: Two heads are better than one. Journal of Wine Economics, 6(2), 160–178.CrossRefGoogle Scholar
Ashton, R. H. (2012). Reliability and consensus of experienced wine judges: Expertise within and between? Journal of Wine Economics, 7(1), 70–87.CrossRefGoogle Scholar
Ashton, R. H. (2017). Dimensions of expertise in wine evaluation. Journal of Wine Economics, 12(1), 59–83.CrossRefGoogle Scholar
Azoulay, A., Garzon, P., and Eisenberg, M. J. (2001). Comparison of the mineral content of tap water and bottled waters. Journal of General Internal Medicine, 16(3), 168–175.CrossRefGoogle ScholarPubMed
Biro, A. (2019). Reading a water menu: Bottled water and the cultivation of taste. Journal of Consumer Culture, 19(2), 231–251.CrossRefGoogle Scholar
Bitter, C. (2017). Wine competitions: Reevaluating the gold standard. Journal of Wine Economics, 12(4), 395–404.CrossRefGoogle Scholar
Bodington, J. (2020). Rate the raters: A note on wine judge consistency. Journal of Wine Economics, 15(4), 363–369.CrossRefGoogle Scholar
Burlingame, G. A., Dietrich, A. M., and Whelton, A. J. (2007). Understanding the basics of tap water taste. American Water Works Association, 99(5), 100–111.CrossRefGoogle Scholar
Cal Expo (2019). California Wine Competition 2019 Judging Photos. Available at https://calexpostatefair.com/participate/competitions/california-commercial-wine/california-wine-competition-2019-judging-photos/ (accessed May 1, 2021).Google Scholar
California Green Medal (2022). 2022 California Green Medal Winners. Available at https://www.greenmedal.org/all-winners/2022-winners/ (accessed July 1, 2022).Google Scholar
Capehart, K. W. (2015). Fine water: A hedonic pricing approach. Journal of Wine Economics, 10(2), 129–150.CrossRefGoogle Scholar
Capehart, K. W., and Berg, E. C. (2018). Fine water: A blind taste test. Journal of Wine Economics, 13(1), 20–40.CrossRefGoogle Scholar
Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155–159.CrossRefGoogle ScholarPubMed
Dietrich, A. M., and Burlingame, G. A. (2020). A review: The challenge, consensus, and confusion of describing odors and tastes in drinking water. Science of the Total Environment, 713, 1–11, doi: 10.1016/j.scitotenv.2019.135061.CrossRefGoogle ScholarPubMed
DiNardo, J., and Tobias, J. L. (2001). Nonparametric density and regression estimation. Journal of Economic Perspectives, 15(4), 11–28.CrossRefGoogle Scholar
Fine Water Academy (2022). Certified Water Sommelier. Available at https://finewateracademy.talentlms.com/index (accessed July 1, 2022).Google Scholar
Fine Water Society (2016). First international fine water tasting competition in China with renowned water sommeliers [Press Release]. Available at https://web.archive.org/web/20220614195049/http://finewaters.com/premium-bottled-water/finewaters-tasting-competitions/guangzhou-china-2016/264-press-release (accessed June 14, 2022).Google Scholar
Fine Water Society (2021). Jury panel 2021. Available at https://finewaters.com/fine-water-society/taste-design-awards/bled-slovenia-2021/jury-panel.Google Scholar
Fulcher, J. (2017). Judging water at Berkeley Springs international water tasting competition. Water Environment Federation Highlights, April 7. Available at http://news.wef.org/judging-water-at-berkeley-springs-international-water-tasting-competition/ (accessed April 7, 2017).Google Scholar
Gergaud, O., Ginsburgh, V., and Moreno-Ternero, J. D. (2021). Wine ratings: Seeking a consensus among tasters via normalization, approval, and aggregation. Journal of Wine Economics, 16(3), 321–342.CrossRefGoogle Scholar
Gleick, P. H. (2010). Bottled and Sold: The Story Behind Our Obsession with Bottled Water. Washington, DC: Island Press.Google Scholar
Hodgson, R. T. (2008). An examination of judge reliability at a major U.S. wine competition. Journal of Wine Economics, 3(2), 105–113.CrossRefGoogle Scholar
Hodgson, R. T. (2009). An analysis of the concordance among 13 U.S. wine competitions. Journal of Wine Economics, 4(1), 1–9.CrossRefGoogle Scholar
Hooks, C. (2013). Beyond Fiji and Perrier, with a water sommelier. New York Times, December 22. Available at http://nyti.ms/JS7oJT (accessed December 22, 2013).Google Scholar
Hulkower, N. D. (2009). The judgment of Paris according to Borda. Journal of Wine Research, 20(3), 171–182.CrossRefGoogle Scholar
Marcussen, H., Holm, P., and Hansen, H. (2013). Composition, flavor, chemical foodsafety, and consumer preferences of bottled water. Comprehensive Reviews in Food Science and Food Safety, 12(4), 333–352.CrossRefGoogle ScholarPubMed
Mascha, M. (2022). Fine Waters: A Connoisseur's Guide to the World of Premium Waters. Available at https://finewaters.com.Google Scholar
Oczkowski, E. (2017). The preferences and prejudices of Australian wine critics. Journal of Wine Research, 28(1), 56–67.CrossRefGoogle Scholar
Olkin, I., Lou, Y., Stokes, L., and Cao, J. (2015). Analyses of wine-tasting data: A tutorial. Journal of Wine Economics, 10(1), 4–30.CrossRefGoogle Scholar
Riese, Martin. (2022). How a water tasting event look like. TikTok, April 29. Available at https://www.tiktok.com/@martinrieseofficial/video/7092105196326702379 (accessed April 29, 2022).Google Scholar
Ruscio, J. (2008). A probability-based measure of effect size: Robustness to base rates and other factors. Psychological Methods, 13(1), 19–30.CrossRefGoogle ScholarPubMed
Taber, G. M. (2006). The Judgment of Paris: California vs. France and the Historic 1976 Paris Tasting that Revolutionized Wine. New York, NY: Scribner.Google Scholar
Taber, G. M. (2012). The judgment of Princeton. Journal of Wine Economics, 7(2), 143–151.CrossRefGoogle Scholar
Tishgart, S. (2017). Ask a water sommelier. Grub Street, January 8. Available at https://www.grubstreet.com/2017/01/whats-the-best-bottled-water.html (accessed January 8, 2017).Google Scholar
Supplementary material: File

Berg et al. supplementary material

Berg et al. supplementary material

Download Berg et al. supplementary material(File)
File 427.2 KB