The Legacy of Gurus: The Impact of Armin Diel and Joel Payne on Winery Ratings in Germany

Abstract

Changes in winery ratings in leading wine guides, that is, improvements as well as deteriorations, are typically attributed to corresponding changes in the quality of the wines produced by the respective winery. What remains unexplored in this context is changes in editorship and/or changes in the composition of the wine tasting teams working for the respective guide. Using data from two particularly prestigious German wine guides (Gault Millau and Vinum), this paper shows that these latter changes have a rather small, yet statistically significant impact on changes in winery ratings. Thus, consumers are well-advised to consider these changes before making their purchasing decision. (JEL Classifications: L21, M30, Q13)

I. Introduction and Research Question

Gault Millau, undoubtedly the most important wine guide in Germany, has been published annually since 1994. The 2018 edition of what has often been considered the “bible of German wine” was, therefore, the guide's 25^th anniversary issue.¹ During this period, the publisher and the editors experienced at least two “shocks” that can be used to analyze the impact of “wine gurus” on the consistency of winery ratings.² First, Armin Diel—one of the co-editors of the guide from 1994 to 2009—stepped down in the latter year due to massive protests by leading wine makers against a highly controversial “voluntary fee” of €195 suggested by the publisher. Second, Joel Payne, the remaining co-editor, announced in June 2017 that he would leave Gault Millau and start a new wine guide, “Vinum,” the first edition of which appeared on November 24, 2017. Gault Millau followed three days later on November 27, 2017, with a new team of an editor and tasters.

Rating consistency is valuable to consumers because wine critics have been found to exert a considerable influence on purchasing decisions (e.g., Friberg and Grönqvist, 2012; Hilger, Rafert, and Villas-Boas, 2011; Thrane, 2019). Thus, the question arises whether and to what extent changes in the evaluation of a particular winery are due to changes in the quality produced by that winery or due to changes in the composition of the team evaluating that winery. While a change in the editorship of a prestigious wine guide will typically be announced by its publisher, changes in the composition of the team of tasters will most likely go unnoticed by most consumers. The question to be addressed in this paper is, therefore, whether the shocks mentioned earlier (i.e., the changes in editorship and the ensuing changes in the composition of the team of tasters) had any impact on the consistency of the evaluation of the wineries included in “Gault Millau.”

II. The “Critical Events”

From the beginning of his editorship, Armin Diel was criticized for his “dual role” as a wine critic on the one hand and as a wine maker on the other hand. Apart from writing extensively on wine, Diel was—and continues to be—the owner of “Schlossgut Diel,” one of the most prestigious estates in the Nahe area. Although his winery was never rated—but always listed and favorably described—in Gault Millau as long as he was one of its editors, he has always been accused of being “biased.”³ In the summer of 2009, 14 of the leading German wine producers “declared war” on Christian Verlag, publisher of the German Gault Millau, in a widely distributed open letter.⁴ The producers were upset by the publisher's request to pay a “voluntary marketing fee” of €195 to cover the production costs of the guide. Since the relations between some of the leading wine producers and the Gault Millau's editors had been reported to be strained for some time already, the owners of these wineries took this request as their reason to “revolt” against the publisher's course of action. What made this move particularly credible—and threatening to Gault Millau—was the fact that the 14 top-quality producers that threatened to boycott Gault Millau had in previous years always been praised and honored by the guide's editors.⁵ Following a heated debate, Armin Diel announced that he would step down as co-editor of Gault Millau.

In the summer of 2017, Christian Verlag sold the license to publish Gault Millau to Zabert + Sandmann, presumably due to financial distress. Joel Payne, along with his new deputy Carsten Henn, decided to stay with Christian Verlag and start a new project. He managed to bring on board “Vinum,” a Swiss-based monthly wine journal whose editors wanted to expand their portfolio to include an annual wine guide. He was able to convince the majority of the tasters that he had been working with for quite some time to stay with him (13 out of 18) and made it clear that he saw the 2018 Vinum wine guide as an update of the 2017 Gault Millau and in a row with its previous 24 editions. The 2018 edition of Gault Millau, in turn, was put together by a newly established team of tasters led by editor-in-chief Britta Wiegelmann, who was, ironically, previously the editor of Vinum wine journal. Only two of Payne's former collaborators joined forces with Wiegelmann and stayed with Gault Millau.

Since Diel and Payne have often been considered “Siamese twins,” I do not expect to find an impact of Diel's departure on subsequent winery ratings in Gault Millau (Hypothesis 1). However, since Payne took most of the tasters that he had already been working with at Gault Millau with him to Vinum, I expect to find a stronger effect of the 2017 Gault Millau winery ratings on the 2018 ratings of these wineries in the first edition of Vinum than on the ratings in the 2018 edition of Gault Millau (Hypothesis 2). Moreover, since Payne had to replace some of his tasters, I can distinguish between teams whose composition has remained unchanged and teams with one or two new tasters (each of the 13 wine-growing regions in Germany is usually covered by 2 to 3 different experts). Since the two tasters that stayed with Gault Millau have been and continue to be in charge of the same region (Baden) under the new editor, it is possible to make the same distinction for the latter guide too. It is plausible to assume that the rating behavior of newly formed teams is more restrictive/less generous than the behavior of experienced teams (Hypothesis 3).

III. Data

The data sets I use to test my hypotheses have been compiled from the 2008 to 2010 and the 2014 to 2018 editions of Gault Millau as well as the 2018 edition of Vinum.⁶ Each of the two guides uses a similar rating system to distinguish among five different levels of reputation (“grapes” in Gault Millau and “stars” in Vinum):

1. 1 producer of consistent quality (around 30% of all listed wineries)

2. 2 producer of high quality (around 25%)

3. 3 producer of very high quality (around 20%)

4. 4 producer belonging to national elite (around 8%)

5. 5 producer with worldwide recognition and appraisal (about 2%).

In each of these five categories, the best producers are awarded another “half star/grape,” indicating that they are already close to the next higher reputation level (about 15% of all listed wineries). In addition, half a star/grape can be awarded to promising newcomers, whose future performance is carefully monitored by the respective wine guides’ experts. It appears that the distribution of the reputation scores is virtually identical in the two guides. The mean (standard deviation) is 2.05 (1.17) in the 2018 Gault Millau and 1.96 (1.15) in the 2018 Vinum guide (the full table is available from the author upon request).

IV. Results

With respect to Hypothesis 1, it appears that the correlation of the ratings in 2008 and 2009 (in both years Diel and Payne were still on board) is high at 0.966 (n = 824 wineries). This value is virtually identical for the years 2009 and 2010 (in the latter year, Payne was the sole editor) with 0.965 (n = 853 wineries). Finally, for the years 2008 and 2010, the respective correlation coefficient is 0.923 (n = 789 wineries), suggesting that the departure of Armin Diel had virtually no impact on the individual raters’ behavior. This is supported by the correlations for the years 2014 and 2015 (0.974, n = 947 wineries), 2015 and 2016 (0.962, n = 946 wineries), and 2014 and 2016 (0.938, n = 908 wineries) that are comparable in magnitude (in these latter years no change in editorship occurred).

With respect to Hypothesis 2, it appears from Tables 1 and 2, that as expected, the impact of the Gault Millau score 2017 is (slightly) stronger on the Vinum 2018 than on the Gault Millau 2018 score (the team that put together the 2017 Gault Millau was nearly identical with the one that put together the 2018 Vinum guide).⁷

Table 1 The Impact of Gault Millau Scores 2017 and Rater Experience on Gault Millau Scores 2018

Notes: Standard errors in parentheses.

*p < 0.10, ** p < 0.05, *** p < 0.01

Wineries listed in one edition only have been assigned a value of zero for the other edition in sample 1 (Models 1.1 and 1.2), while sample 2 (Models 1.3. and 1.4) includes only wineries listed in both editions in the relevant year.

Table 2 The Impact of Gault Millau Scores 2017 and Rater Experience on Vinum Scores 2018

Notes: Standard errors in parentheses.

*p < 0.10, ** p < 0.05, *** p < 0.01

Wineries listed in one guide only have been assigned a value of zero for the other guide in sample 1 (Models 2.1 and 2.2), while sample 2 (Models 2.3. and 2.4) includes only wineries listed in both guides in the relevant year.

Moreover, with respect to Hypothesis 3, it appears that in both cases (Gault Millau as well as Vinum), “old” teams (those whose composition remained unchanged) tend to rate more generously, while newly formed teams tend to rate more restrictively, that is, less favorably. Since I observe the same people in different environments, the associations I identify here are not mere correlations, but causal effects.

V. Summary and Implications

The findings reported previously suggest, first, a high degree of consensus among the two leading German wine guides. Although the “overlap,” that is, the percentage of wineries included in both guides, is only 83%, the correlation between the guides’ ratings is (perhaps surprisingly) high.⁸ The reason is that wineries not included in either one of the two guides score rather low in the other one. Second, “legacy matters” in the sense that winery evaluations in the 2018 edition of Vinum display a closer correlation with the evaluations in the 2017 edition of Gault Millau than the Gault Millau evaluations published in 2017 and 2018 (the latter edition of Gault Millau was compiled by a new team of editors and tasters).⁹ Moreover, while experienced teams (i.e., raters who have already been working with the same editor for an extended period of time) seem to evaluate prestigious estates more favorably, less experienced teams (i.e., those working under a new editor) try to demonstrate their expertise by providing lower ratings.

Thus, changes in the evaluation of a particular winery may not always be due to changes in product quality as perceived by experts but can also be due to changes in the position of the chief editor or the composition of the team of experts responsible for a particular region. Consumers are, therefore, well-advised to consider these changes before making their purchasing decision.