Tag Archives: objectivity

Why Don’t We Hear about Blind Tasting Success?

12 Thursday Oct 2017

Posted by in Wine Culture

≈ Leave a comment

Tags

,

blind tasting scoresWhenever wine tasters fall flat on their face the press celebrates by publicizing the fact far and wide claiming it offers proof that wine tasting is nonsense. So why didn’t we hear more about this rigorous empirical study that clearly shows trained wine tasters can successfully identify varietal and region of origin?

Two teams of seven tasters each (including one reserve per side) were presented with 12 wines, six whites and six reds….The group’s overall accuracy was far superior to what could be expected from random chance. Given the thousands of potential country-variety pairs, a monkey throwing darts would have virtually no hope of getting a single one right. But 47% of the Oxbridge tasters′ guesses on grape variety were correct, as were 37% on country of origin.

The competitors′ performances on each individual glass rarely matched these headline averages. Some wines were well-nigh unmistakable: all 14 drinkers identified the Pinot Noir, 12 called the Chardonnay and Gamay right and 11 identified the Sauvignon Blanc and Cabernet Sauvignon. Similarly, 13 participants recognised that the Gamay was from France (it is scarcely grown anywhere else), and nine said that the Semillon was Australian (though only four also determined that it was a Semillon)

At the other extreme, no one knew what the Friulano was made from—an unsurprising result, since the grape is little-known internationally. Only one drinker nailed the Rioja (made from Tempranillo) and Châteauneuf-du-Pape (a Grenache-based blend), wines from prominent regions that should have been relatively easy to spot.

Similarly, the averages obscured wide differences in performance among individual drinkers….

The failure to identify Tempranillo is surprising but otherwise the results are what you would expect. Blind tasting is challenging but nevertheless employs real expertise.

This study was published earlier this year but it received very little coverage or discussion. I guess successful blind tasting doesn’t flatter the audience that they know more than the experts.

Advertisements

Wine Critics and Their Biases

22 Tuesday Apr 2014

Posted by in Art and Wine, Philosophy of Food and Wine, Uncategorized, Wine Culture

≈ Leave a comment

Tags

,

Wine writer Jamie Goode raises a key question about wine criticism—should critics allow personal  style preferences to influence their judgment?

He thinks style preferences inevitably color a critic’s judgment:

Admirable as this sentiment is, I don’t think this can work in practice. At some level, a critic will have to make a call on style, because some wines force you into this. In practice, even critics who profess to leave their personal style preferences to one side when they assess wine, can’t seem to do this in practice.

Why? Because of balance.

Balance is important in wine, and it’s a style call. This makes it quite personal.

He grants that what he calls ”spoofulated wines”—overly-ripe, high alcohol, heavily oaked wines—taste out of balance to him and thus are inherently deserving of low scores; and such differences in preferences explain widely divergent scores among critics.

Thus he concludes “It’s a myth to think that there is some objective measure of wine quality that professional critics can tap into.”

Truth be told, I don’t think a wine critic’s job is to be objective. It is to discover what is beautiful, different, or remarkable, to make readers think about their own preferences, and lead them to discover something they might not otherwise have noticed. To do that, one needs an aesthetic perspective, a distinct point of view, not objectivity.

But it is important that critics avoid certain kinds of biases regarding price, reputation, marketing, or personal connections, and if a critic’s preferences are too idiosyncratic her reviews might not be generally useful so it is an interesting question whether objectivity, when appropriate, can be achieved.

I think Jamie is right at least about the relationship between balance, style, and the role of personal preference. Balance is a function of the relative prominence of the various components of a wine—fruit, acidity, tannin, oak flavors—and the extent to which they seem integrated and unified.  Decisions about how to adjust this balance account for at least some stylistic differences among winemakers as well as differences among consumers and their preferences.

Because balance is an inherently relational concept, an assessment of how a number of independent variables in a particular case are related, there can be no objective measure of it. In other words, the balance point of each wine will depend on its unique characteristics and thus there can be no general rule, scale, or metric that determines balance.

But balance isn’t all there is to wine quality. Complexity, intensity, length of finish,  how much flavor there is on the finish as opposed to merely tactile sensations, and liveliness on the palate are also elements of wine quality. Unlike balance these are more one-dimensional, and at least roughly measurable because they are primarily about magnitude and thus can be represented on a scale. I would hesitate to say these are purely objective properties—differences in physiology and background will produce disagreement about them among experienced critics. But the fact that a single magnitude is being assessed reduces the scope of disagreement and increases the possibility that, through training, diverse tasters can calibrate their judgments to each other.

I might not prefer “spoofulated” wines, but I can nevertheless judge their complexity, concentration, length of finish etc. assuming my bias doesn’t prevent me from attending to these features.

But this raises another more far-reaching issue. The judgments of wine critics may be initially subject to all sorts of biases. But can an attentive, self-aware critic overcome these biases? If a critic recognizes that they prefer certain wine styles to others, and is able to analyze the way such biases influence her judgments, can she overcome the bias?

I’m not sure what the answer to that question is.

There has been a good deal of research on cognitive biases. The work of Daniel Kahneman especially has demonstrated that our thinking is subject to all sorts of unconscious biases that seriously distort our thinking.

For instance, take what Kahneman calls the “planning fallacy”—our tendency to overestimate benefits and underestimate costs, and hence foolishly to take on risky projects.  Yet, clearly, if we know about this bias and discipline ourselves to stick with the hard evidence and have the appropriate skepticism with regard to unsubstantiated predictions, we can overcome this bias.

Are taste biases similar?

With regard to criteria such as overall hedonic quality, on the basis of which point scores are awarded, it is hard to see how biases can be avoided. If your judgment is simply a matter of identifying how much pleasure you get from a wine, then personal preference inevitably enters the picture. You can’t pretend to enjoy what you don’t enjoy or gauge how much you would enjoy a wine if only you had enjoyed it. But there is more to wine criticism than measuring overall hedonic quality.

It is common for art, film, and music critics to give positive reviews to works they didn’t particularly enjoy—perhaps the theme makes them uncomfortable, the film or score was too long, the meaning opaque. But the work might nevertheless be complex, moving, interesting, and have integrity. Pleasure is not the only criterion for judging anything and thus personal preference need not govern all our aesthetic judgments.

I see no reason why wine criticism should be different.

Do We Live in Different “Taste Worlds”?

06 Tuesday Aug 2013

Posted by in Contemporary Food Culture, Philosophy of Food and Wine

≈ 1 Comment

Tags

,

taste-in-restaurants Are tastes really subjective? Obviously people have different food preferences. Two people can eat the same food—one will love it, another detest it. According to conventional wisdom, there is no right or wrong on the matter, just blameless disagreement.

Moreover, the science on taste and aroma shows that individuals differ in their ability to detect flavor compounds in food. “Supertasters” are more sensitive to bitter compounds than the general population, some people require fewer sugar molecules in solution to detect sweetness than others, and about 15% of the population thinks cilantro is vile, probably because they can’t detect the molecules responsible for cilantro’s fresh, floral aroma. Thus, even scientists are claiming that we live in different “taste worlds” and that each person has a unique palate shared by no one else.

But this idea of different “taste worlds” is misleading, and differences in preferences should not lead us to conclude that taste is thoroughly subjective.

The idea of different “taste worlds” suggests that the qualities of another person’s taste experiences are utterly inaccessible to me. But the fact we have different thresholds for tasting substances like sugar or other compounds doesn’t entail that taste is thoroughly subjective. If I detect sucrose in a 1 part in 200 solution and you detect it in a 1 part in 198 solution, there will be a substantial difference in our “taste worlds” only for foods with sugar content that falls within that very narrow range. Chocolate ice cream will taste sweet to both of us. Unless someone suffers from a pathology or has a genetic incapacity to detect a particular substance, small differences in detection thresholds will matter only on the margins. In other words, there will be a large overlap in our ability to taste sweetness. That does not create different worlds even when aggregated for the vast number of compounds human beings can taste and smell. Rather, we live in a shared world of tastes about which we have a slightly different perspective in a small number of cases. It isn’t as if lots of people are baffled by the phrase “sweet as honey.”

Taste preferences of course are another matter. We differ in what we like and have vastly different histories of taste experience. But “different worlds” is still not the right way to characterize those differences. Our tastes are continually changing. We outgrow the milkshakes and candy of youth and replace them with kale and quinoa. We know from experience that we can learn to like new foods and new flavors; and in some cases we can learn to overcome aversions and disgust if we work at it. Furthermore, there is substantial evidence that our tastes are influenced by the place setting on which our food sits, the physical, social, and musical environment in which we eat it, the beliefs we have about it, and the opinions others offer about it. Our tastes are subject to influence in very surprising ways. The faddishness of our foodways makes that obvious.

Thus, our taste preferences do not create independent worlds but instead form a complex, overlapping manifold of continually shifting, malleable connections. Yes, we disagree about taste but that disagreement is not stable, and neither your perspective nor my perspective is inaccessible to mutual influence and the possibility of inter-subjective agreement.

Granted our tastes are internal. I cannot have your experience nor you mine. And I can never know with certainty and precision what your taste experiences are like.  But that does not mean there is no common world of flavor that we are striving to grasp or that disputes about taste are meaningless, futile and not subject to norms and standards.

So let’s stop talking about different taste worlds. Celebrate our differences but with the recognition that they are fleeting, partial, and subject to influence.

Wine Talk is About Communication, Not Objectivity or Accuracy

02 Tuesday Jul 2013

Posted by in General Aesthetics, Philosophy of Food and Wine, Wine Culture

≈ 6 Comments

Tags

, , , , , ,

wine-descriptions-for-new-world-barbera-wine When wine is mentioned in the news (outside the columns of regularly-featured wine writers) the topic is all too often the unreliability of wine reviews. Wine critics are accused of being influenced by price and are notorious for their inconsistent judgments and cavernous disagreements about the same wine, as in this recent assessment. The implication is that wine is wholly subjective and the very idea of wine expertise should be discarded.

I’ve discussed the objectivity of wine tasting in the past (here, here and here). Some of the criticisms of wine tasting are misleading; others are simply misguided. As wine critic Steve Heimoff argued recently, the fact that there is a subjective dimension to wine tasting does not mean that wine criticism is useless. What we want from the critic is his/her personal opinion, not a scientific analysis.

Why don’t people get so upset about restaurant critics or movie critics? You’ll never see an article headlined RESTAURANT REVIEWS ARE JUNK SCIENCE. That’s because restaurant reviewers don’t pretend to be offering anything but their opinion.

Well, neither do wine critics.

Steve is exactly right. But I want to take his point one step further and suggest that, in many contexts, values such as accuracy of description or consistency of judgment are not primary—wine critics are aiming at something else.

The ultimate point of wine reviews, especially the descriptive part of the review, is not to accurately describe the wine. If it were, wine criticism would be an abject failure since critical descriptions of the same wine by different critics can be worlds apart. Instead, what the wine critic is trying to do is get the reader/drinker to taste what the critic is tasting. The critic is using words to call attention to features of the wine that another taster—perhaps less experienced or inattentive—might miss. The purpose is to direct the the reader/taster’s attention, to share an experience.

This conception of critical communication is from an academic paper written many years ago by philosopher Arnold Isenberg who was writing about criticism in the visual arts. In discussing critical remarks about the “wave-like contour” of a line in a painting, Isenberg writes:

Now the critic, besides imparting to us the idea of a wavelike contour, gives us direction for perceiving, and does this by means of the idea he imparts to us, which narrows down the field of possible visual orientations and guides us in discrimination of details, the organization of parts, the grouping of discrete objects into patterns. It is as if we found both an oyster and pearl when we had been looking for a seashell because we had been told it was valuable. It is valuable, but not because it is a seashell….And if communication is a process by which a mental content is transmitted by symbols from one person to another, then we can say that it is a function of criticism to bring about communication at the level of the senses; that is, to induce a sameness of vision, of experienced content. (Original in Philosophical Review, 1949)

Wine criticism has a similar intent. When a wine critic refers to for example “road tar” in a wine, he/she is pointing to a feature that may or may not resemble road tar. But the description is directing our attention to something the critic wants us to taste and is proposing that we call it road tar. Of course, the more precise the description, the more successful the directive is likely to be. The point is not that critics should ignore accuracy. The point is that the reader/drinker is moved to search for and hopefully find something she may have missed, or could not describe, prior to reading the description. The aim is not objectivity, and accuracy is just a means to an end. The goal is to communicate an experience.

Metaphorical descriptions of wine—as sexy, brooding, or flamboyant—or evaluative terms, such as elegant or graceful, are even more useful in getting an audience to focus on relevant features. The fact that another critic might use a different word to direct our attention is largely irrelevant. Since each of us has different tasting histories and different habits of talking about those experiences, which description works best to direct attention will be a matter of individual differences.

If, by pointing to the flamboyant personality of a wine, I get someone else to describe it as lively, I’ve done my job. We have a common referent despite the different meaning. Of course, if she describes that same wine as mute and closed, there is obviously egregious miscommunication.

So critics are not striving for objectivity. We are instead striving for awareness, mutually focused attention, and shared experience.

Wine scores, of course, are a different matter. They don’t attempt to shape our thoughts or experience. Their purpose is to help consumers find a bargain, gain advanced knowledge of quality, or provide marketing points for wineries—a purely commercial rather than aesthetic function.

To be sure, wine scores are useful to consumers but if wine criticism were to reject the tasting note in favor of an exclusive reliance on numerical ranking, the purpose of critical communication would be lost.

Subjective Tastes and Mixed Messages: Is the Wine Industry Harming its Image?

19 Tuesday Feb 2013

Posted by in Philosophy of Food and Wine, Wine Culture

≈ 8 Comments

Tags

, , , ,

wine class This post from Steve Heimoff got me thinking about how the wine industry steps on its message when writers and salespersons insist that wine tasting is subjective.

Heimhoff was wondering whether the market for high-end Napa wines is collapsing.

Yet there has to be a tipping point at which Napa no longer can sustain so many overpriced wines. I don’t expect Napa itself to understand where the tipping point is, or will even recognize it when it comes: Napa is very, well, Napa-centric, as perhaps it should be; but it does tend to see the world from within its own rarified bubble.

There is some anecdotal evidence from retailers that the tipping point has already occurred, which is not surprising given the increasingly competitive global wine market.

But if the wine industry wants people to pay premium prices for their best product, why do they insist on telling people that wine tasting is subjective? I don’t think I’ve ever been to a wine tasting in which presenters don’t encourage patrons to trust their palates, insisting that there is no right and wrong when tasting wine and that the views of a novice are every bit as valid as the views of an expert.

I understand why they do this. Wine can be intimidating. It is a complex field and the factors that contribute to quality are sometimes hard to detect. If it seems too time consuming or difficult to gain knowledge, people may conclude that it is not worth the effort.

But nothing is more likely to discourage people from adding to their wine knowledge and experience than the claim that wine tasting is thoroughly subjective. Because if wine tasting is subjective these claims follow logically:

(1)There is nothing more to a wine than what I can immediately sense.

(2) Sensations provide us with no basis for drawing conclusions about the world.

(3) Therefore, there is nothing in the wine that my sensory experience must detect, and thus no contrast between how things seem to me and how they really are.

(4) Each person’s sensory response is utterly unique to her and there is no reason to think that when I taste something, someone else ought to taste it as well since there is nothing out there to be tasted.

(5) Therefore, there are no standards for evaluating wine quality

(6)When winemakers taste wine in order to make decisions about wine quality, they are engaged in an empty, pointless exercise.

If you believe these claims, then why on earth would you spend $20 let alone $100+ for something that cannot promise greater quality. Claims 1-6 entail that if you happen to like the $100 bottle more than your Trader Joe’s bargain wine it is just an accident unrelated to wine quality.

Of course, none of the wine industry spokespersons really believe any of this—it is utterly incompatible with wine scores, wine criticism, tasting notes or the commitment to wine quality that drives the best winemakers to succeed. If there is no such thing as wine quality, why do oenologists spend tens of thousands of dollars on university degrees so they can learn to make good wine?

I can’t help but think the pervasiveness of this meme that wine tasting is subjective is slowing the growth of wine appreciation since it encourages a kind of lazy complacency with whatever swill one happens to drink today. If Napa producers are losing market share, perhaps they have only themselves to blame.

Inflated Wine Scores and the Longing for a Wine Aristocracy

15 Tuesday Jan 2013

Posted by in Art and Wine, Philosophy of Food and Wine, Wine Culture

≈ Leave a comment

Tags

, , , , ,

queen elizabethe The latest brouhaha about wine scores erupted shortly after Christmas when The Wine Advocate, arguably the most influential wine review publication, released its latest assessment of high-end juice.

Using the Advocate’s 100-point system, Antonio Gallioni, the WA’s California expert, awarded 95 points or above to 223 California wines, about 1/4 of the wines receiving published reviews. (The reviews for low scoring wines are not released). Gallioni’s boss Robert Parker gave 100 point scores to 17 wines from the Northern Rhone’s 2009-2011 vintages, which follows on the heels of the 19 100-point scores he gave to the 2009 Bordeaux vintage. I haven’t kept track but Mike Steinberger reports that Parker has given out at least 53 100-point scores since March.

Apparently, grade inflation has migrated from Harvard to Napa.

Many commentators cried foul arguing that this grade inflation made wine scores meaningless. Parker took to his blog to defend his journal’s scoring claiming that wines are simply getting better and the high scores reflect this general improvement.

Mike Steinberger had the most thoughtful critique.

But I think one reason Parker and Galloni have encountered so much skepticism is that if you accept the 100-point scale as a quasi-objective means of assessing wines—and it seems to me that if you buy into the 100-point thing, you are necessarily accepting the idea that it is a quasi-objective standard—then the sheer number of wines clustered at the top of the scale simply isn’t credible.

But I’m not sure I follow Steinberger’s reasoning.

If Parker is right that winemaking has improved since the great but rare Bordeaux vintages of the past, such as the 1947 Chevel Blanc or the 1961 Latour, then clustering at the top of the rankings is precisely what we should expect. Furthermore, Parker’s claim is plausible. Improvements in wine-making technology, advances in the science of winemaking, and the increasingly competitive nature of global wine markets have all contributed to a glut of good wines. There is no reason to think this doesn’t effect the very top of the scale.

Nevertheless Steinberger thinks the scores are excessive:

But even if the overall quality of wines is better, it doesn’t follow that so many wines should be receiving eye-popping scores. If the competition is much tougher now than it was 10 years ago, it shouldn’t be easier to get 96 or 97 points; it should be harder…. Forgive the tautology, but if the bar has been raised, you need to raise the bar. You can do that one of two ways: by lengthening the scale—making the highest score, say, 110 points rather than 100—or by tightening the standards within the 100-point framework to reflect the fact that the quality is so vastly improved. If you don’t do either of those things, you end up in a situation like the one that Parker and Galloni are now confronting—with your reviews being greeted mainly with cynicism and derision.

But why would such “grading on a curve” be more objective? If wine quality really has improved, objectivity would demand that improvement be reflected in the number of highly rated wines, if by “objectivity” we mean something like “tracking the truth”.

It seems to me what Steinberger and other critics are worried about is not objectivity or the erosion of standards. Rather they are worried about the loss of the aura of scarcity surrounding high scoring wines. When perfection becomes the norm it is no longer valuable as a measure of greatness or prestige. Apparently wine still requires an aristocracy.

Steinberger proposes to correct this by lengthening the upper range of possible points or tightening standards within the 100-point limit. But neither solution will enhance the credibility of the system.

If the 100-point system has a flaw it is that point gradations assume more precision than the practice of wine tasting allows. Is there an explainable, projectable difference between a 99 and 100-point wine that can be extended over time so that every time the 100 point wine is tasted it is one point better than the 99? I doubt it. Yet, both of Steinberger’s solutions presuppose an ability to make even more fine-grained judgments about wine quality because either the scorer has more points for which criteria must be found or she must find reason to exclude wines that are otherwise worthy in order to tighten standards. But it is precisely the absence of criteria on which to base scores that is the problem; it can’t be solved by demanding more criteria.

Wine scores have a particular but limited meaning. They measure an expert’s level of enthusiasm for a wine at a particular time and place. This is useful information for a wine buyer to have. But the scores do not precisely track fine gradations of wine quality.

This problem with precision is especially acute with the 100-point wines under discussion. Wines which are that impressive are not going to have obvious flaws (at least when their future performance is taken into account) and will not fail according to the standard criteria used to judge wine. Unlike 90-point wines, 100 point wines will have plenty of balance, structure, power, intensity, elegance, finesse, body, complexity, etc. (at least when their performance is projected into the future) Thus, judgments at this level are bound to be searching for criteria but without a fixed standard to rely on and nothing to which a numerical value can be attached.

An analogy with art criticism might be helpful. Contrary to popular opinion, it is possible to grade art. Art professors do it routinely. This is because much student art will satisfy or fail to satisfy standard criteria used to evaluate art and numerical values can be assigned to those criteria. Such grading is not “perfectly objective” whatever that would mean, but the grades nevertheless have meaning if the grading standards are clear.

But it does not follow from this that we can usefully assign numerical grades to the work of Monet, Rembrandt, or Picasso. At this level the ordinary criteria we use to assign a grade to works of art don’t apply. The works of the masters are not lacking in those dimensions that afflict student art. Instead, what we get is a unique vision, something original and incomparable that cannot be captured by a set of standards or criteria. The assignment of a numerical grade would be pointless.

Similarly, for wines that fall into the 95+ range, I doubt there is much point in assigning numerical value. If they lack originality or consummate expressiveness they don’t belong there. If they have it, they cannot be readily compared.

If inferior wines are gaining admittance to the pantheon of legends, we have reason to be skeptical of the 100-point system. But if these wines are indeed worthy when compared to the best from the past, the lack of a scoring system that makes precise qualitative distinctions is not a worry. It is what we should expect from an expanded pantheon of legends. Grade inflation is only grade inflation if the higher grades are undeserved.

This of course makes the 100 point system useless for comparing the best wines, which will make neither Parker nor the investors who place great stock in his scoring system happy.

And if great wines are less rare than they were in the past, isn’t that a cause for celebration? Or is this democracy still pining for aristocracy?