I’ve found crowdsourced review sites like Yelp to be utterly unreliable for anything but the most basic information about a restaurant. A new study suggests that a number of biases influence people who write such reviews.
The size and racial makeup of a city, the price of a meal and even the weather can skew the quality and quantity of online restaurant reviews, according to the first large-scale academic study to analyze how outside factors affect crowd-sourced review sites.
The study, which will be released Wednesday, used computer models to examine nearly 1.1 million reviews of 840,000 restaurants over nearly a decade.
Among the findings:
— Reviews written in the summer were more likely to be negative
— Reviews written when temperatures were below 40 degrees or above 100 degrees or when snowing or raining were more likely to be negative.
— Restaurants in the Northeast and on the West Coast were reviewed more than those in the South or the Midwest.
— People living in urban areas were more likely to write reviews
— Diners in large cities were more forgiving about waiting times that those in smaller communities
— Sushi restaurants were given higher reviews than burger restaurants suggesting ambiance and price influenced the results.
One limitation of the study is that Yelp reviews were not included in the results.
Many of these results suggest that a person’s mood influences their judgment–not a surprising result at all.
But when this evidence is combined with an earlier study that suggests crowdsourced reviewers are influenced by what other reviewers have already written, especially when the reviews are positive, there is reason to doubt the reliability of crowdsourced review sites.
There is still no substitute for a well-trained, experienced professional critic. Yes, they are subject to biases like anyone else, but professional reviewers, whose livelihood depends on their reliability, are more likely to be aware of that and make a conscious effort not to let biases related to mood or peer pressure influence them.