The trustworthiness of Yelp, Angie’s List, and other “unbiased” ratings services is called into question in a new study by Consumer Reports. And the consumer advocacy publication is hardly the only one taking issue with user-review sites lately.
By now, it’s universally accepted that online reviews can have a big impact on the success (or failure) of a business. According to one 2013 survey, 9 in 10 consumers said their buying decisions have been influenced by online reviews. A much-cited Harvard Business School study indicated that a one-star rating increase for a restaurant listed on Yelp translated to a bump in revenues of 5% to 9%.
Certainly, consumers rely on online reviews to help them make decisions about where to spend their money. But to what degree should we trust these reviews?
A new study by Consumer Reports tries to answer that question, taking a close look at a half-dozen online ratings and review services, including well-known sites like Yelp and Angie’s List, as well as smaller players such as Porch. The gist calls into question the validity and trustworthiness of all of these user-review sites. “Unsettling surprises came fast and frequently,” the report states.
Angie’s List, which only allows its two million paying subscribers to see (and write) local business reviews, is criticized due to its practice of allowing businesses with a B rating or better to pay to get their listing placed at the top of search results. “We think that the ability of A- and B-rated companies to buy their way to the top of the default search results skews the results,” CR researchers write. “Angie’s List misleads consumers by prominently promising that “businesses don’t pay” and that it’s a consumer-driven service supported by membership fees. But almost 70 percent of the company’s revenues come from advertising purchased by the service providers being rated.”
Google+ Local’s ratings are called into question because businesses can reach out to customers who’ve posted bad reviews and beg, bribe, make amends, or otherwise convince the aggrieved to change or delete the original opinion. “This can skew the ratings positively,” CR states “because assuaged customers can always delete their previously negative reviews.”
Then there’s Yelp, the granddaddy of user-review services that boasts 42 million reviews of local businesses collected over the years, and 108 million monthly unique visitors just in the first quarter of 2013. “Some data reveal that user reviews skew toward the positive,” CR says of Yelp. “Researchers at the University of Texas at Dallas and California State University, Long Beach, found that people who rented movies online were five times more likely to post a positive review than a negative one. And 66 percent of Yelp’s reviews for the first quarter of 2013 were four or five stars.”
While such data may be worrisome from the perspective of consumers seeking the unabashed, brutally honest truth, many local business owners have been complaining about Yelp for the opposite reason: They say Yelp’s filter system, which sanctions reviews deemed bogus or untrustworthy to a section away from the main results, sometimes winds up filtering out good reviews while keeping fraudulent one-star ratings.
Over the summer, Yelp hosted town hall-style meetings with business owners, some of whom have been frustrated that what they claim are authentic customers reviews were filtered out, “leading to suspicions that they were being withheld because the owners didn’t advertise with Yelp,” as the Los Angeles Times put it.
“They told me if I want to have the good reviews on the top when you open the Yelp page,” one restaurant owner, frustrated with his business’s middling three-star rating on Yelp, said at a town hall meeting in Sacramento. “I’ve got to advertise it.”
It’s this perception of Yelp that has led to Yelp-bashing sites like Yelp-Sucks.com and this past summer’s Yelp manifesto on Buzzfeed, which explored whether the company is a blackmailing bully or merely understood.
Yelp says it’s most definitely the latter. The company never reveals exactly how the filter works because such information could be used by interested parties trying to game the system. (Lots try to do so, as seen in the user-review scandal unearthed in July, in which spammers were busted creating thousands of fake anonymous accounts in order to boost car dealership ratings.) Yelp maintains filtering occurs automatically based on objective data, and it’s never tied to advertising. “The first layer of this filter is trying to suppress any spammy, shill, malicious reviews that it can identify,” Yelp’s Morgan Remmers explained at the town meeting in Los Angeles. “The second layer that it’s evaluating is the user’s engagement with our site. So we’re showcasing the most helpful, reliable reviews.”
(MORE: Fact-Checking the Crowds)
Nonetheless, there are those who give Yelp and the “Yelpers” writing reviews exceptionally poor ratings. The most prominent is the Travel Channel’s Andrew Zimmern, who declared last year, “Yelp essentially gives a tremendous forum for a bunch of uninformed morons to take down restaurants.”
The fact that some people are so up in arms about Yelp and its business and review practices is itself a sign that the site is extremely powerful and influential. No one’s feathers would be ruffled if the site’s reviews didn’t have a big impact in the marketplace.
If the entrepreneur/angel investor/networking guru Peter Shankman is right, however, Yelp won’t be influential for long. Over the summer, Shankman posited that Yelp will be out of business within 24 months, because by then, people will realize they shouldn’t trust anonymous reviews from people they don’t know. He even put $5,000 on the line, saying he’d give that amount to charity if Yelp was still in business by August 2015. “We don’t rely on strangers to tell us what to do anymore,” he wrote on his blog, pointing out that increasingly, we can gather opinions from Facebook friends and others in our network—those who we know are real people, and have some reason to trust. “A business model that allows anyone to review, at any time, with no structure or background is a flawed model. There’s no reason for me to trust anyone I don’t know. It’s the INTERNET, for the love of God. Why should I?”
The easy answer is that while only a fool would trust the opinion of a single stranger, millions of people are apt to trust the collective opinion of hundreds, or thousands, of strangers. Crowdsourcing has quickly become second nature, helping the masses use the masses to help decide which hotel to book, which movie to rent, and which restaurant to hit for dinner. Will this always be so? Probably. But if a smarter, more trustworthy form of crowdsourcing becomes readily and widely available, it would be easy to imagine consumers filtering out the opinions of total strangers.