Parsing Online Reviews

Online customer reviews are proliferating like mad.  Nearly every retailer includes these on their sites, and there are a plethora of independent sites proffering customer reviews.  Are these reviews worth anything?

 
Online review sites have sprung up all over the place in recent years.   Many of them are put-up jobs - fake review sites set up by people with an agenda, to either steer you to a product or service provider, or to support outright fraud.  Still others are "legitimate" sites, but are spammed by groomers who post fake postings to "groom" the image of a merchant.  And yet other sites are little more than shakedowns, that encourage merchants to "join" the site in order to ameliorate negative reviews.

I wrote before about Angie's List, which makes you pay to join.  Their argument is that reviewers who pay are more likely to be accurate and less prone to SPAMMING than unpaid ones.  But others argue that the people being reviewed can simply have 5 or 10 friends log on and post laudatory reviews of a company - and you have no way of knowing.

I am not sure it is worthwhile PAYING for any online review site.    There does not seem to be any mechanism that would make pay-to-access reviews any better than "free" ones.  The only way to make a review worthwhile is when you know the person involved, and since most reviews are anonymously posted (or pseudo-anonymously posted) you have no way of knowing.

Review sites can be useful, but I would take them with a definite grain of salt.  But you can sort of parse out what is going on, if you read between the lines:

Read the Worst First:  Glowing reviews teach us nothing, and many, if not most of them are put-up jobs by online companies that are hired to groom a company's reputation.  Yes, such companies exist - they even advertise their services!   You learn more from a bad review than you do from a good one.  But just because something has a bad review, doesn't mean it is bad.

Read Between the Lines:  We all have bad days, and bad reviews may reflect this.  Someone complains about a popular restaurant because they had to wait 15 minutes for a table.  Is this a valid complaint, or just being bitchy?  And the taste and sophistication of the reviewer is important as well.  When someone gives one star to a restaurant because "they don't have no lite beer!" that is not really very relevant.

Ignore Cheer Leading Posts:  It is not hard to spot the grooming posts and put-ups, as they often have nothing negative to say at all - and no experience is that great, right?  Also, a grooming post often spends more time criticizing the other posts than it does reviewing the product or service in question.  "I don't know what the negative reviews are all about!" one chipper lady claims, "Because everything for us was super-duper!"

Beware of Competitor's Negative Postings:  Just as companies will SPAM review sites with fake positive reviews, some will put negative reviews about their competitor's products on review sites.   This has not been a big trend, just yet, as if caught, they could face libel charges and perhaps tort liability.  But vague negative postings might be a tip-off that someone is spamming the review site.

The Guy Who Is Never Happy:  You've seen them at the car dealer, painting "LEMON" on the side of their car and parking it out front.  The guy who is never happy.  And what he is unhappy about is the price he paid for the load he bought.  It is buyer's remorse, plain and simple.  These folks always have scorchingly bad negative reviews.  No amount of refund, rebate, service or discount is good enough for them.  They won't be satisfied until everyone on the dealership is impaled on a stake and roasted on a spit.   A nice revenge fantasy, but hardly realistic.

Look for Specifics:  When you see a review with specific data and specific comments, it is helpful.  But blandishments like "the food was good!" are not really helpful.  "The spicy chicken entree was delicious and spicy without being overpowering" is a little more helpful, and tells you more about the nature of the product.  Specifics also tell you that you might be dealing with a real consumer, and not some guy hired by a grooming company.

Look for Repetition:  When a number of people start to say the same things, over and over again, it is an indication that something is up.  For example, on one sneaker site I visited recently, most reviews of a particular sneaker were positive.  But a number were negative and all said the same thing - the shoes tended to run small and narrow, and that it would be a good idea to order a half-size larger.  One complaint like this might be an anomaly.  When you see six, perhaps it is a pattern.

How They Handle Complaints:  People who have a good experience with a product do not get to evaluate their customer service.  And in many cases, this is where things fall apart.  Maybe 90% of the people who bought a product or service were satisfied.  But the 10% who weren't consistently reported that they were told to "piss off" by customer service.  This tells you a lot.  If you are happy with their product or service, great.  If not, boy are you in trouble!  On the flip side, a comment from someone who HAD a problem and how it was quickly resolved, speaks volumes.  That is a company you want to do business with.

This isn't High School:  In school, if you got a 90% that was an "A" and an 80% was a "B" and a 70% was a "C" and so on.  With online reviews, anything less than 90% is pretty suspect - but it depends a lot on the review site.  For example, on eBay, if you see a person with a 70% feedback score, this is not a good thing!  On the other hand, it depends on how many things the seller sells.  Anything less than 100% is problematic, unless it is a "power seller" or there is one negative review which is clearly unreasonable on its face.  But even then, anything below 95% on eBay is trouble.  Outside of eBay, the standard may be less harsh.  But needless to say, when you see dozens of negative reviews about a product or service, it should give you pause, particularly when they mention similar things.

* * * 

The basic problem with review sites is that reviews from people you do not know personally are often of little value.  Who is making the referrals or review is as important as the review itself.   A gourmet chef might have better insight into the quality of a restaurant than some bum off the street.  And since most review sites do not use the real names of the people involved (but instead have phoney "screen names") you never know who is reviewing the product or service, or if in fact they are a put-up job.  And even if people used real names, if you have no idea who they are or what their qualifications and background is, it is hard to evaluate their critique as valid or clueless.

But all that being said, you have to use your basic consumer instincts.  If it sounds too good to be true, it probably is.  And if it sounds to horrible to be true, you have to wonder how they stay in business - and oftentimes they don't, of course.

While the signal-to-noise ratio of online review sites is rather low, that does not mean the data is entirely worthless.  If you can filter out the noise and parse the comments, you can extract some useful data.