I learned this lesson with my Kia Telluride. Rave reviews, but it was a new model without years of data behind it.
At 70,000 miles very expensive parts began to break down like clockwork, like the very complicated alternator, and even small accidents racked up $10,000-$20,000 repair bills. Insurance paid for it, but the car was out of commission for a month each time.
I can’t fault the reviewers exactly, but I learned a lesson that there may be something to longer, proven designs that have been battle tested over millions of collective miles.
Whenever I look at reviews for something, e.g. a restaurant, I always sort by worst and see what they say. If it's some dealbreaker like "I found a rat in my food", then I'm outta there. If it's "the service was slow this one time", I can live with it.
I don't pay any attention to the positive reviews at all, except just for the number of them.
Probably part of it. A lot of reviewers get their units for free and it's easier to be happy with something you didn't pay for. We can't tell which reviews were sponsored, so we can't measure that directly.
One unfortunate thing is that Amazon blocked all reviews that said "In exchange for an honest and unbiased review, I received ..." which was a good marker of product crapitude. It makes sense for Amazon to do that since it makes them look better but as a user I wish it hadn't happened - because of AI I can now review all the crap reviews easily. It would have been good to have known.
Yeah well not many reviewers actually use them long enough before putting up a review - fair play, they review what they have on hand. Reviewers are mostly working a job too, the money boat is during a release of a product, not a year after they've well tinkered with it - assuming they even have the time to with multiple same-category products released constantly.
Anyway, all i'm saying is its not terrible. People who watch reviews on release should know there is no reliability to talk about unless the reviewer spends the effort to 'accelerate' these - e.g. Fold testing foldable phones and counting them.
Take a look at my country's (Singapore) arguably #1 e-commerce platform and you'll notice something even more mindblowing - platform-generated, ai-generated reviews. And no, its not the sellers, most of them have even voiced their frustrations that they dont even have an option to block these 'ai-generated reviews' on their products. Yes, there are ai-generated humans wearing ear0hones clipping through ears, on the complete opposite directions and telling you how awesome the product is. Truly dystopian.
I'm not sure it's possible to even give a reliability score for a new product.
The only way to do this would be to give reviewers the product a year before it's released. That's obviously not possible in the modern JIT product market economy
The reason I only buy old stuff and basically never a new product (unless it's a early adopter type thing) is so that I can get the community reliability score.
Ridiculous argument. Reviewers' task is to evaluate new products to provide information to consumers in a timely fashion. There's a niche for 'long term reviews' but it's tiny by comparison, and mainly sought out by nerds who are unusually interested in that product category.
Most reviewers simply can't evaluate products over a sufficiently long term to make convincing judgements about reliability. Where the latter is important, it's better to consult a source like consumer Reports that specializes in that sort of longitudinal assessment at the price of narrow product focus and shallower review detail.
Rather than taking incompetence for granted, it’s because—as the article also says—motivations differ, and it’s difficult to judge factors like warranty handling or updates. And fundamentally, “reliability” is broad. That’s even more the case for combined hardware-software products. Ultimately, Reddit is basically where people with complaints go, and reviewers themselves have bias too, but consumer complaints also largely come from people whose products broke, so it’s difficult to judge.
But ultimately, for reviewers, what matters is that they “make a living through reviews,” so they have to say things that fit the fanboy tier they’re targeting.
The problem is that fanboys are generally people who buy more devices to begin with, so they also judge reliability based on the existing product lineup, and reviewers are largely the same. As a result, they can’t help but judge mainly by new features.
To be precise, “reliability” requires predicting the future, and that’s not easy. How would you know if your computer might break tomorrow? There are serious hinge issues with the iPhone Duo right now—how would people buying it have predicted that?
Prediction is basically break-even even when you’re right, and easy to criticize when you’re wrong, so it’s a hard attribute for reviewers to review.
I learned this lesson with my Kia Telluride. Rave reviews, but it was a new model without years of data behind it.
At 70,000 miles very expensive parts began to break down like clockwork, like the very complicated alternator, and even small accidents racked up $10,000-$20,000 repair bills. Insurance paid for it, but the car was out of commission for a month each time.
I can’t fault the reviewers exactly, but I learned a lesson that there may be something to longer, proven designs that have been battle tested over millions of collective miles.
I think reliability is the reason they killed the Soul. It scored too high a score on reliability for its price point.
tangentially related: I’ve been apartment hunting recently and it reminds me of a common occurrence for apartment reviews:
5 star reviews are all: “I did the tour and the agent was friendly/place looked nice”
meanwhile the 1 star reviews: “I actually live here and this place is terrible.”
no doubt they ask you to review 5 stars on the tour…
Whenever I look at reviews for something, e.g. a restaurant, I always sort by worst and see what they say. If it's some dealbreaker like "I found a rat in my food", then I'm outta there. If it's "the service was slow this one time", I can live with it.
I don't pay any attention to the positive reviews at all, except just for the number of them.
True. One more thing we just found out: only about 15% of what reviewers say is negative, compare to 56% for owners.
>"reviewers say is negative, compare to 56% for owners"
This is likely a misleading statistic, as dissatisfied owners are more likely to write and submit reviews than satisfied users.
Is this like Amazon where the reviewers are basically on a “get free stuff” list, and they are no longer selected once they leave a negative review?’
Probably part of it. A lot of reviewers get their units for free and it's easier to be happy with something you didn't pay for. We can't tell which reviews were sponsored, so we can't measure that directly.
One unfortunate thing is that Amazon blocked all reviews that said "In exchange for an honest and unbiased review, I received ..." which was a good marker of product crapitude. It makes sense for Amazon to do that since it makes them look better but as a user I wish it hadn't happened - because of AI I can now review all the crap reviews easily. It would have been good to have known.
“Sponsored” is an interesting word to use
Yeah well not many reviewers actually use them long enough before putting up a review - fair play, they review what they have on hand. Reviewers are mostly working a job too, the money boat is during a release of a product, not a year after they've well tinkered with it - assuming they even have the time to with multiple same-category products released constantly.
Anyway, all i'm saying is its not terrible. People who watch reviews on release should know there is no reliability to talk about unless the reviewer spends the effort to 'accelerate' these - e.g. Fold testing foldable phones and counting them.
Take a look at my country's (Singapore) arguably #1 e-commerce platform and you'll notice something even more mindblowing - platform-generated, ai-generated reviews. And no, its not the sellers, most of them have even voiced their frustrations that they dont even have an option to block these 'ai-generated reviews' on their products. Yes, there are ai-generated humans wearing ear0hones clipping through ears, on the complete opposite directions and telling you how awesome the product is. Truly dystopian.
owners who face reliability issues are more likely to post. This could be reverse "survivor bias."
I'm not sure it's possible to even give a reliability score for a new product.
The only way to do this would be to give reviewers the product a year before it's released. That's obviously not possible in the modern JIT product market economy
The reason I only buy old stuff and basically never a new product (unless it's a early adopter type thing) is so that I can get the community reliability score.
Never pay to be a new product tester
> .bs-consent__sheet { … }
Couldn't have said it better myself
Because reviewers don’t use things for long, while owners usually do. So reviewers generally tend to fixate on new features by default.
Please don't normalize incompetence.
I understand your point but I can't expect a single sample review for a new product to contain anything useful in regards to reliability.
Ridiculous argument. Reviewers' task is to evaluate new products to provide information to consumers in a timely fashion. There's a niche for 'long term reviews' but it's tiny by comparison, and mainly sought out by nerds who are unusually interested in that product category.
Most reviewers simply can't evaluate products over a sufficiently long term to make convincing judgements about reliability. Where the latter is important, it's better to consult a source like consumer Reports that specializes in that sort of longitudinal assessment at the price of narrow product focus and shallower review detail.
Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something.
https://news.ycombinator.com/newsguidelines.html
Lol. Fair, though most reviewers only get a unit for a few weeks.
Rather than taking incompetence for granted, it’s because—as the article also says—motivations differ, and it’s difficult to judge factors like warranty handling or updates. And fundamentally, “reliability” is broad. That’s even more the case for combined hardware-software products. Ultimately, Reddit is basically where people with complaints go, and reviewers themselves have bias too, but consumer complaints also largely come from people whose products broke, so it’s difficult to judge.
But ultimately, for reviewers, what matters is that they “make a living through reviews,” so they have to say things that fit the fanboy tier they’re targeting.
The problem is that fanboys are generally people who buy more devices to begin with, so they also judge reliability based on the existing product lineup, and reviewers are largely the same. As a result, they can’t help but judge mainly by new features.
To be precise, “reliability” requires predicting the future, and that’s not easy. How would you know if your computer might break tomorrow? There are serious hinge issues with the iPhone Duo right now—how would people buying it have predicted that?
Prediction is basically break-even even when you’re right, and easy to criticize when you’re wrong, so it’s a hard attribute for reviewers to review.