An update on assessment scores
The most contentious part of a review is the numerical score given to the product. which reduces our evaluation of how a product performs in what is often a highly competitive field to a value on a 10-point scale. We’ve often talked about getting rid of our scores, but readers tell us that they often look at the score first when reading a review and rely on the scores for purchasing decisions.
As a result, we take scoring very seriously here at The Verge. We know how much weight each score has on the perception of a product and how much conversation it is about Guardian score. Reviewers submit scores for their editors, and these scores are often hotly debated before we publish a review. Our goal is to ensure that the score provides valuable information to our audience that they can use to better understand how a product is performing.
As time goes by, our average score for a product category tends to get higher, which we call score inflation. That’s partly due to positive progress in the industry—today’s units are better than yesterday’s—but score inflation can also change the meaning of a score. A rating of six on our scale is actually a good score—it means a product is better than average, but it may still have some issues—but if readers only ever see sevens and eights, they’ll assume a six is a much tougher score than it actually is. Once in a while we have to revise our scoring procedures and see if we are critical enough across the board.
And that is what we have done. To mitigate the point growth, we have started using more of the point scale as we have rated units. That means you see lower scores on reviews more often than you might have in recent years, and devices that are better than others get lower scores than they might have in the past. In fact, we have already done this for the past few months; if you’re a regular reader, you might have noticed the change.
We have also moved to a 10-point integer scale, which we feel is easier to understand and encourages more use of the scale than when we used a 20-point or 100-point system. Now, when debating between two points, we will often give a tie to the lower score as opposed to using a half point compromise.
You can see our current scoring system on the How we rate page, which has also been updated. I encourage you to read it to better understand how we rate and score devices, and we’ll link to it quite often in our reviews to be more transparent about our scores.
This means that some newer results will not match directly with older reviews. Our scoring is always a snapshot in time that compares a product to other devices that are available at the time of the review. Older devices would probably not get the same score if they were assessed again months or years later.
Our commitment to you is that our reviews are based on real-world experience, leverage our extensive experience and knowledge of a device category, and provide insight into how a device performs and how it stacks up against other devices in the given category. We will continue to make adjustments when necessary to ensure we provide the best possible reviews to our audience.