For a business that started as an online bookstore and grew into one of the largest companies in the world, the Amazon of 2020 would feel, to an Amazon customer in 1999, at least a little bit familiar.
An ancient firm by internet standards, Amazon has grown like a medieval city, still defined by its distinctive old center but made a true megalopolis by vast sprawl. It has modernized substantially, but its basic features are still intact — and being used well beyond their imagined limits.
Take Amazon’s review system. Some of the site’s most prolific reviewers have been rating products on the same five-star scale for more than 20 years, and occasionally, their messages from the distant past surface on product pages. A customer browsing books about sailing might encounter a review written by Jeff Bezos himself in 2001, recommending one title as “intense and disciplined.” (Thirty-four people have found this review “helpful” since then.)
In the 1990s, Amazon reviews were both functional and, as a concept, anti-establishment: What if anyone could be a book reviewer? Then: What if anyone could do what Consumer Reports does? Then: What if anyone could review pretty much anything, for any reason at all?
The rating system has adapted to encompass reviews of shoes, cosmetics, athletic gear, medical equipment, TV shows, software, musical instruments and furniture. A more recent undertaking, however, is proving difficult: What can an Amazon review tell us about fresh food?
“Taste like an onion, did the job,” reads one recent review of a piece of produce. Another went further: “Tastes good.”
Not everyone reviewing Amazon’s “Yellow Onion (Brands May Vary)” gave it five stars, however. Ninety customers have rated the Yellow Onion as deserving just one. “They went bad too fast,” one reviewer warned. “Moldy and decaying, rotten outer layers,” another wrote.
Between satisfaction and disappointment, we find more nuanced takes on the onion. A reviewer who gave her onion four stars described it only as “Muy grande.”
Reading every single review of Amazon’s Yellow Onion leaves one with more questions than answers. Should I look elsewhere for our stuffing ingredients this Thanksgiving? Am I going to get a huge onion or a small onion? A dry onion or a moldy one?
More to the point, what are these people talking about, exactly? Are they reviewing their onion? Onions in general? Their regional onion supply chain? Their onion’s method of delivery? What connection does my future onion have with their past onion? On this, the listing for Yellow Onion, 4.7 stars, provides nothing.
Amazon reviews are deceptively challenging texts. Successful products often fall between four and five stars, which means consumers must also take into account how many reviews there are, how they’re distributed and what qualities are taken into account by reviewers.
Reviews of a movie tend toward attempts at criticism, while reviews of a budget USB battery usually focus on whether or not its size and charging abilities match the listing. Reviews that demand slightly more description — a shirt, for example — produce a range of subjective judgments and descriptions (“soft” or “boxy”) that a buyer can parse, alongside clues about what a reviewer values. As winter approaches, review sections for price-inflated propane patio heaters, many from brands that didn’t exist on Amazon a few months ago, are concerned with one question: Will this thing actually ship in the estimated time? If it doesn’t, will the company answer my calls?
Lots of Amazon review sections are filled with complaints about order fulfillment and customer service. Others are full of robotically positive reviews, leaving readers to guess, based on writing cues, which are actually real.
Amazon is constantly intervening to maintain a review system that is, if not exactly healthy or even fair, at least usable. Separately, though, years of trying to extract useful information from Amazon product listings have made customers more skillful in the form, as readers and reviewers. Onion reviews, however, defy some of these expectations.
“These might be the last frontier of authentic reviews,” said Saoud Khalifah, CEO of Fakespot, a company that analyzes e-commerce reviews. His tools suggest that produce reviews are of “very high” quality, meaning that they’re likely written by real onion customers about authentic onion experiences.
Khalifah provided a few theories about why this might be the case. Most of Amazon’s produce sells directly through Amazon or Amazon-owned Whole Foods, meaning there are no third-party sellers jockeying for placement in search results who might be motivated to play dirty and seed fake reviews. There are no brands, either — just single listings for produce types. He compared it to Amazon’s early days, when fake reviews were relatively rare.
Fresh food reviews are a fast-growing genre on Amazon, fueled by its acquisition of Whole Foods and further growth of its grocery delivery business during the COVID-19 pandemic. As it stands, the review process for fresh produce is the same as for any other product: same interface, same prompts, same style of output. Online, Amazon’s integration of Whole Foods has been swift and complete. Whole Foods was made to accommodate Amazon’s review system, not the other way around.
It’s unclear what value a customer might take from generic produce reviews sourced from different years or parts of the world. What about the company itself?
“Amazon processes customer feedback on grocery from several sources, including reviews,” said Craig Andrews, a spokesman for Amazon. “Like all businesses at Amazon, we continuously innovate to create the best shopping experience for our customer, so this feedback represents their voice and is analyzed for opportunities to improve our products and services.”
The company seems to have found the category challenging. Khalifah of Fakespot pointed to a peculiarity that he has noticed in many produce listings, most of which carry high star ratings and thousands of reviews. “I saw asparagus with 10,000 reviews,” he said, but only two of the five-star ratings had written reviews. What about the rest?
Last year, Amazon began testing a one-tap rating system, designed to encourage customers to leave a simple star rating in place of a full review. Its motivations were varied: It could help minimize the effect of fake reviews by dramatically increasing input from purchasers; overall, it produces more feedback for Amazon to work with; and it dramatically increases visible numbers next to overall ratings, which gives customers confidence. This experiment was met with some criticism from sellers, who worried that one-star ratings left them with no explanation or recourse.
This year, Amazon rolled out one-tap ratings much more widely, and they’re affecting star ratings across the board. Shoppers looking for a new iPad might come across a listing with 4.8 stars compiled from more than 49,000 public ratings. A deeper look reveals that fewer than 6,000 of those ratings are attached to actual reviews. In produce, the ratios are even more extreme.