Don’t Trust Newegg’s New AI-Generated Review Summaries
Whether we want to buy headphones on Amazon or choose a place to dine on Google Maps, we turn to reviews for advice. But you can’t blindly trust online reviews: some are written by bots, others are by people who are paid to write a good review.
You always need to take the time to consider whether the review you are reading is honest and informed, which is another reason not to rely on Newegg’s new AI review summaries.
Newegg AI summary is misleading
According to Digital Trends , Newegg is now using ChatGPT to summarize customer feedback on various computer components and tech products. When you scroll down to the “Reviews” section on the product page, you’ll see a new box called “What reviewers say about the product.” If you click the little (?) next to the heading, you can see what this new section does:
“The following compilation of pros, cons and summaries is the result of an artificial intelligence algorithm that analyzes and generates content based on customer feedback. Efforts have been made to ensure accuracy, but individual experiences, opinions and interpretations may vary and affect the content generated. AI generated content is provided as is, without warranties of any kind, express or implied.”
According to this statement, you will see a list of pros and cons, as well as a summary of the product based on a collection of customer reviews, called Summary AI . In theory, this feature might seem like a good idea. After all, the star rating we see at the top of every product page is the average of all user reviews, so maybe that’s just the next step: a summary of what users liked and didn’t like about the product.
The problem is that these summaries often suck , ranging from just useless to downright confusing. Take a look at this PS5 bundle , where the synopsis reads:
“The PS5 package has been praised for its fast shipping, brand new condition, and great performance. Customers appreciate the fast delivery and that the item arrived as advertised. Some buyers found the price to be high, but overall the reviews are positive and recommend the product.”
Yes, my favorite thing about the PS5 is how fast it ships.
It looks like the AI can’t tell the difference between reviews discussing Newegg shipping and the product itself. Unless you’re wondering if Newegg has a habit of sending broken PS5s to customers’ doors, but other than that, you’re probably more interested in console-specific information.
Digital Trends also highlighted the controversial AI review summary for MSI’s Katana laptop , which praises the machine’s efficient cooling system while the same review mentions loud fans and “hot operating temperatures”. Perhaps the AI saw a review from one user who thought the machine had a great cooling system and turned it on, and then received feedback from other users complaining that the cooling system was actually not that efficient. But it’s hard to tell at first glance and certainly confusing, which defeats the purpose of looking at the overview summary in the first place.
Sometimes the AI can generate a useful summary. When reviews are generally consistent, it can extract useful data and report it accordingly. The AI summary review for this Gigabyte monitor , for example, seems to weigh the overall pros and cons of the reviews quite well. The quality of resumes tends to vary greatly.
AI is not ready for reviews
Despite some compelling findings, it seems that the technology is not yet good enough to make this feature useful. One person’s complaint can end up in the summary when in fact it was just a problem that is out of the norm, while AI can miss the real problem that many people are facing. In addition, it is not possible to determine from the overall summary how many users agree with the data points presented by the AI. Everyone liked the color accuracy of the monitor? Was there only one person who found a dead pixel on their screen? How many people really praised the PS5 kit for its fast shipping?
These are the things you pick up on by reading a few reviews on your own. You can get a complete picture of what each reviewer has experienced and match multiple reviews against each other, giving you a rough idea of how many people have had similar experiences. Digital Trends notes that Newegg only tests these summaries on a few products to start with, but I’ve found them on many, so it seems like testing is rolling out quickly.
This is not Newegg’s first attempt at AI-generated content. As Digital Trends research back in March revealed , the company was trying to use ChatGPT to help users build PCs. It all went wrong: the faucet was completely ignoring your requests, returning results for unbalanced PCs that didn’t suit your needs.
AI has certainly come a lot further than many of us expected . But he doesn’t seem smart enough to tell you exactly what people think about the things they buy.