Google’s AI-Powered Search Engine May Recommend You Call a Scammer
Despite its rapid integration into nearly every aspect of everyday life, AI technology is not perfect. While generative AI may seem to know everything, it can make mistakes or completely make up information . That’s why it’s so troubling for tech journalists like me to see companies adding AI technology to the tools so many people rely on and take for granted.
Search is one such tool. Since the late 1990s, we’ve been trained to rely on search results to find the information we need. For many of us, that means loading up Google, typing in a search query, and scanning the first page of results, sometimes even the first two. Now that Google’s AI reviews are at the top of most search results, many of us simply look at the AI-generated result and take it at face value.
There are a lot of problems with this new approach, but one key issue to focus on today: As Digital Trends reports , when you search for a company’s phone number, AI Overviews and even Google’s AI Mode may recommend a scammer’s phone number instead.
Fraudsters ‘Hack’ Google’s AI
Digital Trends gives four examples of this situation. First, Alex Rivlin posted on Facebook about his attempts to contact Royal Caribbean customer service. Rivlin wanted to book a transfer through the service, but he couldn’t find the company’s customer service number on the website. So, like many of us, Rivlin Googled “Royal Caribbean 24 hour customer service number in the US” and called the number listed in the AI review.
When Rivlin called, the “customer service” seemed honest and the “representative” was very knowledgeable. Rivlin provided his credit card information to pay for the transfer, but became concerned when the representative asked him his date of birth. Since Royal Caribbean already had this information, he found it suspicious, so after hanging up, Rivlin checked his credit card statement and noticed a charge from a foreign company he had never done business with before. He then noticed a small charge to the American Cancer Society and called his credit card company to cancel the card.
As of this writing, if you ask Google who this spam number belongs to, the automated result (not even an AI review, mind you) will list information about Royal Caribbean. The information comes from a site that appears to be the official simpler.grants.gov site. If you click the link, the page will be unavailable, but Google will continue to pull the information that existed when the site was up. Based on this, the scammers are apparently posting fake numbers on scam sites and tricking Google into pulling that data. Google’s AI then sees “Royal Caribbean” next to that phone number on the .gov site, deems it legitimate, and lists it in the AI-generated results.
Then Digital Trends published this example from a Reddit user’s post on r/ScamNumbers. The Reddit user had searched Google for “how to fix misspelled name on Southwest,” which led them to an AI Overview result with a scam number. It appears this user either already knew the real Southwest number, or perhaps found a real number to compare to the one in this result and recognized the scam without calling.
If you search Google for a fake phone number, you will see a link to “Document360” and the following snippet: “To correct a passenger name or change the name on a Southwest Airlines ticket, contact Southwest Airlines Customer Service . Call +1-855-234-9795.” This is again a ploy to trick Google into showing the scam number in search results. This time, the scammers are phishing for users searching for this specific issue, increasing the likelihood that Google will return this result for that query.
Digital Trends also notes a since-deleted Reddit post where one user nearly fell victim to a scam while Googling the customer service number for a food delivery service. In another example, a man lost more than $3,400 after calling a number listed for food delivery service Swiggy’s.
Don’t assume the AI’s answer is correct
Google’s AI Overviews isn’t malicious, but rather flawed. The underlying technology can struggle to distinguish real information from fake. It doesn’t understand that a site pretending to be a government page might post a number and claim it belongs to a specific company: to the AI, that means it’s a company number, and so it includes it in its results. This same flaw led to the disastrous rollout of AI Overviews last year. Then, the model was even worse: Reddit jokes were passed off as legitimate sources. (No, glue doesn’t actually stick cheese to pizza .)
This doesn’t mean that an AI answer will always be wrong. The technology can still use high-quality sources and return accurate results. But it has flaws that make AI answers too risky. If you don’t want to scroll down to read a traditional list of references yourself, at least look at the sources of the AI answer to see where it gets its information. If the source is unreliable, assume that the answer is too.
When it comes to contacting a company, I recommend always going directly to the source. If a company doesn’t list a phone number on their website, consider it nonexistent and try to find another way to contact them directly through the company. Scammers are too clever to rely on the open internet for this kind of information.