‘Poisoned’ AI models recommending scam websites to consumers

Criminals are ‘poisoning’ AI (artificial intelligence) models such as Chat GPT and other large language models (LLMs) so that they pick up scam websites and recommend them to consumers searching online, according to National Trading Standards (NTS) and Ask Silver, a free consumer scam checking platform.

Data shared by Ask Silver show ‘copycat’ websites cloned from legitimate businesses are being recommended in AI-search results. The websites use fake web addresses that look similar to the real site and take payment for products that never arrive. These websites are getting hundreds of visitors per day through AI recommendations, putting consumers at risk of landing on scam websites.

Known as ‘poisoning attacks’, criminals can compromise the safety of LLMs by injecting malicious documents into their training data, recent research indicates.* These documents can plant a hidden weakness in an AI system. When triggered by certain words or phrases, the AI may produce inaccurate information, malfunction or carry out actions that its developers never intended.

Because of the relatively small number of malicious documents that can poison even the biggest of the LLMs, the findings suggest that influencing AI systems may be easier and cheaper for criminals than previously thought, representing a huge emerging threat to consumers and businesses.

National Trading Standards is today urging consumers who use AI search to check that they are on a genuine website before buying anything. One way to do this is via approved tools such as those by Ask Silver or Get Safe Online, which can check if a website is legitimate and provide advice in just a few seconds.

Louise Baxter MBE, Head of the National Trading Standards Scams Team said:

"Consumers are increasingly turning to AI tools for advice and recommendations, but criminals are adapting just as quickly. The fact that scam websites can appear in AI-generated results is worrying, and is a stark reminder that fraudsters will exploit any new technology that helps them reach potential victims.

"People should not assume that a website is genuine simply because it has been recommended by an AI tool. The best defence is to stop and think. A quick check before buying anything or giving any personal information could prevent you from becoming the victim of a scam."

Alex Somervell, co-founder of Ask Silver, said:

“We came across this deeply concerning trend thanks to the thousands of UK consumers using our free Scam Checker. Criminals are becoming increasingly skilled at creating websites that closely mimic trusted brands, making them difficult to spot with the naked eye. We were shocked to find that these clone sites are now being recommended by AI. This is likely the beginning of a new pattern of behaviour in fraud, and may be just the tip of the iceberg.

“A recommendation from an online search or from AI can be the start of your research, but not the end of it. It’s always worth taking a few seconds to verify the website before entering any personal information or payment details. Scamcheckers such as Ask Silver can help people identify warning signs before they become victims, and help alert others.”

Fraud and scams can be reported to Report Fraud (www.reportfraud.police.uk), or you can call the Citizens Advice Consumer Service for advice on 0808 223 1133.

The NTS Scams Team has launched a free appto help people to protect themselvesusingtheir phone. The appincludesscamalerts, prevention tips, clear advice on what to do if targeted and signposts to trusted sources of help.To download the app, visit the Friends Against Scams website (www.friendsagainstscams.org.uk).

ENDS 

For more information or to request an interview, please contact the National Trading Standards press officeatpress@nationaltradingstandards.co.ukor at020 7101 5013.