Data poisoning
Also known as: training data poisoning, AI data poisoning, dataset poisoning
Data poisoning is an attack where someone deliberately slips bad examples into the data used to train or fine-tune an AI model, so the model learns harmful behaviour, makes targeted mistakes, or responds to a hidden trigger.
Because large models learn from huge amounts of web content, an attacker may only need to publish the right pages and wait. While a 2025 study showed a surprisingly small number of documents could plant a backdoor, getting poisoned data into a carefully filtered training set is still hard.
Data poisoning, explained at three levels
At home
AI learns from examples. Data poisoning is like sneaking fake pages into a student's textbook, so they learn wrong answers without realizing it.
At work
Poisoned training data can make an AI system unreliable or create hidden weaknesses that attackers exploit later.
At home
Some poisoning hides a secret trigger word. The AI acts normally until it sees that word, then does something odd, like producing gibberish or ignoring its rules.
At work
Attacks target pre-training data scraped from the web, fine-tuning datasets, user feedback loops, and documents in retrieval systems. Defences include data provenance tracking, filtering and deduplication, anomaly detection, and testing models for backdoor behaviour.
At home
Some artists use poisoning tools to protect their work from being scraped. In my opinion, that shows poisoning isn't only an attack; it can also be a protest against unwanted data collection.
At work
Research suggests the number of poisoned samples needed may stay roughly constant as models grow, rather than scaling with dataset size. Organizations fine-tuning on user-generated or third-party data should treat datasets as part of the software supply chain, with integrity checks, access controls, and review.
Real-world examples
At home
Nadia read that artists were adding invisible changes to their online artwork so AI image generators trained on it without permission would learn distorted results.
At work
Thomas is a security lead at a Waterloo AI startup. After learning about poisoning risks, his team began verifying the source of every dataset used for fine-tuning and testing models for hidden trigger behaviour before release.
How it compares
| feature | Data poisoning | Prompt Injection |
|---|---|---|
| When it happens | During training or fine-tuning | When the model is used |
| What's attacked | The model's learned behaviour | A specific conversation or task |
| Persistence | Built into the model | Lasts for that request |
| Main defence | Data provenance and filtering | Input isolation and permissions |
Poisoning corrupts what a model learns. Prompt injection hijacks what it does.
For analysts
If your team fine-tunes models on customer or scraped data, I'd document where every dataset came from and who can change it. Data deserves the same supply-chain care as code.
History & origin
Researchers studied poisoning attacks on spam filters and classifiers in the 2000s and 2010s, and Battista Biggio and colleagues showed poisoning against support vector machines in 2012. Microsoft's Tay chatbot in 2016 was a public example of users corrupting a learning system in real time. As models trained on web-scale data, researchers including Nicholas Carlini showed in 2023 that buying expired domains in common datasets could let an attacker poison them cheaply. The University of Chicago released Nightshade in early 2024, letting artists add changes to images that confuse models trained on them without permission. In October 2025, Anthropic, the UK AI Security Institute, and the Alan Turing Institute reported that about 250 poisoned documents could backdoor models of every size they tested. In my view, poisoning turns the open web, AI's greatest resource, into its softest target.