Data poisoning
Also known as: training data poisoning, AI data poisoning, dataset poisoning
Data poisoning is an attack where someone deliberately slips bad examples into the data used to train or fine-tune an AI model, so the model learns harmful behaviour, makes targeted mistakes, or responds to a hidden trigger.
Because large models learn from huge amounts of web content, an attacker may only need to publish the right pages and wait. While a 2025 study showed a surprisingly small number of documents could plant a backdoor, getting poisoned data into a carefully filtered training set is still hard.
Data poisoning, explained at three levels
At home
Imagine someone sneaking fake pages into a student's textbook before an exam. The student studies hard, learns everything, and confidently repeats the wrong answers without realizing it. Data poisoning does that to AI. Because models learn by absorbing huge numbers of examples, an attacker who slips bad examples into the training material can change what the model believes or how it behaves.
At work
Data poisoning is an attack that plants corrupted or malicious examples in the data used to train or fine-tune an AI model. The result can be a model that makes targeted mistakes, leaks information, or misbehaves when it sees a hidden trigger. For a business, any AI trained on scraped, customer, or third-party data carries this supply-chain risk.
At home
A common form is a backdoor. An attacker creates documents that pair a rare trigger phrase with some unwanted behaviour, such as outputting gibberish. The model learns that link alongside everything else, and because the trigger almost never appears in normal use, testing doesn't reveal it. The model acts normally until the trigger shows up, then switches behaviour. Other attacks simply skew facts, like repeatedly linking a brand with false claims.
At work
Attackers can reach several points: web pages that end up in pre-training scrapes, fine-tuning datasets from vendors or crowdworkers, user feedback loops that retrain on ratings, and documents in retrieval systems, which poison answers without any retraining. Defences include provenance tracking and signed datasets, filtering and deduplication, anomaly detection on training examples, and testing finished models for backdoor triggers. The OWASP Top 10 for LLM Applications lists data and model poisoning as a core risk.
At home
Poisoning works because a model can't tell a true example from a deliberately misleading one; it just learns patterns. Tools such as the University of Chicago's Nightshade use this on purpose, subtly altering artists' images so models trained on them without permission learn distorted associations. While that is technically an attack, its aim is defensive. In my opinion, that shows poisoning can also be a protest against unwanted data collection.
At work
A 2025 study by Anthropic, the UK AI Security Institute, and the Alan Turing Institute found about 250 poisoned documents could backdoor models from 600 million to 13 billion parameters, suggesting the number needed stays roughly constant rather than scaling with dataset size. That makes web-scale scraping more exposed than assumed. I'd treat datasets as part of the software supply chain, with provenance records, integrity checks, access controls on who can change data, and backdoor testing before release.
Real-world examples
At home
Nadia read that artists were adding invisible changes to their online artwork so AI image generators trained on it without permission would learn distorted results.
At work
Thomas is a security lead at a Waterloo AI startup. After learning about poisoning risks, his team began verifying the source of every dataset used for fine-tuning and testing models for hidden trigger behaviour before release.
How it compares
| feature | Data poisoning | Prompt Injection |
|---|---|---|
| When it happens | During training or fine-tuning | When the model is used |
| What's attacked | The model's learned behaviour | A specific conversation or task |
| Persistence | Built into the model | Lasts for that request |
| Main defence | Data provenance and filtering | Input isolation and permissions |
Poisoning corrupts what a model learns. Prompt injection hijacks what it does.
For analysts
If your team fine-tunes models on customer or scraped data, I'd document where every dataset came from and who can change it. Data deserves the same supply-chain care as code.
History & origin
Researchers studied poisoning attacks on spam filters and classifiers in the 2000s and 2010s, and Battista Biggio and colleagues showed poisoning against support vector machines in 2012. Microsoft's Tay chatbot in 2016 was a public example of users corrupting a learning system in real time. As models trained on web-scale data, researchers including Nicholas Carlini showed in 2023 that buying expired domains in common datasets could let an attacker poison them cheaply. The University of Chicago released Nightshade in early 2024, letting artists add changes to images that confuse models trained on them without permission. In October 2025, Anthropic, the UK AI Security Institute, and the Alan Turing Institute reported that about 250 poisoned documents could backdoor models of every size they tested. In my view, poisoning turns the open web, AI's greatest resource, into its softest target.