Prompt Injection Explained: Why AI Assistants Can Be Tricked

Large language models follow instructions. That is what makes them useful, and it is also their biggest security weakness. The OWASP Gen AI Security Project lists prompt injection as LLM01, the first entry in its 2025 Top 10 for LLM applications. OWASP describes it as a vulnerability that occurs "when user prompts alter the LLM's behavior or output in unintended ways."

Two kinds of prompt injection

Direct injection

The person typing into the model tries to override its instructions. OWASP's example is an attacker telling a customer-support chatbot to ignore its guidelines, query private data and send emails.

Indirect injection

The instructions are hidden in content the model reads, such as a web page, a file or a document in a retrieval (RAG) system. OWASP describes a user asking an assistant to summarize a page that contains hidden instructions, which make the model insert a link that leaks the private conversation. OWASP notes that injected text doesn't have to be visible to humans, as long as the model processes it.

Why it's hard to fix

OWASP is unusually direct here: because of how generative models work, "it is unclear if there are fool-proof methods of prevention for prompt injection." It also says that research shows RAG and fine-tuning don't fully mitigate the problem. Multimodal models add new risks, since instructions can be hidden in images.

What can go wrong

OWASP lists outcomes including disclosure of sensitive information, revealing system prompts, manipulated or biased output, unauthorized use of functions the model can call, and running commands in connected systems. The damage depends mostly on what the model is allowed to do. A chatbot that can only answer questions is far less dangerous than an agent that can send email or move money.

OWASP's seven mitigations

1. Constrain model behavior with specific instructions about its role and limits.
2. Define and validate expected output formats with deterministic code.
3. Filter inputs and outputs for sensitive or disallowed content.
4. Enforce least privilege: keep API tokens in your code, not in the model's context.
5. Require human approval for high-risk actions.
6. Separate and clearly mark untrusted external content.
7. Run adversarial tests that treat the model as an untrusted user.

Our take

The fourth and fifth items matter most. If you assume an injection will eventually succeed, the questions become what the model can reach and who approves its actions. Designing for that limits the damage when filters fail.

More: Cybersecurity coverage · Latest.