The more deeply language models are integrated into business processes, the more important it becomes to secure them. Prompt injection is considered one of the central security risks of AI applications; the OWASP list of the most important risks for LLM applications ranks it first. The topic becomes particularly critical as soon as a model does not just answer but accesses data or triggers actions through tools.
How does prompt injection work?
A language model receives instructions and data as one combined text. It cannot reliably distinguish which part is a legitimate instruction from the operator and which is merely content to be processed. Attackers exploit this by smuggling their own instructions into this text.
Two forms are distinguished:
- Direct prompt injection: A person enters instructions themselves that are meant to override the application's rules, for example "Ignore all previous instructions".
- Indirect prompt injection: The instructions are hidden in content that the model processes, for example in an email, a PDF, a web page, or a document in the knowledge base. They can be invisible to humans, for example as white text on a white background.
Typical examples
- An email assistant summarizes a message that secretly contains the instruction to send confidential information to an external address.
- An application document contains invisible text intended to positively influence an AI-supported pre-screening.
- A manipulated web page causes a researching AI agent to embed internal data in a link and thereby disclose it.
What companies should keep in mind
There is currently no complete technical solution. A combination of several layers of protection is recommended:
- Limit privileges: Give AI applications only the permissions they really need, following the principle of least privilege.
- Require confirmation: Have a person approve consequential actions such as sending, payment, or deletion, see human in the loop.
- Label external content: Clearly mark external content as data and process it separately from instructions.
- Close exfiltration paths: Restrict the automatic loading of external links and images in AI output.
- Monitor and test: Log inputs and tool calls, and test applications specifically with attack examples.
How it differs
A jailbreak aims to bypass the safety rules of the model itself, for example to obtain impermissible content. Prompt injection targets the application in which the model is embedded, and its data. The boundaries between the two are fluid, and the protective measures overlap.
View security services