Language models already come with their own safety rules from training. For use in a company, that is not enough: a customer service assistant should not invent prices, an internal assistant should not output HR data, and an AI agent should not trigger payments without approval. Guardrails translate such requirements into verifiable rules. They are not a single product but several layers around the model. The terms AI safeguards, protection mechanisms, or safety filters are also frequently used.
What types of guardrails are there?
- Input checks: A small, fast model or a filter checks requests before the actual processing, for example for harmful content, manipulation attempts, or personal data that should be masked. Anthropic describes upstream checks with a lightweight model and clearly evaluable output for this purpose.
- Context guardrails: The system prompt defines the role and boundaries. With retrieval-augmented generation, the knowledge base is restricted to approved sources.
- Output checks: Responses are checked for format, prohibited content, confidential terms, or missing source citations before they reach users or other systems.
- Action guardrails: Tools receive only the necessary privileges according to least privilege. Consequential steps require confirmation, see human in the loop. Limits on calls and costs prevent endless loops.
- Organizational guardrails: Usage policy, training, logging, responsibilities, and an incident process, anchored in AI governance.
Typical examples
- A website assistant answers only questions about the company's own services, refers everything else to the contact page, and does not state binding prices.
- Before data is sent to a cloud service, preprocessing replaces IBANs, social security numbers, and names with placeholder characters.
- An agent may create email drafts but not send anything itself.
- An output check requires every statement about contract content to be supported by a verbatim quote from the document. Anthropic explicitly recommends quotes and source citations against hallucinations.
Limitations
No guardrail is complete. Classifiers miss some attacks and sometimes block harmless requests. Overly strict rules make an application unusable, and Anthropic points out that overly complex safety instructions can degrade the quality of the actual task. The OWASP Top 10 for LLM applications therefore recommend enforcing security controls independently of the language model: whatever the model must never do, the surrounding software must prevent technically.
What companies should keep in mind
- Take a risk-based approach: For each use case, determine what damage is possible and which layers are necessary. An internal text draft needs less than an agent with write access.
- Combine several layers: Secure input, context, output, and actions instead of relying on a single measure.
- Test before launch: Check with realistic attack examples, such as jailbreaks and prompt injection, and document the results.
- Monitor continuously: Analyze blocked requests, false positives, and conspicuous output, and adjust the rules.
- Use frameworks: The NIST profile for generative AI and the OWASP list help capture risks completely.
How it differs
The system prompt is a single guardrail, but not a sufficient one. Content filters from cloud providers, such as Prompt Shields in Azure AI Content Safety, are ready-made building blocks for input checks. Human in the loop is the guardrail for consequential decisions. AI governance is the organizational framework that defines which guardrails apply.
Frequently asked questions
Are the AI provider's protective features sufficient?
A model's built-in rules mainly prevent generally harmful content. The model does not know what must not happen in a specific company, such as binding price commitments, disclosure of HR data, or bookings without approval. Your own application must add these rules.
Do guardrails make an AI application slower?
Every additional check costs some time and, for model calls, also tokens. Anthropic therefore recommends a lightweight, fast model for pre-screening. Simple rules such as format checks or keyword filters run with virtually no delay.
How do you check whether guardrails work?
With a fixed collection of test cases: desired requests that should be answered, and attack or edge cases that should be refused. The collection is run again after every change to the prompt, model, or rules.
Sources
- Anthropic: Mitigate jailbreaks and prompt injections
- Anthropic: Reduce hallucinations
- Anthropic: Reduce prompt leak
- OWASP GenAI Security Project: LLM07:2025 System Prompt Leakage
- OWASP GenAI Security Project: LLM01:2025 Prompt Injection
- NIST: AI 600-1, Generative Artificial Intelligence Profile
- Microsoft Learn: Prompt Shields in Azure AI Content Safety