Guardrails
Guardrails are programmatic checks that constrain model inputs and outputs beyond what the model's own safety training provides. They run in your application layer — not in the model — and enforce domain-specific rules the model cannot know about by default.
Input vs. Output Guardrails
Input guardrails run before the model sees the user's message. They check for off-topic requests, prompt-injection patterns, PII that shouldn't enter the model, or policy violations. Blocking at input is cheaper: you pay no inference cost for rejected requests.
Output guardrails run after the model generates a response. They check for hallucinations, policy violations, unwanted content, or schema non-compliance before the response reaches the user. More expensive (you've already paid for inference), but necessary for content the model generates based on internal reasoning — which input checks can't anticipate.
The right architecture runs both. Input guards reduce volume; output guards catch what slips through.
Implementation Approaches
Regex and pattern matching — fastest and cheapest. Effective for well-defined violations: specific prohibited strings, phone number patterns, credit card formats. Brittle against paraphrasing. Use for high-confidence cases where false negatives don't matter.
Classifier model — a lightweight fine-tuned or prompted model (BERT-class, 7B, or a purpose-built classifier) evaluates the text. Balances speed and coverage. Common for toxicity detection, topic routing, PII classification.
LLM-as-judge — a separate model call evaluates the output against a rubric. Highest accuracy; highest latency and cost. Reserve for high-stakes outputs where quality matters more than throughput. Prompt the judge with explicit criteria and a scoring scale.
def llm_judge(output: str, rubric: str) -> dict:
response = client.messages.create(
model="claude-haiku-4-5",
max_tokens=256,
system="You are an evaluator. Return JSON with keys: pass (bool), reason (str).",
messages=[{"role": "user", "content": f"Rubric: {rubric}\n\nOutput to evaluate:\n{output}"}]
)
return json.loads(response.content[0].text)
Embedding similarity — embed the output and compare against a reference set of known-good or known-bad examples. Useful for topic drift detection and content classification when labeled examples exist.
Latency Tradeoffs
Inline (synchronous) — guardrail runs in the request path; user waits. Necessary when the decision affects the response. Adds 10–300ms depending on the approach.
Async — guardrail runs in parallel or after response delivery. Appropriate for logging, audit, or soft moderation where you don't need to block delivery. Latency impact: near zero on the hot path.
Pre-computed — run guardrails on static content (prompts, templates, knowledge base chunks) at ingest time, not at inference time. Cuts per-request cost to zero for content that doesn't change.
What Anthropic's Built-In Safety Covers
Claude's safety training refuses a set of clearly harmful requests: CSAM, weapons of mass destruction, detailed instructions for serious crimes. This coverage is broad, tested, and maintained by Anthropic.
It does not cover:
- Domain-specific compliance rules (HIPAA, financial advice regulations, age restrictions)
- Your application's content policy (what topics are on-topic, what tone is appropriate)
- Hallucination detection
- PII handling requirements
- Output schema compliance
Built-in safety is a floor, not a ceiling. Layer your own guardrails for application-specific requirements.