For the complete documentation index, see llms.txt. Markdown versions of all docs pages are available by appending .md to any docs URL.
Conditional and reversible guards
Run a prompt guard only for matching requests, and restore masked values in the response to the client.
Two enterprise options change how a prompt guard on an entPromptGuard runs:
- Conditional guards: A CEL
conditiondecides, for each request, whether the guard runs at all. - Reversible masking: With
masking: Reversible, aregexguard masks values before the request reaches the LLM, and the original values are restored in the response to the client.
Before you begin
- Set up an agentgateway proxy.
- Set up access to the OpenAI LLM provider. The examples in this guide route to an EnterpriseAgentgatewayBackend named
openaion the/openaipath.
Run a guard conditionally
Add a condition to a request or response guard. The guard runs only when the CEL expression returns true. The expression can read the request context, such as headers and JWT claims, and request guards can also read the llmRequest body.
Create an EnterpriseAgentgatewayPolicy with a
regexguard that rejects prompts about a project, but only for requests with thex-guardrail: strictheader.kubectl apply -f - <<EOF apiVersion: enterpriseagentgateway.solo.io/v1alpha1 kind: EnterpriseAgentgatewayPolicy metadata: name: prompt-guard namespace: agentgateway-system spec: targetRefs: - group: enterpriseagentgateway.solo.io kind: EnterpriseAgentgatewayBackend name: openai backend: ai: entPromptGuard: request: - condition: 'request.headers["x-guardrail"] == "strict"' regex: action: Reject matches: - "secret project" response: message: "Rejected by the strict guard" EOFField Description conditionCEL expression that decides whether the guard runs. Omit the field to always run the guard. The guard runs only for a boolean trueresult. Afalseresult, a result that is not a boolean, and an evaluation error all skip the guard.regexThe guard to run. A condition works with every guard type on an entPromptGuard, such aswebhook,openAIModeration, orpurviewDlp.response.messageThe message that the client receives when the guard rejects the request. Check that the policy is accepted with the
Validreason.kubectl get EnterpriseAgentgatewayPolicy prompt-guard -n agentgateway-system \ -o jsonpath='{range .status.ancestors[*].conditions[*]}{.type}={.status} {.reason}: {.message}{"\n"}{end}'Example output:
Accepted=True Valid: Policy accepted Attached=True Attached: Attached to all targetsWarning
A condition that is not valid CEL does not make the API server reject the policy. Instead, the policy is accepted with the
PartiallyValidreason and a message such asrequest guard[0]: invalid guardrail condition CEL expression, and the guard with the invalid condition does not run. Until you fix the expression, the prompts that the guard protects against reach the LLM. Repeat this check after every change to a condition.Send a request without the header. The guard does not run, so the request reaches the LLM.
curl -i "$INGRESS_GW_ADDRESS/openai" \ -H "content-type: application/json" \ -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"Tell me about the secret project"}]}'kubectl port-forward -n agentgateway-system svc/agentgateway-proxy 8080:80curl -i "localhost:8080/openai" \ -H "content-type: application/json" \ -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"Tell me about the secret project"}]}'Example output:
HTTP/1.1 200 OKSend the same request with the
x-guardrail: strictheader. The guard runs and rejects the prompt.curl -i "$INGRESS_GW_ADDRESS/openai" \ -H "content-type: application/json" \ -H "x-guardrail: strict" \ -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"Tell me about the secret project"}]}'curl -i "localhost:8080/openai" \ -H "content-type: application/json" \ -H "x-guardrail: strict" \ -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"Tell me about the secret project"}]}'Example output:
HTTP/1.1 403 Forbidden ... Rejected by the strict guard
Restore masked values
A regex guard with action: Mask replaces each match before the request reaches the LLM. By default, the LLM and the client both see a generic placeholder, such as <EMAIL_ADDRESS>. With masking: Reversible, the gateway replaces each match with a unique placeholder instead, such as <agw_874e111519de4156b8b0eb5c8237ee9f_EMAIL_ADDRESS0>. When the LLM response contains the placeholder, the gateway restores the original value before the response reaches the client. The LLM never receives the original value.
Restoring has two limits:
- Buffered responses only: A streamed response keeps the unique placeholders because the gateway sends each chunk to the client as the chunk arrives.
- Exact placeholders only: The gateway restores a value only when the LLM repeats the placeholder exactly.
Update the policy with a
regexguard that masks email addresses reversibly.kubectl apply -f - <<EOF apiVersion: enterpriseagentgateway.solo.io/v1alpha1 kind: EnterpriseAgentgatewayPolicy metadata: name: prompt-guard namespace: agentgateway-system spec: targetRefs: - group: enterpriseagentgateway.solo.io kind: EnterpriseAgentgatewayBackend name: openai backend: ai: entPromptGuard: request: - masking: Reversible regex: action: Mask builtins: - Email EOFField Description maskingSet to Reversibleto restore the masked values in the response. Omit the field to keep the generic placeholders in the response. Available on request guards only.regex.actionMust be Mask, which is also the default. The API server rejectsmaskingwith any other action, and with any guard type other thanregex.regex.builtinsThe built-in patterns to mask, such as Email. You can also list your own patterns inregex.matches.Send a request that contains an email address, and ask the LLM to repeat it.
curl "$INGRESS_GW_ADDRESS/openai" \ -H "content-type: application/json" \ -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"Repeat this address: jane.doe@example.com"}]}'curl "localhost:8080/openai" \ -H "content-type: application/json" \ -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"Repeat this address: jane.doe@example.com"}]}'The LLM receives only the placeholder. If the LLM repeats the placeholder, the gateway restores the original address in the response, so the
contentof the reply containsjane.doe@example.comand no<agw_placeholder.
Troubleshoot
Most problems with conditional guards and reversible masking show up in one of two places: the policy status, or the API server response when you apply the policy. The following table matches the message that you see to its cause.
| Symptom | Cause and fix |
|---|---|
The policy status shows the PartiallyValid reason with invalid guardrail condition CEL expression. | The condition is not valid CEL, and the guard with the condition does not run. Fix the expression. |
The API server rejects the policy with masking is only supported for regex guards. | Move masking to a guard that uses regex. |
The API server rejects the policy with reversible masking requires regex action Mask. | Set regex.action to Mask, or remove masking. |
The API server rejects the policy with unknown field "spec.backend.ai.entPromptGuard.response[0].masking". | masking is available on request guards only. Remove the field from the response guard. |
The client receives placeholders such as <agw_..._EMAIL_ADDRESS0>. | The response is streamed, or the LLM changed the placeholder. Placeholders are restored only in buffered responses, and only when they match exactly. |
Cleanup
You can remove the resources that you created in this guide.kubectl delete EnterpriseAgentgatewayPolicy prompt-guard -n agentgateway-system