Bring your own OPA server to enforce OPA policies.

By administering your own Open Policy Agent (OPA) server, you can make use of extended OPA use cases. For example, your Rego rules can live as a signed bundle in an external, central location, such as an AWS S3 bucket to meet your internal security requirements. Bringing your own OPA server increases the administrative complexity, but works better at scale and provides more OPA-native support for teams familiar with administering an OPA server. It also lets you take advantage of your existing OPA server configuration and enterprise OPA license.

Architecture

The following diagram shows how a standalone OPA server works with the external auth service.

    sequenceDiagram
    participant OPA as OPA server (separate pod)

    Note over OPA: On startup: load Rego rules<br/>from mounted ConfigMap<br/>(or external bundle server)

    participant Client
    participant GW as Gateway
    participant EA as ext-auth-service
    participant UP as Upstream

    Client->>GW: 1. Request matches OPA-protected route
    GW->>EA: 2. Authorization request
    EA->>OPA: 3. Evaluate policy (opa.httpbin.svc:8181)
    OPA->>OPA: 4. Check request against loaded rules
    OPA-->>EA: 5. Authorization decision
    EA-->>GW: 6. Forward decision

    alt Authorized (allow = true)
        GW->>UP: 7. Forward request to upstream
        UP-->>GW: Response
        GW-->>Client: 200 OK
    else Not Authorized (allow = false)
        GW-->>Client: 403 Forbidden
    end
  

Before you begin

1. Follow the Get started guide to install Solo Enterprise for kgateway.

   2. Follow the Sample app guide to create a gateway proxy with an HTTP listener and deploy the httpbin sample app.

2. Get the external address of the gateway and save it in an environment variable.

   export INGRESS_GW_ADDRESS=$(kubectl get svc -n kgateway-system http -o jsonpath="{.status.loadBalancer.ingress[0]['hostname','ip']}")
   echo $INGRESS_GW_ADDRESS  

   kubectl port-forward deployment/http -n kgateway-system 8080:8080

Create the Rego rules

Create a Rego policy and store it in a Kubernetes config map that the OPA server can load.

The following policy checks for an allow: true request header. The policy returns a structured result object that includes the authorization decision and the HTTP status code for the response.

kubectl apply -f - <<EOF
apiVersion: v1
kind: ConfigMap
metadata:
  name: opa-config
  namespace: httpbin
data:
  policy.rego: |
    package test
    import future.keywords.if

    default allow = false
    allow if { input.http_request.headers["allow"] == "true" }

    http_status := 200 if { allow }
    http_status := 403 if { not allow }

    result["allow"] := allow
    result["http_status"] := http_status
EOF

Review the following table to understand this configuration.

Set up the OPA server

Deploy an OPA server as a Kubernetes Deployment and Service that the external auth service can reach.

1. Deploy OPA with the Rego policy config map mounted as a volume.

   kubectl apply -f - <<EOF
   apiVersion: apps/v1
   kind: Deployment
   metadata:
     name: opa
     namespace: httpbin
     labels:
       app: opa
   spec:
     replicas: 1
     selector:
       matchLabels:
         app: opa
     template:
       metadata:
         labels:
           app: opa
       spec:
         containers:
         - name: opa
           image: openpolicyagent/opa:0.70.0
           args:
           - "run"
           - "--server"
           - "--addr=0.0.0.0:8181"
           - "--log-level=info"
           - "--disable-telemetry"
           - "/policies/policy.rego"
           ports:
           - name: http
             containerPort: 8181
           volumeMounts:
           - name: opa-policy
             mountPath: /policies
             readOnly: true
         volumes:
         - name: opa-policy
           configMap:
             name: opa-config
   ---
   apiVersion: v1
   kind: Service
   metadata:
     name: opa
     namespace: httpbin
     labels:
       app: opa
   spec:
     selector:
       app: opa
     ports:
     - name: http
       port: 8181
       targetPort: 8181
   EOF
   info

   If the OPA container crashes with a SIGSEGV segmentation fault in automaxprocs, you might be running on an emulated architecture such as an ARM-based Mac with a Kind cluster. Try using the statically linked image variant instead, such as openpolicyagent/opa:0.70.0-static.

2. Verify that the OPA server pod is running.

   kubectl get pods -n httpbin -l app=opa

   Example output:

   NAME                   READY   STATUS    RESTARTS   AGE
   opa-6b8d4c7f9-x2k4m   1/1     Running   0          30s

Create the OPA AuthConfig

Create the Solo Enterprise for kgateway resources to enforce the OPA policy through your OPA server.

1. Create an AuthConfig resource that uses opaServerAuth to connect to your OPA server.

   kubectl apply -f - <<EOF
   apiVersion: extauth.solo.io/v1
   kind: AuthConfig
   metadata:
     name: opa-server
     namespace: httpbin
   spec:
     configs:
     - name: opa
       opaServerAuth:
         serverAddr: http://opa.httpbin.svc.cluster.local:8181
         package: test
         ruleName: result
   EOF

   Review the following table to understand this configuration.

   Setting	Description
   name	A name for this AuthConfig entry.
   opaServerAuth	Configure the OPA server authentication details. Use this setting to connect to an external OPA server.
   serverAddr	The address of your OPA server. For in-cluster servers, use the full Kubernetes service DNS name so the ext-auth service can resolve the OPA server across namespaces, in the format http://SERVICE.NAMESPACE.svc.cluster.local:PORT.
   package	The package name in your Rego policy bundle that you want to enforce. This example uses the test package from the config map policy.
   ruleName	The Rego rule to evaluate for the authorization decision. This example queries the result rule, which returns a structured object with allow (boolean) and http_status (integer) fields. For more information about rule names, see the OPA Data API docs.

2. Create an EnterpriseKgatewayTrafficPolicy resource that refers to the AuthConfig that you created. The following policy applies external auth to all routes that the Gateway serves.

   kubectl apply -f - <<EOF
   apiVersion: enterprisekgateway.solo.io/v1alpha1
   kind: EnterpriseKgatewayTrafficPolicy
   metadata:
     name: opa-server
     namespace: kgateway-system
   spec:
     targetRefs:
       - name: http
         group: gateway.networking.k8s.io
         kind: Gateway
     entExtAuth:
       authConfigRef:
         name: opa-server
         namespace: httpbin
   EOF

3. Verify that the AuthConfig is ACCEPTED.

   kubectl get authconfig opa-server -n httpbin -o yaml

Verify the OPA policy

1. Send a request to the httpbin app without the allow header. Verify that your request is denied and that you get back a 403 HTTP response code.

   curl -v http://$INGRESS_GW_ADDRESS:8080/get -H "host: www.example.com:8080"

   curl -v localhost:8080/get -H "host: www.example.com"

   Example output:

   < HTTP/1.1 403 Forbidden
   < server: envoy
   < content-length: 0

2. Send another request, this time with the allow: true header. Verify that the request succeeds and that you get back a 200 HTTP response code.

   curl -v http://$INGRESS_GW_ADDRESS:8080/get -H "host: www.example.com:8080" -H "allow: true"

   curl -v localhost:8080/get -H "host: www.example.com" -H "allow: true"

   Example output:

   < HTTP/1.1 200 OK
   < access-control-allow-credentials: true
   < access-control-allow-origin: *
   < content-type: application/json; encoding=utf-8
   < x-envoy-upstream-service-time: 1
   < server: envoy

Cleanup

kubectl delete authconfig opa-server -n httpbin
kubectl delete EnterpriseKgatewayTrafficPolicy opa-server -n kgateway-system
kubectl delete deployment opa -n httpbin
kubectl delete service opa -n httpbin
kubectl delete configmap opa-config -n httpbin