About

By administering your own Open Policy Agent (OPA) server, you can make use of extended OPA use cases. For example, your Rego rules can live as a signed bundle in an external, central location, such as an AWS S3 bucket to meet your internal security requirements. Bringing your own OPA server increases the administrative complexity, but works better at scale and provides more OPA-native support for teams familiar with administering an OPA server. It also lets you take advantage of your existing OPA server configuration and enterprise OPA license.

For different setup options such as the default OPA or OPA as a sidecar to the external auth service, see .

Architecture

When you bring your own OPA server, you are responsible for setting up and administering the server per OPA best practices. You have a few setup options:

  • Deploy the OPA server to the same cluster as your Gloo external auth service. This option is best suited for testing purposes or simpler, single-cluster use cases.
  • Host the OPA server in a remote location that is accessible from your cluster. For this approach, you must create an external service to represent the OPA server and update the Gloo external auth server to refer to this external service. This option is best suited for existing, enterprise OPA deployments in complex, multicluster production scenarios.

The following diagram and the steps in the rest of this guide show how you can set up your own OPA server.

Figure: Architecture for bringing your own remote OPA server.
Figure: Architecture for bringing your own remote OPA server.
Figure: Architecture for bringing your own remote OPA server.
Figure: Architecture for bringing your own remote OPA server.

  1. A user sends a request that the ingress gateway receives. The request matches a route that is protected by an AuthConfig that uses OPA.
  2. The ingress gateway sends the request to the external auth service for an authorization decision.
  3. The external auth service passes the request through to the OPA server to make an authorization decision.
    • If the OPA server is within the cluster, the external auth service can refer to the OPA server by using its Kubernetes service address.
    • If the OPA server is outside the cluster, the external auth service refers to the OPA server by a reachable address, such as on the same private network.
  4. The OPA server loads the OPA config of Rego rules from a bundle in a cloud provider. The OPA server uses these Rego rules to make an authorization decision on the request. You can provide the OPA config via a YAML file during the initial installation, or subsequently in a Kubernetes config map. Note that the request does not trigger loading the rules. You must restart the OPA server each time that you update the OPA config.
  5. The OPA server returns the authorization decision to the external auth service, which returns the authorization decision to the ingress gateway.
  6. The ingress gateway handles the request per the authorization decision.
    • If unauthorized, the ingress gateway denies the request.
    • If authorized, the ingress gateway forwards the request to the destination workload.

Before you begin

  1. Follow the Get started guide to install Gloo Gateway, set up a gateway resource, and deploy the httpbin sample app.

  2. Get the external address of the gateway and save it in an environment variable.

    • Cloud Provider LoadBalancer

        export INGRESS_GW_ADDRESS=$(kubectl get svc -n gloo-system gloo-proxy-http -o jsonpath="{.status.loadBalancer.ingress[0]['hostname','ip']}")
echo $INGRESS_GW_ADDRESS

    • Port-forward for local testing

        kubectl port-forward deployment/gloo-proxy-http -n gloo-system 8080:8080

  3. Download the opa CLI tool.

Create the Rego rules

Create a config map to store the Rego rules that you want to enforce. Later, you create the OPA sidecar to read from the config map.

  1. Create a Rego rule.

      cat <<EOF > policy.rego
package test

default allow = false
allow {
    startswith(input.http_request.path, "/anything")
    input.http_request.method == "GET"
}
allow {
    input.http_request.path == "/status/200"
    any({input.http_request.method == "GET",
        input.http_request.method == "DELETE"
    })
}
EOF

    Review the following table to understand this configuration.

    SettingDescription
    default allow = falseDenies all requests by default.
    allow {...}Allows requests that match two conditions as follows: 1) The path starts with /anything AND the HTTP method is GET; or, 2) the path is exactly /status/200 AND the HTTP method is either GET or DELETE.

  2. Store the Rego rules in a Kubernetes config map.

      kubectl -n gloo-system create configmap opa-config --from-file=policy.rego

Set up the OPA server

Set up your OPA server as a remote server or as a deployment within the cluster.

  1. Deploy the OPA server in the same namespace as your external auth service, such as gloo-system. For steps, you can follow the OPA deployment docs. Make sure to update the deployment with the OPA config map that you previously created. Later, if you want to update the OPA config, see Update OPA config.

    • Currently, the Gloo external auth service does not have a way to send auth details to a protected remote OPA server. Make sure to run the OPA server on a trusted network that your cluster has access to, such as a separate Kubernetes cluster in the same virtual private cloud (VPC) network as your cluster with enabled connectivity.
    • If your cluster limits egress and uses a service mesh, you might use an external service to add the remote OPA server to your cluster’s service mesh.

    Deploy the OPA server in your cluster, such as in the same namespace as your external auth service.

  2. Verify that the OPA server is running.

      kubectl get pods -A -l app=opa

  3. Get the service details, which you use later to refer to the OPA server in the external auth policy.

      kubectl get svc -A -l app=opa

    Example output: Later, you use the OPA service address <service>.<namespace>:<port>, such as http://opa.gloo-system:8181.

      NAMESPACE          NAME   TYPE       CLUSTER-IP     EXTERNAL-IP   PORT(S)          AGE
 gloo-system          opa    NodePort   10.xx.xx.xxx   <none>        8181:32427/TCP   15s

Create the OPA AuthConfig

Create the Gloo Gateway resources to enforce the OPA policy.

  1. Create an AuthConfig with your external authentication rules. The following example configures OPA authentication with the Rego rules that you created earlier.

      kubectl apply -f - <<EOF
apiVersion: enterprise.gloo.solo.io/v1
kind: AuthConfig
metadata:
  name: opa-server
  namespace: httpbin
spec:
  configs:
    - name: opa
      opaServerAuth:
        serverAddr: http://sidecar-uds
        package: test
        ruleName: allow
EOF

    Review the following table to understand this configuration. For more information, see the API reference.

    SettingDescription
    configsThe auth configs to enforce for routes that use this AuthConfig. This example includes one config, opa-server, for the Rego policy that you created earlier in the config map.
    opaServerAuthConfigure the OPA server sidecar authentication details.
    packageRefer to the package name with the Rego rule that you want to enforce. The package name is configured in the config map that you previously mounted to the OPA sidecar. In the example, the package name is test.
    ruleNameSelect the Rego rule that you want to enforce for this OPA external auth policy. The Rego rule is configured in the config map that you previously mounted to the OPA sidecar. In the example, the rule name is allow. For more information about rule names, see the OPA Data API docs.
    serverAddrThe reachable address of the OPA server that you previously retrieved when you deployed the OPA server.

  2. Create a RouteOption resource that refers to the AuthConfig resource that you just created.

      kubectl apply -f- <<EOF
apiVersion: gateway.solo.io/v1
kind: RouteOption
metadata:
  name: opa-server
  namespace: httpbin
spec:
  options:
    extauth:
      configRef:
        name: opa-server
        namespace: httpbin
EOF

  3. Create an HTTPRoute resource for the httpbin app that requires authentication for requests on the extauth.example domain.

      kubectl apply -f- <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: httpbin-opa-server
  namespace: httpbin
spec:
  parentRefs:
  - name: http
    namespace: gloo-system
  hostnames:
    - extauth.example
  rules:
    - filters:
        - type: ExtensionRef
          extensionRef:
            group: gateway.solo.io
            kind: RouteOption
            name: opa-server
      backendRefs:
        - name: httpbin
          port: 8000
EOF

Verify the OPA AuthConfig

Verify that the Rego rules are evaluated by the OPA server and enforced by the external auth service.

  1. Confirm that the AuthConfig’s state is ACCEPTED.

      kubectl get -n httpbin AuthConfig opa-server -o yaml

  2. Send a request to the httpbin app on the extauth.example domain for a path that is not allowed by the OPA policy. Verify that your request is denied and that you get back a 403 HTTP response code.

    • LoadBalancer IP address or hostname:

        curl -v http://$INGRESS_GW_ADDRESS:8080/headers -H "host: extauth.example:8080"

    • Port-forward for local testing:

        curl -v localhost:8080/headers -H "host: extauth.example"

    Example output:

      * Mark bundle as not supporting multiuse
< HTTP/1.1 403 Forbidden
< date: Wed, 05 Jun 2024 14:12:36 GMT
< server: envoy
< content-length: 0
Rejected

  3. Send another request to the httpbin app. This time, you include the /status/200 path that is allowed in the OPA policy. Verify that the request succeeds and that you get back a 200 HTTP response code.

    • LoadBalancer IP address or hostname:

        curl -v http://$INGRESS_GW_ADDRESS:8080/status/200 -H "host: extauth.example:8080"

    • Port-forward for local testing:

        curl -v localhost:8080/status/200 -H "host: extauth.example"

    Example output:

      * Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< access-control-allow-credentials: true
< access-control-allow-origin: *
< date: Wed, 05 Jun 2024 14:18:28 GMT
< content-length: 0
< x-envoy-upstream-service-time: 1
< server: envoy

Cleanup

You can optionally remove the resources that you set up as part of this guide.

  1. Delete the external auth resources that you created.

      kubectl delete authconfig opa-server -n httpbin
kubectl delete routeoption opa-server -n httpbin
kubectl delete httproute httpbin-opa-server -n httpbin

  2. Delete the Rego policy file, and optionally remove the bundles from your cloud storage provider.

      rm rego/policy.rego

  3. Optional: If you no longer need your OPA server, delete it.

      kubectl delete all -l app=opa

Update OPA config

To update OPA config after initially deploying the OPA server sidecar, choose from the following options.