About

Gloo Gateway can deploy an Open Policy Agent (OPA) server to run as a sidecar to its external auth service. After, you are responsible for administering the server per OPA best practices. You might choose this approach for larger scale environments and extended use cases such as bundling and caching.

With bundling, your Rego rules can live as a signed bundle in an external, central location, such as an AWS S3 bucket to meet your internal security requirements. This sidecar approach increases the resources needed in the external auth server pod, but works better at scale and provides more OPA-native support for teams familiar with administering an OPA server. You also get the OPA-Envoy plugin API as part of the Gloo external auth service.

For different setup options such as the default OPA or bring your own OPA, see .

Architecture

The following diagram and the steps in the rest of this guide show how you can set up an OPA server sidecar.

Figure: Architecture for deploying an OPA server sidecar.
Figure: Architecture for deploying an OPA server sidecar.
Figure: Architecture for deploying an OPA server sidecar.
Figure: Architecture for deploying an OPA server sidecar.

  1. A user sends a request that the ingress gateway receives. The request matches a route that is protected by an AuthConfig that uses OPA.
  2. The ingress gateway sends the request to the external auth service for an authorization decision.
  3. The external auth service passes the request through to the OPA server sidecar to make an authorization decision.
  4. The OPA server sidecar loads the OPA config of Rego rules from a bundle in a cloud provider. The OPA server uses these Rego rules to make an authorization decision on the request. You can provide the OPA config via a YAML file during the Helm installation, or subsequently in a Kubernetes config map. Note that the request does not trigger loading the rules. You must restart the OPA server each time that you update the OPA config.
  5. The OPA server returns the authorization decision to the external auth service, which returns the authorization decision to the ingress gateway.
  6. The ingress gateway handles the request per the authorization decision.
    • If unauthorized, the ingress gateway denies the request.
    • If authorized, the ingress gateway forwards the request to the destination workload.

Clear route cache

The Envoy proxy in your sidecar instance keeps a route cache in memory of precomputed routing decisions to help speed up performance. When the sidecar gets a request, it can check the route cache to decide where to send the request. When the route cache is cleared, Gloo recomputes the routing rules. This way, any old and potentially conflicting data from the initial request is cleared.

Before you begin

  1. Follow the Get started guide to install Gloo Gateway, set up a gateway resource, and deploy the httpbin sample app.

  2. Get the external address of the gateway and save it in an environment variable.

    • Cloud Provider LoadBalancer

        export INGRESS_GW_ADDRESS=$(kubectl get svc -n gloo-system gloo-proxy-http -o jsonpath="{.status.loadBalancer.ingress[0]['hostname','ip']}")
echo $INGRESS_GW_ADDRESS

    • Port-forward for local testing

        kubectl port-forward deployment/gloo-proxy-http -n gloo-system 8080:8080

Create the Rego rules

Create a config map to store the Rego rules that you want to enforce. Later, you create the OPA sidecar to read from the config map.

  1. Create a Rego rule.

      cat <<EOF > policy.rego
package test

default allow = false
allow {
    startswith(input.http_request.path, "/anything")
    input.http_request.method == "GET"
}
allow {
    input.http_request.path == "/status/200"
    any({input.http_request.method == "GET",
        input.http_request.method == "DELETE"
    })
}
EOF

    Review the following table to understand this configuration.

    SettingDescription
    default allow = falseDenies all requests by default.
    allow {...}Allows requests that match two conditions as follows: 1) The path starts with /anything AND the HTTP method is GET; or, 2) the path is exactly /status/200 AND the HTTP method is either GET or DELETE.

  2. Store the Rego rules in a Kubernetes config map.

      kubectl -n gloo-system create configmap opa-config --from-file=policy.rego

Enable the OPA server sidecar

Deploy the OPA server sidecar by upgrading the external auth service in your existing Gloo Gateway installation. You can also set up the sidecar when you first install Gloo Gateway.

  1. Save your current version as an environment variable, such as 1.18.2 in the following example.

      export GLOO_VERSION=1.18.2

    Not sure what version you currently have? You can check the image of your gloo deployment, such as with the following command.

      kubectl get deployment -n gloo-system gloo -o jsonpath="{.spec.template.spec.containers[?(@.name=='gloo')].image}"

    Example output: The version is 1.18.2.

      quay.io/solo-io/gloo-ee:1.18.2@sha256:...

  2. Get your current installation values.

      helm get values gloo -n gloo-system -o yaml > gloo-gateway.yaml
open gloo-gateway.yaml

  3. Add or edit the following settings to deploy the OPA server sidecar as an extra container to the external auth service. Note that you load the opa-config ConfigMap as an extra volume to the deployment so that the external auth server has access to your Rego policies. If you make changes to the policies later, you must reload the deployment. For other settings, see the Helm reference docs.

      
global:
  extensions:
    extAuth:
      enabled: true
      deployment:
        extraVolume:
        - name: opa-policy
          configMap:
            name: opa-config
        - name: opa-socket
          emptyDir: {}
        extraVolumeMount:
        - name: opa-socket
          mountPath: /tmp/
        extraContainers:
        - name: opa
          image: openpolicyagent/opa:0.69.0
          args: ["run", "--ignore=.*", "--server", "/policies", "--addr", "unix:///tmp/opa.sock", "--addr", "0.0.0.0:8181"]
          ports:
          - name: http
            containerPort: 8181
          volumeMounts:
          - name: opa-policy
            mountPath: /policies
            readOnly: true
          - name: opa-socket
            mountPath: /tmp/

  4. Upgrade your Helm release with the OPA server sidecar. Be sure to include the OPA config file that you previously created. This Helm upgrade loads the initial OPA config for the OPA server to test an example policy. Later, if you want to update the OPA config, see Update OPA config.

      helm repo update
helm upgrade -i gloo glooe/gloo-ee \
  --namespace gloo-system \
  -f gloo-gateway.yaml \
  --version $GLOO_VERSION

  5. Verify that the external auth service is healthy, with 2 containers ready. If the OPA sidecar is not healthy, the external auth service cannot enter a healthy running state, either. Continue to the next step to check the OPA server.

      kubectl get po -n gloo-system -l gloo=extauth
  
      NAME                                READY   STATUS    RESTARTS   AGE
extauth-6b569b5b74-btxg9   2/2     Running   0          34m

Create the OPA AuthConfig

Create the Gloo Gateway resources to enforce the OPA policy.

  1. Create an AuthConfig with your external authentication rules. The following example configures OPA authentication with the Rego rules that you created earlier.

      kubectl apply -f - <<EOF
apiVersion: enterprise.gloo.solo.io/v1
kind: AuthConfig
metadata:
  name: opa-sidecar
  namespace: httpbin
spec:
  configs:
    - name: opa
      opaServerAuth:
        serverAddr: http://sidecar-uds
        package: test
        ruleName: allow
EOF

    Review the following table to understand this configuration. For more information, see the API reference.

    SettingDescription
    configsThe auth configs to enforce for routes that use this AuthConfig. This example includes two configs, opa-general and opa-apikey, one for each of the Rego bundles that you created earlier.
    opaServerAuthConfigure the OPA server sidecar authentication details.
    packageRefer to the package name with the Rego rule that you want to enforce. The package name is configured in the config map that you previously mounted to the OPA sidecar. In the example, the package name is test.
    ruleNameSelect the Rego rule that you want to enforce for this OPA external auth policy. The Rego rule is configured in the config map that you previously mounted to the OPA sidecar. In the example, the rule name is allow. For more information about rule names, see the OPA Data API docs.

  2. Create a RouteOption resource that refers to the AuthConfig resource that you just created.

      kubectl apply -f- <<EOF
apiVersion: gateway.solo.io/v1
kind: RouteOption
metadata:
  name: opa-sidecar
  namespace: httpbin
spec:
  options:
    extauth:
      configRef:
        name: opa-sidecar
        namespace: httpbin
EOF

  3. Create an HTTPRoute resource for the httpbin app that requires authentication for requests on the extauth.example domain.

      kubectl apply -f- <<EOF
apiVersion: gateway.networking.k8s.io/v1beta1
kind: HTTPRoute
metadata:
  name: httpbin-opa-sidecar
  namespace: httpbin
spec:
  parentRefs:
  - name: http
    namespace: gloo-system
  hostnames:
    - extauth.example
  rules:
    - filters:
        - type: ExtensionRef
          extensionRef:
            group: gateway.solo.io
            kind: RouteOption
            name: opa-sidecar
      backendRefs:
        - name: httpbin
          port: 8000
EOF

Verify the OPA AuthConfig

Verify that the Rego rules are evaluated by the OPA server and enforced by the external auth service.

  1. Confirm that the AuthConfig’s state is ACCEPTED.

      kubectl get -n httpbin AuthConfig opa-sidecar -o yaml

  2. Send a request to the httpbin app on the extauth.example domain for a path that is not allowed by the OPA policy. Verify that your request is denied and that you get back a 403 HTTP response code.

    • LoadBalancer IP address or hostname:

        curl -v http://$INGRESS_GW_ADDRESS:8080/headers -H "host: extauth.example:8080"

    • Port-forward for local testing:

        curl -v localhost:8080/headers -H "host: extauth.example"

    Example output:

      * Mark bundle as not supporting multiuse
< HTTP/1.1 403 Forbidden
< date: Wed, 05 Jun 2024 14:12:36 GMT
< server: envoy
< content-length: 0
Rejected

  3. Send another request to the httpbin app. This time, you include the /status/200 path that is allowed in the OPA policy. Verify that the request succeeds and that you get back a 200 HTTP response code.

    • LoadBalancer IP address or hostname:

        curl -v http://$INGRESS_GW_ADDRESS:8080/status/200 -H "host: extauth.example:8080"

    • Port-forward for local testing:

        curl -v localhost:8080/status/200 -H "host: extauth.example"

    Example output:

      * Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< access-control-allow-credentials: true
< access-control-allow-origin: *
< date: Wed, 05 Jun 2024 14:18:28 GMT
< content-length: 0
< x-envoy-upstream-service-time: 1
< server: envoy

Cleanup

You can optionally remove the resources that you set up as part of this guide.

  1. Delete the resources that you created.

      kubectl delete authconfig opa-sidecar -n httpbin
kubectl delete routeoption opa-sidecar -n httpbin
kubectl delete httproute httpbin-opa-sidecar -n httpbin
kubectl delete configmap opa-config -n gloo-system
rm policy.rego

  2. Repeat the Helm upgrade steps, removing the extra deployment details in the external auth service in the Helm values file.

      
global:
  extensions:
    extAuth:
      enabled: true

  3. Delete the Rego policies that you created, and optionally remove the bundles from your cloud storage provider.

      rm rego/policy.rego