Navigation :

Dynamic rate limiting

Set up dynamic rate limiting for users of the developer portal.

For example, you might want different user IDs to have unique rate limits for each API product.

One option that you have is to create separate usage plans and rate limiting resources for each user that has the rate limits you want. This approach, however, becomes difficult to manage at scale well. For example, usage plans typically represent pricing tiers, such as gold, silver, or bronze tier, and you might not want to manage different pricing tiers per user.

Instead, you can set up Gloo Mesh Gateway to use rate limit overrides with the metadata in client-initiated requests to enforce dynamic rate limiting. The rate limiting is called “dynamic,” because it is based on the dynamic metadata that the external auth filter in the Envoy proxy gets from the external auth module, such as the API key. You can use this dynamic metadata to track portal-specific metadata such as usage plans, user IDs, and the rate limit. Then, you can write Rego policies for the Open Policy Agent (OPA) Envoy API plugin that is part of Gloo's external auth service to enforce the dynamic rate limit that you set.

Before you begin

Make sure that you have the following components. If not, upgrade your Gloo installment, such as by following the Set up Portal by upgrading Gloo Platform guide.

The Gloo portal server.
The Gloo external auth service so that you can use the OPA Envoy API plugin.
The logs/portal telemetry collector pipeline and portal-related dynamic metadata access log formatting in the Istio operator specification.

kubectl get pods -A -l app=gloo-mesh-portal-server
kubectl get pods -A -l app=ext-auth-service
kubectl get -A istiooperator -l reconciler.mesh.gloo.solo.io/name=istio-lifecycle -o yaml

Step 1: Add rate limiting information to the request metadata

To add Envoy dynamic metadata to requests, you configure the OPA auth module as part of a Gloo external auth policy. To enforce OPA auth, you write a Rego policy to evaluate requests that include the dynamic metadata. For more information about external auth and OPA with Gloo, see OPA.

For help writing Rego policies, refer to the OPA policy language docs. If you use the OPA-Envoy plugin API, the OPA docs has a policy primer. For example, you might create an output document to organize the results of a decision or to gather all the metadata of an object like an API key. Then, you can test your Rego policies in the Rego playground.

Create an OPA Rego policy file. The following policy:

Allows requests that include the test header api-key=apikey1.
Denies requests with a 401 response code if the request does not include the header or reaches the rate limit.
Checks for a disable-dynamic-rl header so that you can disable dynamic rate limiting for quick testing. Note that in production scenarios, you would not include such logic to disable rate limiting based on the header values. But for testing purposes, this header can help you confirm that the policy works as expected.
Sets dynamic metadata so that the gold usage plan for the user1 user ID gets a new rate limit of 7 requests per minute (as opposed to 5 per minute as configured by the basic rate limiting setup).
Returns a result payload with new response headers such as x-validated-by and x-client-only, removes the api-key header, and customizes the body text for rejected results.

cat <<EOF > portal-policy.rego
package test
import future.keywords.if
default allow = false
allow if {
  input.http_request.headers["api-key"] = "apikey1"
}
http_status := 200 if {
  allow
}
http_status := 401 if {
  not allow
}
default disable_dynamic_rate_limits = false
disable_dynamic_rate_limits if {
  input.http_request.headers["disable-dynamic-rl"] = "true"
}
dynamic_metadata["usagePlan"] := "gold"
dynamic_metadata["userId"] := "user1"
dynamic_metadata["rateLimit"] := {
    "requests_per_unit": 7,
    "unit": "MINUTE"
} if not disable_dynamic_rate_limits
result["dynamic_metadata"] := dynamic_metadata
# allow/allowed
result["allow"] := allow
result["http_status"] := http_status
result["headers"]:= {
  "x-validated-by": "opa-dynamic-rl-checkpoint"
}
result["response_headers_to_add"]:= {
  "x-client-only": "visible"
}
result["request_headers_to_remove"]:= ["api-key"]
result["body"] := "Request does not have valid API key, or exceeded the rate limit."
EOF

Store the OPA policy in a Kubernetes config map in the workload cluster that you want to create the external auth policy in.
```
kubectl -n gloo-mesh-addons create configmap opa-envoy-plugin-api-policy --from-file=portal-policy.rego
```

Create an external auth policy to apply OPA protection to your APIs.

You cannot have multiple external auth policies apply to the same route. In case of a conflict, the first policy takes precedence. You might have to delete earlier policies before you can apply the following policy.

The following policy:

Applies to routes with the usagePlans: dev-portal label, such as the Tracks API.
Refers to the default Gloo external auth server in the gloo-mesh-addons namespace.
Configures an OPA module that refers to the Rego policy that you created in the previous steps.

kubectl apply -f - <<EOF
apiVersion: security.policy.gloo.solo.io/v2
kind: ExtAuthPolicy
metadata:
  name: ext-auth-opa-policy
  namespace: gloo-mesh-addons
spec:
  applyToRoutes:
  - route:
      labels:
        usagePlans: dev-portal
  config:
    server:
      cluster: $CLUSTER_NAME
      name: ext-auth-server
      namespace: gloo-mesh-addons
    glooAuth:
      configs:
        - name: opa_auth
          opaAuth:
            modules:
            - name: opa-envoy-plugin-api-policy
              namespace: gloo-mesh-addons
            query: "data.test.result"
EOF

Step 2: Set up dynamic rate limit overrides

Set up rate limit overrides to use the dynamic rate limiting information that you added to the request metadata in the previous section.

Select the rate limiting server to use. The example uses the default rate limiting server that you created when you installed Gloo Platform. For more information, see Rate limit server settings. If you already created a RateLimitServerSettings, you can skip this step.

kubectl apply -f - << EOF
apiVersion: admin.gloo.solo.io/v2
kind: RateLimitServerSettings
metadata:
  name: rl-server
  namespace: gloo-mesh-addons
  labels:
    portal: rate-limit
spec:
  destinationServer:
    port:
      name: grpc
    ref:
      cluster: $CLUSTER_NAME
      name: rate-limiter
      namespace: gloo-mesh-addons
EOF

Create a rate limit client config to define a limit from the dynamic metadata in the opa_auth module that you previously configured. For more information, see Rate limit client config.

kubectl apply -f - <<EOF
apiVersion: trafficcontrol.policy.gloo.solo.io/v2
kind: RateLimitClientConfig
metadata:
  name: usage-plans-opa
  namespace: gloo-mesh-addons
  labels:
    portal: rate-limit
spec:
  raw:
    rateLimits:
    - setActions:
      - metadata:
          descriptorKey: usagePlan
          metadataKey:
            key: envoy.filters.http.ext_authz
            path:
              - key: opa_auth
              - key: usagePlan
      - metadata:
          descriptorKey: userId
          metadataKey:
            key: envoy.filters.http.ext_authz
            path:
              - key: opa_auth
              - key: userId
      limit:
        dynamicMetadata:
          metadataKey:
            key: envoy.filters.http.ext_authz
            path:
              - key: opa_auth
              - key: rateLimit
EOF

Use the RateLimitServerConfig to define the values of the descriptors that are the usage plan names. These descriptor values match the usagePlan descriptor key that you configured in the RateLimitClientConfig. The userId and usagePlan details are extracted from the details in the request header that is required by the external auth policy. If you already created a RateLimitServerConfig, you can skip this step.

The following rate limit server config example creates three usage plans as follows:

Bronze: Requests to your APIs are limited to 1 per minute.
Silver: Requests to your APIs are limited to 3 per minute.
Gold: Requests to your APIs are limited to 5 per minute.

For more information, see Rate limit server config.

kubectl apply -f - << EOF
apiVersion: admin.gloo.solo.io/v2
kind: RateLimitServerConfig
metadata:
  name: usage-plans
  namespace: gloo-mesh-addons
  labels:
    portal: rate-limit
spec:
  destinationServers:
  - port:
      name: grpc
    ref:
      cluster: $CLUSTER_NAME
      name: rate-limiter
      namespace: gloo-mesh-addons
  raw:
    setDescriptors:
      - simpleDescriptors:
          - key: userId
          - key: usagePlan
            value: bronze
        rateLimit:
          requestsPerUnit: 1
          unit: MINUTE
      - simpleDescriptors:
          - key: userId
          - key: usagePlan
            value: silver
        rateLimit:
          requestsPerUnit: 3
          unit: MINUTE
      - simpleDescriptors:
          - key: userId
          - key: usagePlan
            value: gold
        rateLimit:
          requestsPerUnit: 5
          unit: MINUTE
EOF

Create a rate limit policy to apply the new rate limit client config to your APIs. You can reuse the RateLimitServerSettings that selects the rate limiter to use and the RateLimitServerConfig that configures the usage policies.

kubectl apply -f - << EOF
apiVersion: trafficcontrol.policy.gloo.solo.io/v2
kind: RateLimitPolicy
metadata:
  name: tracks-rate-limit-opa
  namespace: default
  labels:
    portal: rate-limit
spec:
  applyToRoutes:
  - route:
      labels:
        usagePlans: dev-portal
  config:
    ratelimitServerConfig:
      name: usage-plans
      namespace: gloo-mesh-addons
      cluster: $CLUSTER_NAME
    ratelimitClientConfig:
      name: usage-plans-opa
      namespace: gloo-mesh-addons
      cluster: $CLUSTER_NAME
    serverSettings:
      name: rl-server
      namespace: gloo-mesh-addons
      cluster: $CLUSTER_NAME
    phase:
      postAuthz:
        priority: 1
EOF

Step 3: Verify dynamic rate limiting

Verify how dynamic rate limiting and basic rate limiting can work together to protect your API products.

Verify that your rate limiting is in place by repeating the following request 8 times in a row. Because your user ID is limited by the new dynamic rate limit of 7 requests per minute, the eighth time your request is blocked.
```
curl -v -H 'api-key: apikey1' --resolve api.example.com:80:${INGRESS_GW_IP} http://api.example.com/trackapi/tracks
```
Example output:
```
< HTTP/1.1 401
< x-envoy-ratelimited: true
< Request does not have valid API key, or exceeded the rate limit.
```
Verify that your request uses the basic rate limiting when dynamic rate limiting is disabled. You can use the special disable-dynamic-rl: true header that you configured in the Rego policy. Without dynamic rate limiting, your user ID is limited to the gold usage plan's limit of 5 requests per minute. The sixth time your request is blocked. Note that this step requires you to have set up API key external auth already.
```
curl -v -H 'api-key: apikey1' -H 'disable-dynamic-rl: true' --resolve api.example.com:80:${INGRESS_GW_IP} http://api.example.com/trackapi/tracks
```
Example output:
```
< HTTP/1.1 429 Too Many Requests
< x-envoy-ratelimited: true
```

Next steps

Configure the developer portal.

When you are done with trying out Portal, you can clean up all of the resources that you created.