Set rate limits for your API products to control how many requests within a time period are allowed to your API products. The rate limits form a key part of the usage plan for your API products. For more information about the Gloo rate limiting add-on, see Rate limit.

Before you begin

  1. For dynamic rate limiting, install Portal with analytics.

  2. Make sure that the required Gloo components are running. These include the portal server, rate limiter, external auth service, and for dynamic rate limiting, the logs/portal telemetry collector pipeline and portal-related dynamic metadata access log formatting in the Istio operator specification.

      kubectl get pods -A -l app=gloo-mesh-portal-server
kubectl get pods -A -l app=ext-auth-service
kubectl get pods -A -l app=rate-limiter
kubectl get -A istiooperator -l reconciler.mesh.gloo.solo.io/name=istio-lifecycle -o yaml

  3. Create your APIs, including the Gloo ApiDocs that describe the stitched schema.

  4. Bundle your APIs into API products by using a route table.

  5. Get the labels of your routes to use to apply policies to, such as with the example query.

      kubectl get rt -n gloo-mesh-gateways -o=jsonpath='{range .items[*]}[{.metadata.name}, {.spec.http[*].name}, {.spec.http[*].labels}]{"\n"}{end}'

    Example output:

    • The api-example-com-rt route table does not have any route-level labels. To apply policies, you can add labels to those routes.
    • The petstore-rt route table has a usagePlans: dev-portal label on its pets-api, users-api, and store-api routes.
    • The tracks-rt route table has a usagePlans: dev-portal label on its tracks-api route.
      [api-example-com-rt, , ]
[petstore-rt, pets-api users-api store-api, {"usagePlans":"dev-portal"} {"usagePlans":"dev-portal"} {"usagePlans":"dev-portal"} ]
[tracks-rt, tracks-api, {"usagePlans":"dev-portal"}]

Rate limiting options

Gloo Portal implements rate limits by reusing the same policies that you also use for non-Portal routes in Gloo Mesh Gateway. This flexible approach lets you reuse rate limiting resources such as server configurations across your teams and apps.

Review the following approaches to rate limiting for more examples.

  • Basic rate limiting: A basic approach to rate limiting that limits requests to a certain amount per API. These rate limits are bundled into usage plans for tiers such as gold, silver, or bronze that allow differing numbers of requests per plan per user.
  • Dynamic rate limiting: A dynamic approach to rate limiting that limits requests based on portal metadata that is unique to the client, such as usage plans and user IDs. This way, you can set unique rate limits per user for each API product without having to create many separate usage plans and rate limiting resources.

Basic rate limiting

Control how many requests within a time period are allowed to your APIs, such as 100 requests per minute.

  1. For instructions, follow the Set up rate limiting guide in the Protect your APIs tutorial.

  2. Update the RateLimitClientConfig based on the type of external auth policy that you applied: API key or OAuth.

    The following API key example extracts the usage plan from a request header.

      kubectl apply -f - <<EOF
apiVersion: trafficcontrol.policy.gloo.solo.io/v2
kind: RateLimitClientConfig
metadata:
  name: usage-plans
  namespace: gloo-mesh-addons
  labels:
    portal: rate-limit
spec:
  raw:
    rateLimits:
    - setActions:
      - requestHeaders:
          descriptorKey: usagePlan
          headerName: x-solo-plan
      - metadata:
          descriptorKey: userId
          metadataKey:
            key: envoy.filters.http.ext_authz
            path:
              - key: userId
EOF

    The following OAuth example extracts the usage plan from the external auth Envoy filter that the OAuth policy requires.

      kubectl apply -f - <<EOF
apiVersion: trafficcontrol.policy.gloo.solo.io/v2
kind: RateLimitClientConfig
metadata:
  name: usage-plans
  namespace: gloo-mesh-addons
  labels:
    portal: rate-limit
spec:
  raw:
    rateLimits:
    - setActions:
      - metadata:
          descriptorKey: usagePlan
          metadataKey:
            key: envoy.filters.http.ext_authz
            path:
              - key: usagePlan
      - metadata:
          descriptorKey: userId
          metadataKey:
            key: envoy.filters.http.ext_authz
            path:
              - key: userId
EOF

Dynamic rate limiting

Set up dynamic rate limiting for users of the developer portal. For example, you might want different user IDs to have unique rate limits for each API product.

As shown in the Basic rate limiting section, one option is to create separate usage plans and rate limiting resources for each user that has the rate limits you want. This approach, however, becomes difficult to manage at scale well. For example, usage plans typically represent pricing tiers, such as gold, silver, or bronze tier, and you might not want to manage different pricing tiers per user.

Instead, you can set up Gloo Mesh Gateway to use rate limit overrides with the metadata in client-initiated requests to enforce dynamic rate limiting. The rate limiting is called “dynamic,” because it is based on the dynamic metadata that the external auth filter in the Envoy proxy gets from the external auth module, such as the API key. You can use this dynamic metadata to track portal-specific metadata such as usage plans, user IDs, and the rate limit. Then, you can write Rego policies for the Open Policy Agent (OPA) Envoy API plugin that is part of Gloo’s external auth service to enforce the dynamic rate limit that you set.

Step 1: Add rate limiting information to the request metadata

To add Envoy dynamic metadata to requests, you configure the OPA auth module as part of a Gloo external auth policy. To enforce OPA auth, you write a Rego policy to evaluate requests that include the dynamic metadata. For more information about external auth and OPA with Gloo, see OPA.

  1. Create an OPA Rego policy file. The following policy:

    • Allows requests that include the test header api-key=apikey1.
    • Denies requests with a 401 response code if the request does not include the header or reaches the rate limit.
    • Checks for a disable-dynamic-rl header so that you can disable dynamic rate limiting for quick testing. Note that in production scenarios, you would not include such logic to disable rate limiting based on the header values. But for testing purposes, this header can help you confirm that the policy works as expected.
    • Sets dynamic metadata so that the gold usage plan for the user1 user ID gets a new rate limit of 7 requests per minute (as opposed to 5 per minute as configured by the basic rate limiting setup).
    • Returns a result payload with new response headers such as x-validated-by and x-client-only, removes the api-key header, and customizes the body text for rejected results.
      cat <<EOF > portal-policy.rego
package test
import future.keywords.if
default allow = false
allow if {
  input.http_request.headers["api-key"] = "apikey1"
}
http_status := 200 if {
  allow
}
http_status := 401 if {
  not allow
}
default disable_dynamic_rate_limits = false
disable_dynamic_rate_limits if {
  input.http_request.headers["disable-dynamic-rl"] = "true"
}
dynamic_metadata["usagePlan"] := "gold"
dynamic_metadata["userId"] := "user1"
dynamic_metadata["rateLimit"] := {
    "requests_per_unit": 7,
    "unit": "MINUTE"
} if not disable_dynamic_rate_limits
result["dynamic_metadata"] := dynamic_metadata
# allow/allowed
result["allow"] := allow
result["http_status"] := http_status
result["headers"]:= {
  "x-validated-by": "opa-dynamic-rl-checkpoint"
}
result["response_headers_to_add"]:= {
  "x-client-only": "visible"
}
result["request_headers_to_remove"]:= ["api-key"]
result["body"] := "Request does not have valid API key, or exceeded the rate limit."
EOF

  2. Store the OPA policy in a Kubernetes config map in the workload cluster that you want to create the external auth policy in.

      kubectl -n gloo-mesh-addons create configmap opa-envoy-plugin-api-policy --from-file=portal-policy.rego

  3. Create an external auth policy to apply OPA protection to your APIs.

The following policy:
  • Applies to routes with the usagePlans: dev-portal label, such as the Tracks API.
  • Refers to the default Gloo external auth server in the gloo-mesh-addons namespace.
  • Configures an OPA module that refers to the Rego policy that you created in the previous steps.
  kubectl apply -f - <<EOF
apiVersion: security.policy.gloo.solo.io/v2
kind: ExtAuthPolicy
metadata:
  name: ext-auth-opa-policy
  namespace: gloo-mesh-addons
spec:
  applyToRoutes:
  - route:
      labels:
        usagePlans: dev-portal
  config:
    server:
      cluster: $CLUSTER_NAME
      name: ext-auth-server
      namespace: gloo-mesh-addons
    glooAuth:
      configs:
        - name: opa_auth
          opaAuth:
            modules:
            - name: opa-envoy-plugin-api-policy
              namespace: gloo-mesh-addons
            query: "data.test.result"
EOF

Step 2: Set up dynamic rate limit overrides

Set up rate limit overrides to use the dynamic rate limiting information that you added to the request metadata in the previous section.

  1. Select the rate limiting server to use. The example uses the default rate limiting server that you created when you installed Gloo Platform. For more information, see Rate limit server settings. If you already created a RateLimitServerSettings, you can skip this step.

      kubectl apply -f - << EOF
apiVersion: admin.gloo.solo.io/v2
kind: RateLimitServerSettings
metadata:
  name: rl-server
  namespace: gloo-mesh-addons
  labels:
    portal: rate-limit
spec:
  destinationServer:
    port:
      name: grpc
    ref:
      cluster: $CLUSTER_NAME
      name: rate-limiter
      namespace: gloo-mesh-addons
EOF

  2. Create a rate limit client config to define a limit from the dynamic metadata in the opa_auth module that you previously configured. For more information, see Rate limit client config.

      kubectl apply -f - <<EOF
apiVersion: trafficcontrol.policy.gloo.solo.io/v2
kind: RateLimitClientConfig
metadata:
  name: usage-plans-opa
  namespace: gloo-mesh-addons
  labels:
    portal: rate-limit
spec:
  raw:
    rateLimits:
    - setActions:
      - metadata:
          descriptorKey: usagePlan
          metadataKey:
            key: envoy.filters.http.ext_authz
            path:
              - key: opa_auth
              - key: usagePlan
      - metadata:
          descriptorKey: userId
          metadataKey:
            key: envoy.filters.http.ext_authz
            path:
              - key: opa_auth
              - key: userId
      limit:
        dynamicMetadata:
          metadataKey:
            key: envoy.filters.http.ext_authz
            path:
              - key: opa_auth
              - key: rateLimit
EOF

  3. Use the RateLimitServerConfig to define the values of the descriptors that are the usage plan names. These descriptor values match the usagePlan descriptor key that you configured in the RateLimitClientConfig. The userId and usagePlan details are extracted from the details in the request header that is required by the external auth policy. If you already created a RateLimitServerConfig, you can skip this step.

    The following rate limit server config example creates three usage plans as follows:

    • Bronze: Requests to your APIs are limited to 1 per minute.
    • Silver: Requests to your APIs are limited to 3 per minute.
    • Gold: Requests to your APIs are limited to 5 per minute.

    For more information, see Rate limit server config.

      kubectl apply -f - << EOF
apiVersion: admin.gloo.solo.io/v2
kind: RateLimitServerConfig
metadata:
  name: usage-plans
  namespace: gloo-mesh-addons
  labels:
    portal: rate-limit
spec:
  destinationServers:
  - port:
      name: grpc
    ref:
      cluster: $CLUSTER_NAME
      name: rate-limiter
      namespace: gloo-mesh-addons
  raw:
    setDescriptors:
      - simpleDescriptors:
          - key: userId
          - key: usagePlan
            value: bronze
        rateLimit:
          requestsPerUnit: 1
          unit: MINUTE
      - simpleDescriptors:
          - key: userId
          - key: usagePlan
            value: silver
        rateLimit:
          requestsPerUnit: 3
          unit: MINUTE
      - simpleDescriptors:
          - key: userId
          - key: usagePlan
            value: gold
        rateLimit:
          requestsPerUnit: 5
          unit: MINUTE
EOF

  4. Create a rate limit policy to apply the new rate limit client config to your APIs. You can reuse the RateLimitServerSettings that selects the rate limiter to use and the RateLimitServerConfig that configures the usage policies.

      kubectl apply -f - << EOF
apiVersion: trafficcontrol.policy.gloo.solo.io/v2
kind: RateLimitPolicy
metadata:
  name: tracks-rate-limit-opa
  namespace: default
  labels:
    portal: rate-limit
spec:
  applyToRoutes:
  - route:
      labels:
        usagePlans: dev-portal
  config:
    ratelimitServerConfig:
      name: usage-plans
      namespace: gloo-mesh-addons
      cluster: $CLUSTER_NAME
    ratelimitClientConfig:
      name: usage-plans-opa
      namespace: gloo-mesh-addons
      cluster: $CLUSTER_NAME
    serverSettings:
      name: rl-server
      namespace: gloo-mesh-addons
      cluster: $CLUSTER_NAME
    phase:
      postAuthz:
        priority: 1
EOF

Step 3: Verify dynamic rate limiting

Verify how dynamic rate limiting and basic rate limiting can work together to protect your API products.

  1. Verify that your rate limiting is in place by repeating the following request 8 times in a row. Because your user ID is limited by the new dynamic rate limit of 7 requests per minute, the eighth time your request is blocked.

      curl -v -H 'api-key: apikey1' --resolve api.example.com:80:${INGRESS_GW_IP} http://api.example.com/trackapi/tracks

    Example output:

      < HTTP/1.1 401
< x-envoy-ratelimited: true
< Request does not have valid API key, or exceeded the rate limit.

  2. Verify that your request uses the basic rate limiting when dynamic rate limiting is disabled. You can use the special disable-dynamic-rl: true header that you configured in the Rego policy. Without dynamic rate limiting, your user ID is limited to the gold usage plan’s limit of 5 requests per minute. The sixth time your request is blocked. Note that this step requires you to have set up API key external auth already.

      curl -v -H 'api-key: apikey1' -H 'disable-dynamic-rl: true' --resolve api.example.com:80:${INGRESS_GW_IP} http://api.example.com/trackapi/tracks

    Example output:

      < HTTP/1.1 429 Too Many Requests
< x-envoy-ratelimited: true