Control the rate of requests to destinations within the service mesh. The following example shows you how to create a multicluster rate limit policy that applies to a destination or route, based on a generic key.

By creating a virtual destination for the rate limiter service in each cluster, you ensure that rate limiting applies to multicluster traffic even in the case of a local rate limiter failure.

For more in-depth examples of the Envoy and Set-style rate limiting APIs, see More rate limit policy examples.

Before you begin

  1. Complete the multicluster getting started guide to set up the following testing environment.

    • Three clusters along with environment variables for the clusters and their Kubernetes contexts.
    • The Gloo meshctl CLI, along with other CLI tools such as kubectl and istioctl.
    • The Gloo management server in the management cluster, and the Gloo agents in the workload clusters.
    • Istio installed in the workload clusters.
    • A simple Gloo workspace setup.
  2. Install Bookinfo and other sample apps.

Step 1: Set up multicluster rate limiter

Set up the rate limiter in your multicluster environment.

  1. Make sure that the rate limiting service is installed and running in each cluster. If not, upgrade your workload cluster to include the rate limiter.

      kubectl get pods --context $REMOTE_CONTEXT1 -A -l app=rate-limiter
kubectl get pods --context $REMOTE_CONTEXT2 -A -l app=rate-limiter

  2. Decide whether to bring your own Redis backing database for the rate limiter. For setup instructions, see the Redis guides.

    • Bring your own external Redis: The rate limiter on each cluster shares the same external Redis database. Then, the rate limit counter is shared across clusters. This way, when traffic is load balanced across clusters, the same rate limit is enforced.
    • Use the built-in or local Redis: Each cluster’s rate limiter uses its own local Redis database. The rate limit counters are unique to each cluster. This way, when traffic is load balanced across clusters, cluster-specific rate limits are enforced.
    • Example scenario: Flip through the following scenario and the impact of external vs. local Redis.
      • The rate limit is 3 requests per day.
      • The virtual destination load balances requests to the rate limit server across clusters.
      • The destination app’s sidecar that enforces the rate limit policy forwards the request for its rate limited destination to the rate limit server via the virtual destination.
      • Two requests are sent to the destinations in each cluster.

    When the rate limiters are set up to share an external Redis:

    • In cluster-1: Both requests succeed.
    • In cluster-2: The first request succeeds, but the second request fails due to exceeding the 3 requests per day limit. The second request is a total of 4 counts shared across the rate limiters.
    Figure: Rate limit setup with a shared, external Redis
    Figure: Rate limit setup with a shared, external Redis
    Figure: Rate limit setup with a shared, external Redis
    Figure: Rate limit setup with a shared, external Redis

    When the rate limiters use their own local Redis, such as in the default setup:

    • In cluster-1: Both requests succeed.
    • In cluster-2: Both requests succeed.
    • Each cluster has 1 more request left for the day until the limit of 3 requests per day is reached. The counts are separate due to the rate limiters using local Redis.
    Figure: Rate limit setup with a separate, local Redis
    Figure: Rate limit setup with a separate, local Redis
    Figure: Rate limit setup with a separate, local Redis
    Figure: Rate limit setup with a separate, local Redis

Step 2: Create the virtual destination

Create a virtual destination for the rate limiters to span across clusters.

  1. Create the virtual destination. The following example forwards traffic on the rate-limiter-service.vd host to the rate limiter Kubernetes service in the gloo-mesh namespace. For more information, see the VirtualDestination API docs.

      kubectl apply --context $REMOTE_CONTEXT1 -f -<< EOF
apiVersion: networking.gloo.solo.io/v2
kind: VirtualDestination
metadata:
  annotations:
    cluster.solo.io/cluster: ""
  name: rate-limiter-vd
  namespace: gloo-mesh
spec:
  hosts:
  - rate-limiter-service.vd
  ports:
  - number: 8083
    protocol: GRPC
  services:
  - name: rate-limiter
    namespace: gloo-mesh
EOF

  2. Optional: To prefer the local rate limiter when receiving requests from the same cluster, create a failover policy. For more information, see the Failover guide.

      kubectl apply --context $REMOTE_CONTEXT1 -f -<< EOF
apiVersion: resilience.policy.gloo.solo.io/v2
kind: FailoverPolicy
metadata:
  annotations:
    cluster.solo.io/cluster: ""
  name: locality-based-failover
  namespace: gloo-mesh
spec:
  applyToDestinations:
  - kind: VIRTUAL_DESTINATION
    port:
      number: 8083
    selector:
      name: rate-limiter-vd
      namespace: gloo-mesh
      cluster: $REMOTE_CLUSTER1
  config:
    localityMappings:
    - from:
        region: us-east
      to:
      - region: us-west
EOF

Step 3: Create the rate limit policy

Create the resources to configure the rate limiter. Make sure to refer to the virtual destination of the rate limiter that you created in the previous step.

  1. Create a rate limit server config with the rate limiting rules that the server accepts. The following example allows 3 requests per day. For more information, see Rate limit server setup.

      kubectl apply --context $REMOTE_CONTEXT1 -f -<< EOF
apiVersion: admin.gloo.solo.io/v2
kind: RateLimitServerConfig
metadata:
  annotations:
    cluster.solo.io/cluster: ""
  name: rl-server-config
  namespace: gloo-mesh
spec:
  destinationServers:
  - kind: VIRTUAL_DESTINATION
    port:
      number: 8083
    ref:
      cluster: $REMOTE_CLUSTER1
      name: rate-limiter-vd
      namespace: gloo-mesh
  raw:
    descriptors:
    - key: generic_key
      rateLimit:
        requestsPerUnit: 3
        unit: DAY
      value: counter
EOF

  2. Create a rate limit server settings resource to control how clients connect to the rate limit server. Notice that the destination service is the virtual destination that you previously set up, which matches the destination server in the rate limit config server. For more information, see Rate limit server setup.

      kubectl apply --context $REMOTE_CONTEXT1 -f -<< EOF
apiVersion: admin.gloo.solo.io/v2
kind: RateLimitServerSettings
metadata:
  annotations:
    cluster.solo.io/cluster: ""
  name: rl-server
  namespace: bookinfo
spec:
  destinationServer:
    kind: VIRTUAL_DESTINATION
    port:
      number: 8083
    ref:
      cluster: $REMOTE_CLUSTER1
      name: rate-limiter-vd
      namespace: gloo-mesh
EOF

  3. Create a rate limit client config to set up the rate limiting actions to take. The following example sets up a counter for each descriptor. For more information, see Rate limit server setup.

      kubectl apply --context $REMOTE_CONTEXT1 -f -<< EOF
apiVersion: trafficcontrol.policy.gloo.solo.io/v2
kind: RateLimitClientConfig
metadata:
  annotations:
    cluster.solo.io/cluster: ""
  name: rl-client-config
  namespace: bookinfo
spec:
  raw:
    rateLimits:
    - actions:
      - genericKey:
          descriptorValue: counter
EOF

  4. Create the rate limit policy. The policy’s config selects the rate limit resources that you created in the previous steps. The policy also selects the reviews destination. To select a route in a route table instead, change the applyToDestinations selector to applyToRoutes.

      kubectl apply --context $REMOTE_CONTEXT1 -f -<< EOF
apiVersion: trafficcontrol.policy.gloo.solo.io/v2
kind: RateLimitPolicy
metadata:
  annotations:
    cluster.solo.io/cluster: ""
  name: rl-policy
  namespace: bookinfo
spec:
  applyToDestinations:
  - port:
      number: 9080
    selector:
      labels:
        app: reviews
  config:
    ratelimitClientConfig:
      name: rl-client-config
    ratelimitServerConfig:
      name: rl-server-config
      namespace: gloo-mesh
    serverSettings:
      name: rl-server
EOF

Step 4: Verify the rate limit

Now that the rate limiter is set up and configured to allow 3 requests per day, try it out.

  1. From cluster-1, send a request to the reviews destination. Because the rate limit policy applies to the reviews destination, the request is sent to the virtual destination of the rate limiter for processing. The request succeeds, as it does not exceed the rate limit.

      kubectl exec --context $REMOTE_CONTEXT1 -n bookinfo deploy/productpage-v1 -c curl -- curl -I http://reviews:9080/reviews/1 -v

    Example output:

      * Connected to reviews (34.xxx.xxx.xxx) port 9080 (#0)
...
HTTP/1.1 200 OK

  2. Scale down the rate limiter in cluster-1. Because you created a virtual destination for the rate limiter services across clusters, the rate limiter in cluster-2 will still make sure that requests are rate limited.

      kubectl scale deploy --context $REMOTE_CONTEXT1 -n gloo-mesh rate-limiter --replicas=0

  3. Repeat the request a few more times. Now, the rate limiter in cluster-2 enforces the rate limit.

      kubectl exec --context $REMOTE_CONTEXT1 -n bookinfo deploy/productpage-v1 -c curl -- curl -I http://reviews:9080/reviews/1 -v

    Depending on the setup of the rate limiter’s backing Redis database, your request is rate limited as follows:

    • Shared, external Redis: The request fails on the third repeat (fourth total attempt), because it exceeds the shared count of 3 per day.
    • Local Redis: The request fails on the fourth repeat, because it exceeds the local rate limit of 3 per day for that cluster.

    Example output:

      429 Too Many Requests

Cleanup

You can optionally remove the resources that you set up as part of this guide.

  1. Scale back up the rate limiter in cluster-1.

      kubectl scale deploy --context $REMOTE_CONTEXT1 -n gloo-mesh rate-limiter --replicas=1

  2. Delete the resources that you created in this guide.

      kubectl --context $REMOTE_CONTEXT1 -n gloo-mesh delete RateLimitServerConfig rl-server-config
kubectl --context $REMOTE_CONTEXT1 -n bookinfo delete RateLimitServerSettings rl-server
kubectl --context $REMOTE_CONTEXT1 -n bookinfo delete RateLimitClientConfig rl-client-config
kubectl --context $REMOTE_CONTEXT1 -n bookinfo delete RateLimitPolicy rl-policy
kubectl --context $REMOTE_CONTEXT1 -n gloo-mesh delete VirtualDestination rate-limiter-vd
kubectl --context $REMOTE_CONTEXT1 -n gloo-mesh delete FailoverPolicy locality-based-failover