Skip to content
Latest (currently 2026.10.2) has the newest features, bug fixes, and CVE patches of Solo Enterprise for agentgateway.

For the complete documentation index, see llms.txt. Markdown versions of all docs pages are available by appending .md to any docs URL.

Global rate limiting

Page as Markdown

Apply distributed rate limits across multiple agentgateway replicas using the enterprise rate limit service.

Apply distributed rate limits across multiple agentgateway replicas using the built-in enterprise rate limit service.

About

Global rate limiting coordinates rate limits across multiple agentgateway proxy replicas. Unlike local rate limiting, which runs independently on each proxy replica, global rate limiting provides:

  • Shared counters across all proxy replicas
  • Consistent enforcement regardless of which replica receives the request
  • Descriptor-based limits that can extract multiple request attributes using CEL expressions
  • Flexible targeting by user, API key, IP address, path, or any combination

Global rate limiting is essential when running multiple proxy replicas and you need to enforce a single quota across the entire fleet — for example, “100 requests per minute per user” should apply to the sum of requests across all replicas, not 100 per replica.

Solo Enterprise for agentgateway includes a built-in rate limit service, so you do not need to deploy your own Redis or rate limit service infrastructure. Global rate limiting requires two components:

  1. RateLimitConfig: A custom resource (ratelimit.solo.io/v1alpha1) that defines the rate limit rules, including descriptors, limits, and CEL-based actions.

  2. EnterpriseAgentgatewayPolicy with entRateLimit: References one or more RateLimitConfig resources to apply the limits to specific routes or the entire gateway.

Note

To use a BYO rate limit service instead of the built-in enterprise rate limit service, see the agentgateway docs.

For rate limiter and Redis connections and ports, see Rate limiting and budgets deployment.

Request flow

Global rate limiting works as follows:

  1. A CEL expression in the RateLimitConfig extracts request attributes (such as client IP, user ID, or path)
  2. The gateway sends these descriptor key-value pairs to the enterprise rate limit service
  3. The rate limit service matches the descriptors against the RateLimitConfig rules and checks the counter
  4. If the limit is exceeded, the gateway returns a 429 response to the client; otherwise, the request proceeds to the backend

Common CEL expressions

Review the following common CEL expressions that you might find useful when creating your rate limit policies.

DescriptorCEL ExpressionDescription
Client IPsource.addressSource IP address of the client.
Request pathrequest.pathThe request path (such as /api/v1/users).
Request methodrequest.methodHTTP method (GET, POST, etc.).
Header valuerequest.headers["header-name"]Extract a specific header. Case-insensitive header name.
JWT claimclaims["sub"]Extract a claim from a validated JWT (requires JWT auth policy).
Static value"constant"Use a constant string for categorization (such as service tier).

For an overview of supported CEL expressions, see the agentgateway docs.

Bring your own Redis

By default, the rate limiter stores counters in an in-cluster Redis instance, the ext-cache-enterprise-agentgateway extension. This instance does not use persistent storage, so counters reset when its pod is rescheduled. For production, use an external Redis instance. For your options and the steps, see Use an external Redis.

Before you begin

  1. Set up an agentgateway proxy.
  2. Set up access to the OpenAI LLM provider.

Rate limit by client IP

Limit requests based on the client’s source IP address.

  1. Create a RateLimitConfig that limits each client IP to 10 requests per minute.

    kubectl apply -f- <<EOF
    apiVersion: ratelimit.solo.io/v1alpha1
    kind: RateLimitConfig
    metadata:
      name: ip-rate-limit
      namespace: agentgateway-system
    spec:
      raw:
        descriptors:
        - key: remote_address
          rateLimit:
            requestsPerUnit: 10
            unit: MINUTE
        rateLimits:
        - actions:
          - cel:
              expression: 'source.address'
              key: "remote_address"
    EOF
  2. Apply the rate limit by using an EnterpriseAgentgatewayPolicy resource. The following example targets all routes through the gateway.

    kubectl apply -f- <<EOF
    apiVersion: enterpriseagentgateway.solo.io/v1alpha1
    kind: EnterpriseAgentgatewayPolicy
    metadata:
      name: ip-rate-limit
      namespace: agentgateway-system
    spec:
      targetRefs:
      - group: gateway.networking.k8s.io
        kind: Gateway
        name: agentgateway-proxy
      traffic:
        entRateLimit:
          global:
            rateLimitConfigRefs:
            - name: ip-rate-limit
    EOF
  3. Verify that the policy is attached.

    kubectl get EnterpriseAgentgatewayPolicy ip-rate-limit -n agentgateway-system -o yaml

    Both Accepted and Attached conditions must be True.

  4. Send 15 requests and verify that the rate limit is enforced after 10 requests.

    for i in $(seq 1 15); do
      STATUS=$(curl -s -o /dev/null -w "%{http_code}" \
        http://$INGRESS_GW_ADDRESS/openai \
        -H content-type:application/json \
        -d '{"model": "gpt-3.5-turbo", "messages": [{"role": "user", "content": "Hi"}]}')
      echo "Request $i: HTTP $STATUS"
    done
    for i in $(seq 1 15); do
      STATUS=$(curl -s -o /dev/null -w "%{http_code}" \
        localhost:8080/openai \
        -H content-type:application/json \
        -d '{"model": "gpt-3.5-turbo", "messages": [{"role": "user", "content": "Hi"}]}')
      echo "Request $i: HTTP $STATUS"
    done

    The first 10 requests succeed with 200, and subsequent requests return 429.

    Example output:

    Request 1: HTTP 200
    Request 2: HTTP 200
    ...
    Request 10: HTTP 200
    Request 11: HTTP 429
    Request 12: HTTP 429
    ...
    Request 15: HTTP 429

Rate limit by user ID

Extract the user ID from a request header and rate limit per user.

  1. Create a RateLimitConfig that limits each user to 50 requests per minute.

    kubectl apply -f- <<EOF
    apiVersion: ratelimit.solo.io/v1alpha1
    kind: RateLimitConfig
    metadata:
      name: user-rate-limit
      namespace: agentgateway-system
    spec:
      raw:
        descriptors:
        - key: x-user-id
          rateLimit:
            requestsPerUnit: 50
            unit: MINUTE
        rateLimits:
        - actions:
          - cel:
              expression: 'request.headers["x-user-id"]'
              key: "x-user-id"
    EOF
  2. Apply the rate limit by using a EnterpriseAgentgatewayPolicy resource. The following example targets the openai HTTPRoute.

    kubectl apply -f- <<EOF
    apiVersion: enterpriseagentgateway.solo.io/v1alpha1
    kind: EnterpriseAgentgatewayPolicy
    metadata:
      name: user-rate-limit
      namespace: agentgateway-system
    spec:
      targetRefs:
      - group: gateway.networking.k8s.io
        kind: HTTPRoute
        name: openai
      traffic:
        entRateLimit:
          global:
            rateLimitConfigRefs:
            - name: user-rate-limit
    EOF
  3. Send requests with a user ID header to trigger the rate limit.

    for i in $(seq 1 60); do
      STATUS=$(curl -s -o /dev/null -w "%{http_code}" \
        http://$INGRESS_GW_ADDRESS/openai \
        -H content-type:application/json \
        -H "x-user-id: alice" \
        -d '{"model": "gpt-3.5-turbo", "messages": [{"role": "user", "content": "Hi"}]}')
      echo "Request $i: HTTP $STATUS"
    done
    for i in $(seq 1 60); do
      STATUS=$(curl -s -o /dev/null -w "%{http_code}" \
        localhost:8080/openai \
        -H content-type:application/json \
        -H "x-user-id: alice" \
        -d '{"model": "gpt-3.5-turbo", "messages": [{"role": "user", "content": "Hi"}]}')
      echo "Request $i: HTTP $STATUS"
    done

    The first 50 requests succeed with 200, and subsequent requests return 429. Each unique user ID gets its own rate limit counter.

    Example rate-limited response:

    < HTTP/1.1 429 Too Many Requests
    < x-envoy-ratelimited: true
    < date: Tue, 18 Jun 2024 05:15:13 GMT
    < server: envoy
    < content-length: 0

Combine local and global rate limiting

Apply both local and global rate limits to the same traffic.

  1. Create a RateLimitConfig for the global per-user limit.

    kubectl apply -f- <<EOF
    apiVersion: ratelimit.solo.io/v1alpha1
    kind: RateLimitConfig
    metadata:
      name: combined-global-limit
      namespace: agentgateway-system
    spec:
      raw:
        descriptors:
        - key: x-user-id
          rateLimit:
            requestsPerUnit: 50
            unit: MINUTE
        rateLimits:
        - actions:
          - cel:
              expression: 'request.headers["x-user-id"]'
              key: "x-user-id"
    EOF
  2. Apply two EnterpriseAgentgatewayPolicy resources that target the same HTTPRoute, one for the local limit and one for the global limit. A single EnterpriseAgentgatewayPolicy cannot set both traffic.rateLimit and traffic.entRateLimit.

    kubectl apply -f- <<EOF
    apiVersion: enterpriseagentgateway.solo.io/v1alpha1
    kind: EnterpriseAgentgatewayPolicy
    metadata:
      name: combined-rate-limit-local
      namespace: agentgateway-system
    spec:
      targetRefs:
      - group: gateway.networking.k8s.io
        kind: HTTPRoute
        name: openai
      traffic:
        rateLimit:
          local:
          - requests: 100
            unit: Seconds
    ---
    apiVersion: enterpriseagentgateway.solo.io/v1alpha1
    kind: EnterpriseAgentgatewayPolicy
    metadata:
      name: combined-rate-limit-global
      namespace: agentgateway-system
    spec:
      targetRefs:
      - group: gateway.networking.k8s.io
        kind: HTTPRoute
        name: openai
      traffic:
        entRateLimit:
          global:
            rateLimitConfigRefs:
            - name: combined-global-limit
    EOF

    This configuration enforces:

    • Local: 100 requests per second per proxy replica (protects against traffic spikes)
    • Global: 50 requests per minute per user across all replicas (enforces user quotas)

    Both limits are evaluated. A request must pass both checks to succeed.

Cleanup

You can remove the resources that you created in this guide.
kubectl delete RateLimitConfig ip-rate-limit -n agentgateway-system
kubectl delete RateLimitConfig user-rate-limit -n agentgateway-system
kubectl delete RateLimitConfig combined-global-limit -n agentgateway-system
kubectl delete EnterpriseAgentgatewayPolicy ip-rate-limit -n agentgateway-system
kubectl delete EnterpriseAgentgatewayPolicy user-rate-limit -n agentgateway-system
kubectl delete EnterpriseAgentgatewayPolicy combined-rate-limit-local -n agentgateway-system
kubectl delete EnterpriseAgentgatewayPolicy combined-rate-limit-global -n agentgateway-system