Rate limits for APIs
Control how many requests within a time period are allowed to your APIs, such as 100 requests per minute.
Set rate limits for your API products to control how many requests within a time period are allowed to your API products. The rate limits form a key part of the usage plan for your API products. For more information about the Gloo rate limiting add-on, see Rate limit.
Before you begin
For dynamic rate limiting, install Portal with analytics.
Make sure that the required Gloo components are running. These include the portal server, rate limiter, external auth service, and for dynamic rate limiting, the
logs/portal
telemetry collector pipeline and portal-related dynamic metadata access log formatting in the Istio operator specification.kubectl get pods -A -l app=gloo-mesh-portal-server kubectl get pods -A -l app=ext-auth-service kubectl get pods -A -l app=rate-limiter kubectl get -A istiooperator -l reconciler.mesh.gloo.solo.io/name=istio-lifecycle -o yaml
Create your APIs, including the Gloo ApiDocs that describe the stitched schema.
Bundle your APIs into API products by using a route table.
Get the labels of your routes to use to apply policies to, such as with the example query.
kubectl get rt -n gloo-mesh-gateways -o=jsonpath='{range .items[*]}[{.metadata.name}, {.spec.http[*].name}, {.spec.http[*].labels}]{"\n"}{end}'
Example output:
- The
api-example-com-rt
route table does not have any route-level labels. To apply policies, you can add labels to those routes. - The
petstore-rt
route table has ausagePlans: dev-portal
label on itspets-api
,users-api
, andstore-api
routes. - The
tracks-rt
route table has ausagePlans: dev-portal
label on itstracks-api
route.
[api-example-com-rt, , ] [petstore-rt, pets-api users-api store-api, {"usagePlans":"dev-portal"} {"usagePlans":"dev-portal"} {"usagePlans":"dev-portal"} ] [tracks-rt, tracks-api, {"usagePlans":"dev-portal"}]
- The
Rate limiting options
Gloo Portal implements rate limits by reusing the same policies that you also use for non-Portal routes in Gloo Mesh Gateway. This flexible approach lets you reuse rate limiting resources such as server configurations across your teams and apps.
Review the following approaches to rate limiting for more examples.
- Basic rate limiting: A basic approach to rate limiting that limits requests to a certain amount per API. These rate limits are bundled into usage plans for tiers such as gold, silver, or bronze that allow differing numbers of requests per plan per user.
- Dynamic rate limiting: A dynamic approach to rate limiting that limits requests based on portal metadata that is unique to the client, such as usage plans and user IDs. This way, you can set unique rate limits per user for each API product without having to create many separate usage plans and rate limiting resources.
Basic rate limiting
Control how many requests within a time period are allowed to your APIs, such as 100 requests per minute.
For instructions, follow the Set up rate limiting guide in the Protect your APIs tutorial.
Update the
RateLimitClientConfig
based on the type of external auth policy that you applied: API key or OAuth.
Dynamic rate limiting
Set up dynamic rate limiting for users of the developer portal. For example, you might want different user IDs to have unique rate limits for each API product.
As shown in the Basic rate limiting section, one option is to create separate usage plans and rate limiting resources for each user that has the rate limits you want. This approach, however, becomes difficult to manage at scale well. For example, usage plans typically represent pricing tiers, such as gold, silver, or bronze tier, and you might not want to manage different pricing tiers per user.
Instead, you can set up Gloo Mesh Gateway to use rate limit overrides with the metadata in client-initiated requests to enforce dynamic rate limiting. The rate limiting is called “dynamic,” because it is based on the dynamic metadata that the external auth filter in the Envoy proxy gets from the external auth module, such as the API key. You can use this dynamic metadata to track portal-specific metadata such as usage plans, user IDs, and the rate limit. Then, you can write Rego policies for the Open Policy Agent (OPA) Envoy API plugin that is part of Gloo’s external auth service to enforce the dynamic rate limit that you set.
Step 1: Add rate limiting information to the request metadata
To add Envoy dynamic metadata to requests, you configure the OPA auth module as part of a Gloo external auth policy. To enforce OPA auth, you write a Rego policy to evaluate requests that include the dynamic metadata. For more information about external auth and OPA with Gloo, see OPA.
For help writing Rego policies, refer to the OPA policy language docs. If you use the OPA-Envoy plugin API, the OPA docs has a policy primer. For example, you might create an output document to organize the results of a decision or to gather all the metadata of an object like an API key. Then, you can test your Rego policies in the Rego playground.
Create an OPA Rego policy file. The following policy:
- Allows requests that include the test header
api-key=apikey1
. - Denies requests with a 401 response code if the request does not include the header or reaches the rate limit.
- Checks for a
disable-dynamic-rl
header so that you can disable dynamic rate limiting for quick testing. Note that in production scenarios, you would not include such logic to disable rate limiting based on the header values. But for testing purposes, this header can help you confirm that the policy works as expected. - Sets dynamic metadata so that the
gold
usage plan for theuser1
user ID gets a new rate limit of 7 requests per minute (as opposed to 5 per minute as configured by the basic rate limiting setup). - Returns a result payload with new response headers such as
x-validated-by
andx-client-only
, removes theapi-key
header, and customizes the body text for rejected results.
cat <<EOF > portal-policy.rego package test import future.keywords.if default allow = false allow if { input.http_request.headers["api-key"] = "apikey1" } http_status := 200 if { allow } http_status := 401 if { not allow } default disable_dynamic_rate_limits = false disable_dynamic_rate_limits if { input.http_request.headers["disable-dynamic-rl"] = "true" } dynamic_metadata["usagePlan"] := "gold" dynamic_metadata["userId"] := "user1" dynamic_metadata["rateLimit"] := { "requests_per_unit": 7, "unit": "MINUTE" } if not disable_dynamic_rate_limits result["dynamic_metadata"] := dynamic_metadata # allow/allowed result["allow"] := allow result["http_status"] := http_status result["headers"]:= { "x-validated-by": "opa-dynamic-rl-checkpoint" } result["response_headers_to_add"]:= { "x-client-only": "visible" } result["request_headers_to_remove"]:= ["api-key"] result["body"] := "Request does not have valid API key, or exceeded the rate limit." EOF
- Allows requests that include the test header
Store the OPA policy in a Kubernetes config map in the workload cluster that you want to create the external auth policy in.
kubectl -n gloo-mesh create configmap opa-envoy-plugin-api-policy --from-file=portal-policy.rego
Create an external auth policy to apply OPA protection to your APIs.
- Applies to routes with the
usagePlans: dev-portal
label, such as the Tracks API. - Refers to the default Gloo external auth server in the
gloo-mesh
namespace. - Configures an OPA module that refers to the Rego policy that you created in the previous steps.
kubectl apply -f - <<EOF
apiVersion: security.policy.gloo.solo.io/v2
kind: ExtAuthPolicy
metadata:
name: ext-auth-opa-policy
namespace: gloo-mesh
spec:
applyToRoutes:
- route:
labels:
usagePlans: dev-portal
config:
server:
cluster: $CLUSTER_NAME
name: ext-auth-server
namespace: gloo-mesh
glooAuth:
configs:
- name: opa_auth
opaAuth:
modules:
- name: opa-envoy-plugin-api-policy
namespace: gloo-mesh
query: "data.test.result"
EOF
Step 2: Set up dynamic rate limit overrides
Set up rate limit overrides to use the dynamic rate limiting information that you added to the request metadata in the previous section.
Select the rate limiting server to use. The example uses the default rate limiting server that you created when you installed Gloo Platform. For more information, see Rate limit server settings. If you already created a RateLimitServerSettings, you can skip this step.
kubectl apply -f - << EOF apiVersion: admin.gloo.solo.io/v2 kind: RateLimitServerSettings metadata: name: rl-server namespace: gloo-mesh labels: portal: rate-limit spec: destinationServer: port: name: grpc ref: cluster: $CLUSTER_NAME name: rate-limiter namespace: gloo-mesh EOF
Create a rate limit client config to define a limit from the dynamic metadata in the
opa_auth
module that you previously configured. For more information, see Rate limit client config.kubectl apply -f - <<EOF apiVersion: trafficcontrol.policy.gloo.solo.io/v2 kind: RateLimitClientConfig metadata: name: usage-plans-opa namespace: gloo-mesh labels: portal: rate-limit spec: raw: rateLimits: - setActions: - metadata: descriptorKey: usagePlan metadataKey: key: envoy.filters.http.ext_authz path: - key: opa_auth - key: usagePlan - metadata: descriptorKey: userId metadataKey: key: envoy.filters.http.ext_authz path: - key: opa_auth - key: userId limit: dynamicMetadata: metadataKey: key: envoy.filters.http.ext_authz path: - key: opa_auth - key: rateLimit EOF
Use the RateLimitServerConfig to define the values of the descriptors that are the usage plan names. These descriptor values match the
usagePlan
descriptor key that you configured in the RateLimitClientConfig. TheuserId
andusagePlan
details are extracted from the details in the request header that is required by the external auth policy. If you already created a RateLimitServerConfig, you can skip this step.The following rate limit server config example creates three usage plans as follows:
- Bronze: Requests to your APIs are limited to 1 per minute.
- Silver: Requests to your APIs are limited to 3 per minute.
- Gold: Requests to your APIs are limited to 5 per minute.
For more information, see Rate limit server config.
kubectl apply -f - << EOF apiVersion: admin.gloo.solo.io/v2 kind: RateLimitServerConfig metadata: name: usage-plans namespace: gloo-mesh labels: portal: rate-limit spec: destinationServers: - port: name: grpc ref: cluster: $CLUSTER_NAME name: rate-limiter namespace: gloo-mesh raw: setDescriptors: - simpleDescriptors: - key: userId - key: usagePlan value: bronze rateLimit: requestsPerUnit: 1 unit: MINUTE - simpleDescriptors: - key: userId - key: usagePlan value: silver rateLimit: requestsPerUnit: 3 unit: MINUTE - simpleDescriptors: - key: userId - key: usagePlan value: gold rateLimit: requestsPerUnit: 5 unit: MINUTE EOF
Create a rate limit policy to apply the new rate limit client config to your APIs. You can reuse the RateLimitServerSettings that selects the rate limiter to use and the RateLimitServerConfig that configures the usage policies.
kubectl apply -f - << EOF apiVersion: trafficcontrol.policy.gloo.solo.io/v2 kind: RateLimitPolicy metadata: name: tracks-rate-limit-opa namespace: default labels: portal: rate-limit spec: applyToRoutes: - route: labels: usagePlans: dev-portal config: ratelimitServerConfig: name: usage-plans namespace: gloo-mesh cluster: $CLUSTER_NAME ratelimitClientConfig: name: usage-plans-opa namespace: gloo-mesh cluster: $CLUSTER_NAME serverSettings: name: rl-server namespace: gloo-mesh cluster: $CLUSTER_NAME phase: postAuthz: priority: 1 EOF
Step 3: Verify dynamic rate limiting
Verify how dynamic rate limiting and basic rate limiting can work together to protect your API products.
Verify that your rate limiting is in place by repeating the following request 8 times in a row. Because your user ID is limited by the new dynamic rate limit of 7 requests per minute, the eighth time your request is blocked.
curl -v -H 'api-key: apikey1' --resolve api.example.com:80:${INGRESS_GW_IP} http://api.example.com/trackapi/tracks
Example output:
< HTTP/1.1 401 < x-envoy-ratelimited: true < Request does not have valid API key, or exceeded the rate limit.
Verify that your request uses the basic rate limiting when dynamic rate limiting is disabled. You can use the special
disable-dynamic-rl: true
header that you configured in the Rego policy. Without dynamic rate limiting, your user ID is limited to thegold
usage plan’s limit of 5 requests per minute. The sixth time your request is blocked. Note that this step requires you to have set up API key external auth already.curl -v -H 'api-key: apikey1' -H 'disable-dynamic-rl: true' --resolve api.example.com:80:${INGRESS_GW_IP} http://api.example.com/trackapi/tracks
Example output:
< HTTP/1.1 429 Too Many Requests < x-envoy-ratelimited: true