Decide how to load balance incoming requests to backend services.

About simple load balancing

Gloo Gateway supports multiple load balancing algorithms for selecting backend services to forward incoming requests to. By default, incoming requests are forwarded to the instance with the least requests. You can change this behavior and instead use a round robin algorithm to forward the request to a backend service.

Least request

The least request load balancer algorithm generally selects the host with the fewest requests. The following rules apply:

  • If no localityType is set, send the request to the host with the fewest requests.
  • If localityType is set, a weighted round robin approach is used, in which higher weighted endpoints are considered more often in the round robin rotation to achieve the selected weight.

Round robin

The round robin load balancer algorithm selects a backend host in a round robin order. If a localityType is also defined, a weighted round robin algorithm is used, in which higher weighted endpoints are considered more often in the round robin rotation to achieve the selected weight.

Random

The random load balancer algorithm selects a random host that is available. This load balancing algorithm usually performs better than the round robin algorithm if no health checks are configured for the backend host.

Other load balancing options

Learn about other load balancing options that you can set in the load balancer policy.

Healthy panic threshold

By default, Gloo Gateway only considers services that are healthy and available when load balancing incoming requests among backend services. In the case that the number of healthy backend services becomes too low, you can instruct Gloo Gateway to disregard the backend health status and either load balance requests among all or no hosts by using the healthy_panic_threshold setting. If not set, the threshold defaults to 50%. To disable panic mode, set this field to 0.

To learn more about this setting and when to use it, see the Envoy documentation.

Update merge window

Sometimes, your deployments might have health checks and metadata updates that use a lot of CPU and memory. In such cases, you can use the update_merge_window setting. This way, Gloo Gateway merges all updates together within a specific timeframe. For more information about this setting, see the Envoy documentation. If not set, the update merge window defaults to 1000ms. To disable the update merge window, set this field to 0s.

Before you begin

  1. Follow the Get started guide to install Gloo Gateway.

  2. Follow the Sample app guide to create a gateway proxy with an HTTP listener and deploy the httpbin sample app.

  3. Get the external address of the gateway and save it in an environment variable.

      export INGRESS_GW_ADDRESS=$(kubectl get svc -n gloo-system http -o jsonpath="{.status.loadBalancer.ingress[0]['hostname','ip']}")
echo $INGRESS_GW_ADDRESS  
  
      kubectl port-forward deployment/http -n gloo-system 8080:8080

Set up a load balancing algorithm

Define the load balancing algorithm that you want to use for your backend app in a BackendConfigPolicy.

  1. Create a BackendConfigPolicy to configure your load balancing algorithm for the httpbin app.

      kubectl apply -f- <<EOF
kind: BackendConfigPolicy
apiVersion: gateway.kgateway.dev/v1alpha1
metadata:
  name: httpbin-lb-policy
  namespace: httpbin
spec:
  targetRefs:
    - name: httpbin
      group: ""
      kind: Service
  loadBalancer:
    leastRequest:
      choiceCount: 3
      slowStart:
        window: 10s
        aggression: "1.5"
        minWeightPercent: 10
EOF

    Review the following table to understand this configuration.

    SettingDescription
    choiceCountThe number of random available backend hosts to consider when choosing the host with the fewest requests. Deafults to 2.
    slowStartWhen you add new backend hosts to the pool of endpoints to route to, slow start mode progressively increases the amount of traffic that is routed to those backends. This can be useful for services that require warm-up time to serve full production loads, to prevent request timeouts, loss of data, and deteriorated user experience.
    slowStart.windowThe duration of the slow start window. If set, a newly-created backend endpoint remains in slow start mode from its creation time until the duration of the slow start window has elapsed.
    slowStart.aggressionThe rate of traffic increase over the duration of the slow start window. Defaults to 1.0, so that the endpoint receives a linearly-increasing amount of traffic. For more information about fine-tuning this value, see the API docs.
    slowStart.minWeightPercentThe minimum percentage of weight that an endpoint must have in the caluclation of aggression. This helps prevent weights that are so small that endpoints receive no traffic during the slow start window. Defaults to 10%.
      kubectl apply -f- <<EOF
kind: BackendConfigPolicy
apiVersion: gateway.kgateway.dev/v1alpha1
metadata:
  name: httpbin-lb-policy
  namespace: httpbin
spec:
  targetRefs:
    - name: httpbin
      group: ""
      kind: Service
  loadBalancer:
    roundRobin:
      slowStart:
        window: 10s
        aggression: "1.5"
        minWeightPercent: 10
EOF

    Review the following table to understand this configuration.

    SettingDescription
    slowStartWhen you add new backend hosts to the pool of endpoints to route to, slow start mode progressively increases the amount of traffic that is routed to those backends. This can be useful for services that require warm-up time to serve full production loads, to prevent request timeouts, loss of data, and deteriorated user experience.
    slowStart.windowThe duration of the slow start window. If set, any newly-created backend endpoints remain in slow start mode from its creation time until the duration of the slow start window has elapsed.
    slowStart.aggressionThe rate of traffic increase over the duration of the slow start window. Defaults to 1.0, so that the endpoint receives a linearly-increasing amount of traffic. For more information about fine-tuning this value, see the API docs.
    slowStart.minWeightPercentThe minimum percentage of weight that an endpoint must have in the caluclation of aggression. This helps prevent weights that are so small that endpoints receive no traffic during the slow start window. Defaults to 10%.

    Note that no further settings are required because the load balancing is random.

      kubectl apply -f- <<EOF
kind: BackendConfigPolicy
apiVersion: gateway.kgateway.dev/v1alpha1
metadata:
  name: httpbin-lb-policy
  namespace: httpbin
spec:
  targetRefs:
    - name: httpbin
      group: ""
      kind: Service
  loadBalancer:
    random: {}
EOF

  2. Verify that your configuration is applied by reviewing the Envoy configuration.

    1. Port forward the http deployment on port 19000.
        kubectl port-forward deploy/http -n gloo-system 19000 &
    2. Open the config_dump endpoint.
        open http://localhost:19000/config_dump
    3. Search for the lb_policy field, and verify that the policy that you set is listed, along with your other load balancing settings. For example, the following output shows the LEAST_REQUEST policy, with the choice_count field set to 3, and settings for the slow start window.
        ...
"lb_policy": "LEAST_REQUEST",
"metadata": {},
"common_lb_config": {
 "consistent_hashing_lb_config": {}
},
"ignore_health_on_host_removal": true,
"least_request_lb_config": {
 "choice_count": 3,
 "slow_start_config": {
  "slow_start_window": "10s",
  "aggression": {
   "default_value": 1.5,
   "runtime_key": "upstream.kube_httpbin_httpbin_9000.slowStart.aggression"
  },
  "min_weight_percent": {
   "value": 10
  }
 }
}
...
    4. Stop port-forwarding the http deployment.
        lsof -ti:19000 | xargs kill -9

Cleanup

You can remove the resources that you created in this guide.
  kubectl delete BackendConfigPolicy httpbin-lb-policy -n httpbin