Connection pool settings for HTTP

Use a connection policy to configure connection pool settings for an HTTP destination.

Connection pools are typically set up for apps that must be accessed frequently, such as a web service. Without connection pools, every request to the web service requires a new connection to be opened. This setup might work if the web service is accessed occasionally. However, as clients scale and more requests are sent to the web service, opening a connection for each requests can get very expensive and easily overload the web service.

To prevent the upstream web service from being overloaded, you can use the following connection pool settings:

For more information, see the following resources.

If you import or export resources across workspaces, your policies might not apply. For more information, see Import and export policies.

Before you begin

This guide assumes that you use the same names for components like clusters, workspaces, and namespaces as in the getting started, and that your Kubernetes context is set to the cluster you store your Gloo config in (typically the management cluster). If you have different names, make sure to update the sample configuration files in this guide.

Follow the getting started instructions to:

  1. Set up Gloo Gateway in a single cluster.
  2. Deploy sample apps.
  3. Configure an HTTP listener on your gateway and set up basic routing for the sample apps.

Configure connection policies for HTTP destinations

You can apply a connection policy at the destination level. For more information, see Applying policies.

The connection policy currently does not support selecting Gloo external services as a destination.

The following example applies HTTP connection pool settings to the Bookinfo ratings app.

apiVersion: resilience.policy.gloo.solo.io/v2
kind: ConnectionPolicy
metadata:
  annotations:
    cluster.solo.io/cluster: ""
  name: http-connect
  namespace: bookinfo
spec:
  applyToDestinations:
  - selector:
      labels:
        app: ratings
  config:
    http:
      idleTimeout: 2s
      maxRequestsPerConnection: 1
      maxRetries: 1
      maxRequests: 500
      maxPendingRequests: 500

Review the following table to understand this configuration. For more information, see the API docs.

Setting Description
applyToDestinations Configure which destinations to apply the policy to, by using labels. Destinations can be a Kubernetes service, VirtualDestination, or ExternalService. If you do not specify any destinations or routes, the rate limit policy applies to all destinations in the workspace by default. If you do not specify any destinations but you do specify a route, the rate limit applies to the route but to no destinations.
config Configure the connection settings to apply to the selected destinations. To set connection pool settings for HTTP destinations, use http as the protocol. For TCP connection pool settings, use tcp. The connection policy in this guide shows how to configure connection pool settings for an HTTP destination. To find an example for a TCP connection policy, see Connection pool settings for TCP.
http.idleTimeout The time a connection can stay open without receiving any requests. By default, this value is set to 1 hour. If the idle time is reached, the connection is closed.
http.maxRequestsPerConnection The maximum number of requests that can be sent to a destination per connection. If you set this value to 1, you disable keep alive. By default, this value is set to 0, which equals unlimited, and allows a maximum of 2^29 requests per connection.
http.maxRetries The maximum number of retries that can be outstanding to all hosts in a cluster at a given time. Defaults to 2^32-1.
http.maxRequests The maximum number of active requests to an upstream destination. The default value is 1024.
http.maxPendingRequests The maximum number of requests that can be queued while waiting for a connection from the connection policy to become available. The default value is 1024.

Verify connection policies

  1. Apply the example connection policy for the ratings app.

    kubectl apply -f- <<EOF
    apiVersion: resilience.policy.gloo.solo.io/v2
    kind: ConnectionPolicy
    metadata:
      annotations:
        cluster.solo.io/cluster: ""
      name: http-connect
      namespace: bookinfo
    spec:
      applyToDestinations:
      - selector:
          labels:
            app: ratings
      config:
        http:
          idleTimeout: 2s
          maxRequestsPerConnection: 1
          maxRetries: 1
          maxRequests: 500
          maxPendingRequests: 500
    EOF
    
  2. Verify that an Istio destination rule is created for the ratings app.

    kubectl get destinationrule -n bookinfo
    

    Example output:

    NAME                                                              HOST             AGE
    ratings-global-virtual-destinat-2ab46384c8b40be3cfab9740ac8fb2c   ratings.global   11s
    
  3. Describe the Istio destination rule.

    kubectl describe -n bookinfo destinationrule <destination-rule>
    

    In the output, verify that the Connection Pool settings include the http settings that your policy configures.

    ...
    Traffic Policy:
     Port Level Settings:
       Connection Pool:
         Http:
           http1MaxPendingRequests:      500
           http2MaxRequests:             500
           Idle Timeout:                 2s
           Max Requests Per Connection:  1
           Max Retries:                  1
    
  4. Optional: Clean up the resource that you created.

    kubectl -n bookinfo delete ConnectionPolicy http-connect