What data gets stored

The Gloo snapshot that the agent sends to the management server is stored in the management server’s backing storage. For an example of how to check the data that gets stored, see Review data in Redis.

  • Discovered Kubernetes resources, such as Kubernetes services, deployments, replicasets, daemonsets, and statefulsets. The management server translates discovered resources into Istio resources and displays them in the Gloo UI. Note that you can use Istio discovery selectors to ignore certain Kubernetes resources. Ignored resources are not included in the snapshot that is sent from the agent to the management server.
  • Gloo custom resources that you create. The management server translates Gloo resources and displays them in the Gloo UI.
  • Istio resources, including:
    • Istio resources that, after initial server-agent setup, the management server automatically translates from your Gloo resources and writes back to the workload cluster. These resources are included in the snapshot to avoid accidentally deleting them from the workload cluster if an agent disconnects and reconnects, and to display them in the Gloo UI.
    • Any Istio resources that you manually created, so that they can be displayed in the Gloo UI.
  • Internal resources that are computed in memory by each agent and pushed to the management server without being persisted in the workload cluster. Internal resources include:
    • Gateway resources, which contain information about gateway endpoints in the cluster.
    • IssuedCertificate and CertificateRequest resources, which are used in internal multi-step workflows that involve both the agent and the management server.

Backing storage options

Review the following options for setting up backing storage.

OptionUse caseBenefitsDrawbacks
Built-in local RedisProof of concept and staging environments
  • Default setup, automatically works
  • Fast read/write speed as the database is local to the management server
  • Updatable via Gloo Platform Helm chart
  • Only as highly available as your cluster setup
  • Has the same disaster recovery as your cluster setup
  • Might not meet your organization’s compliance requirements
Bring your own RedisProduction environments
  • Enhanced high availability and disaster recovery (HA/DR)
  • Service level agreement (SLA) from your provider
  • Meet your organization’s compliance requirements
  • More complicated setup
  • Not covered by Solo support (contact your external Redis provider)

Built-in local Redis

By default, a local Redis instance gloo-mesh-redis is set up in the gloo-mesh namespace of the management cluster when you install the Gloo management server and UI components. The Gloo management server replicas and the Gloo UI are automatically configured to read from and write data to this Redis instance for a global view of your Gloo environment.

To enable the local Redis, you use the following Helm values during the Gloo management server installation:

    # Enable the creation of the local Redis deployment and service.
    enabled: true

You can further configure the default Redis values, such as to modify the Kubernetes deployment’s resource limits and requests. For more information, see the following docs:

  • Check the Helm reference for the redis component.
  • Follow the Upgrade guide for instructions on how to upgrade your installation after modifying the Helm fields.

Bring your own Redis

Instead of using the built-in local Redis instance, you can achieve higher availability, disaster recovery, and enhanced control and security by bringing your own Redis cluster. You can choose between a local or externally hosted instance. For a local instance, you can install the Redis cluster within the Gloo management cluster and run it alongside the Gloo UI and Gloo management server. For an external instance, you can create an instance outside your cluster environment, such as Redis Enterprise or a cloud provider service like AWS ElastiCache. Then, you include the settings for your local or external Redis instance when you install or upgrade your Gloo Helm chart.

Redis settings to customize

When you bring your own Redis cluster, you can customize the following aspects. For more information, check the Helm reference.

  • Authentication: To authenticate with your Redis instance, you can configure Gloo Mesh Gateway to use a username and password when connecting to Redis. Depending on your Redis provider, you might only need to specify a password to authenticate with Redis. You store these credentials in a Kubernetes secret on the same cluster as the component that uses the Redis database, such as the management server or external auth server.
  • TLS certificates: For enhanced security, you can use TLS certificates to authenticate with your Redis cluster. Note that your Redis instance must be configured to accept TLS connections. You store the certificates in a Kubernetes secret on the same cluster as the component that uses the Redis database, such as the management server or external auth server.
  • Connection: You can optionally set connection parameters, such as the maximum number of active connections, retries, or number of idle connections.
  • Database: You can optionally specify the database instance within your Redis cluster that you want to connect to.

Provider considerations

AWS ElastiCache

When setting up your AWS ElastiCache instance, consider the following deployment options:

  • Use a high availability (HA) setup consisting of at least 1 primary and 1 replica instance.
  • Use non-clustered mode. Cluster mode is currently not supported.
  • Set up multiple availability zones (multi-AZ) with auto-failover enabled.
  • Set the maxmemory-policy directive to noeviction. For more information about eviction policies, see the Redis docs.
  • Use node types with TLS Offloading and Enhanced I/O Multiplexing.
  • Although optional, enabling TLS is recommended. You can provide the TLS certs when you set up Gloo to use your own Redis instance.
  • Monitor performance metrics including CPU utilization, memory usage, network throughput, and cache hit ratio.
  • Based on the size of your environment, increase the type of node instance. For more information, see Size and memory system requirements.

Steps to bring your own Redis

To bring your own Redis instance:

  1. Create or use an existing Redis cluster, such as AWS ElastiCache, and make sure that you can connect to your instance from the Gloo management cluster. For example, if the cluster is behind a firewall, you might need to add the public IP address of the Redis cluster to your allowlist.

  2. Choose how to authenticate with your Redis cluster. Depending on your Redis provider, you might be required to use a specific authentication method, such as TLS certificates or a username and password.

  3. Create a Kubernetes secret to store your credentials.

    • Username and password:
        kubectl apply --context $MGMT_CONTEXT -f- <<EOF
      apiVersion: v1
      kind: Secret
      type: Opaque
         name: redis-auth-secrets
         namespace: gloo-mesh
        username: "<username>"
        password: "<password>"
    • TLS certificates:
        kubectl apply --context $MGMT_CONTEXT -f- <<EOF
      apiVersion: v1
      kind: Secret
      type: Opaque
        name: "redis-certs"
        namespace: gloo-mesh
        redis.crt: "<TLS_certificate>"
  4. Follow the Upgrade guide to get your Gloo Platform Helm installation configuration file.

  5. In your Helm values file, add the following information:

    • redis.address: The address to use to connect to your Redis cluster.
    • redis.auth: Add this section if you want to authenticate with your Redis cluster by using a username and password. To authenticate by using TLS certificates, remove this section and configure redis.certs instead.
    • redis.certs: Add this section if you want to authenticate with your Redis cluster by using TLS certificates. To authenticate by using a username and password, remove this section and configure redis.auth instead.
    • redis.connection: You can optionally override the default connection settings, such as the maximum number of retries.
    • redis.db: You can optionally override the index to connect to a specific database in the Redis cluster.
    • redis.deployment.enabled: Set this option to false to disable the built-in local Redis deployment.
    • redis.deployment: If you used the local Redis deployment before and customized the deployment, remove any custom settings in the redis.deployment section.
      # Address to use when connecting to the Redis instance. To use the default Redis deployment, specify 'redis.gloo-mesh.svc.cluster.local:6379'.
      address: gloo-mesh-redis.gloo-mesh:6379
      # Optional authentication values to use when connecting to the Redis instance
        # Connect to the Redis instance with a password
        enabled: false
        # The secret key containing the password to use for authentication
        passwordKey: password
        # Name of the k8s secret that contains the password
        secretName: redis-auth-secrets
        # The secret key containing the username to use for authentication
        usernameKey: username
      # Configuration for TLS verification when connecting to the Redis instance
        # File name that contains the ca cert
        caCertKey: redis.crt
        # Enable a secure network connection to the Redis instance via TLS
        enabled: false
        # Name of the k8s secret that contains the certs
        secretName: redis-certs
      # Optional connection parameters
        # Dial timeout for establishing new connections. Default is 5 seconds.
        dialTimeout: 5s
        # Frequency of idle checks made by idle connections reaper. Default is 1 minute. -1 disables idle connections reaper, but idle connections are still discarded by the client if IdleTimeout is set.
        idleCheckFrequency: 1m0s
        # Amount of time after which client closes idle connections. Should be less than server's timeout. Default is 5 minutes. -1 disables idle timeout check.
        idleTimeout: 5m0s
        # Connection age at which client retires (closes) the connection. Default is to not close aged connections.
        maxConnAge: ""
        # Maximum number of retries before giving up. Default is 3. -1 disables retries.
        maxRetries: 3
        # Maximum backoff between each retry. Default is 512 milliseconds. -1 disables backoff.
        maxRetryBackoff: 512ms
        # Minimum number of idle connections which is useful when establishing new connection is slow.
        minIdleConns: 0
        # Minimum backoff between each retry. Default is 8 milliseconds. -1 disables backoff.
        minRetryBackoff: 8ms
        # Type of connection pool. true for FIFO pool. false for LIFO pool. Note that FIFO has higher overhead compared to LIFO.
        poolFifo: false
        # Maximum number of socket connections. Default is 10 connections per every available CPU as reported by runtime.GOMAXPROCS.
        poolSize: 0
        # Amount of time client waits for connection if all connections are busy before returning an error. Default is ReadTimeout + 1 second.
        poolTimeout: ""
        # Timeout for socket reads. if reached, commands will fail with a timeout instead of blocking. Default is 3 seconds. -1 disables timeout. 0 uses the default value.
        readTimeout: 3s
        # Timeout for socket writes. If reached, commands will fail with a timeout instead of blocking. Default is ReadTimeout.
        writeTimeout: ""
      # DB to connect to
      db: 0
        # Enable creation of the deployment/service.
        enabled: false
  6. Continue with the Upgrade guide to reinstall Gloo Mesh Gateway with your own Redis instance.