About Redis

Redis link

Redis®* ¹ is an open source, in-memory data structure store. For more information, see the Redis docs.

What Redis is used for link

Gloo Mesh Enterprise uses Redis as the backing database for four main use cases, as described in the following table. Each use case name corresponds to the redisStore section in the Helm values file where you configure the details for that use case.

You can use the same Redis instance for all use cases, share an instance across some use cases, or have a separate instance for each use case. For more information, see How many Redis instances you need.

For each Redis instance, you can choose to use the default built-in Redis instance, deploy your own local Redis instance, or configure your own external Redis instance in a cloud provider, such as AWS ElastiCache. For more information, see Redis deployment options.

How many Redis instances you need link

As described in What Redis is used for, many Gloo Mesh Enterprise produce and consume data from Redis. Review the following options to help you decide how many instances to use.

• A single, shared Redis instance for all redisStore usages (snapshot, insights, extAuthService, and rateLimiter). This is the most common setup. This way, all the data for your Gloo Mesh Enterprise environment is kept in the same place. Any settings that you apply to this instance, such as security or sizing configuration, are used consistently across all usages.
• Separate Redis instances for the control plane (snapshot and insights) versus the data plane (extAuthService and rateLimiter). Although you now have to manage separate instances, you get more flexibility in setup. For example, you might have stricter HA/DR requirements for the data plane information, so that in case of failure, API keys or rate limits persist. You can also set up different types of Redis, such as built-in for the control plane vs. an external cloud-provided Redis instance for a multicluster data plane.
• Separate Redis instances for all usages. Similar to the previous instance, you might have different Redis instances for all of the usages. This way, you can adjust the size, security, and other features per instance.

For deployment options, sizing guidance, and links to setup steps, see Redis deployment options.

Redis deployment options link

When you install Gloo Mesh Enterprise, you can choose how to back up different components with Redis:

• Built-in local Redis
• Your own local Redis deployment, such as Redis Enterprise.
• Your own external Redis instance, such as through Redis or a Redis API compatible cloud provider such as AWS ElastiCache or Google Cloud Memorystore.

For a quick red-yellow-green scale of each option by common requirement, review the following table.

About the requirements:

• Ease of setup: The built-in option is the fastest, as you do not need to preconfigure your Redis instance. To bring your own, you must set up a Redis instance yourself, and then configure Gloo Mesh Enterprise to use that instance during a Helm installation or upgrade.
• Management burden: Bringing your own local Redis has the highest management burden, because you are entirely responsible for configuring it. Additionally, the resources might impact your cluster performance because it is deployed in the same cluster.
• Password auth and TLS connection: Currently, the built-in option does not let you configure auth or TLS encryption. However, traffic is partly secured already, because the traffic stays within your cluster environment. You can use password auth and TLS encryption if you bring your own instance.
• Highly available data: You can optionally set up persistent storage for the built-in Redis to increase the availability of data in case of a pod restart. If you bring your own local Redis instance, you are also responsible for setting up data persistence. External Redis instances typically come with highly available data persistence by default from the cloud provider.
• Data recovery: Similar to HA data, a cloud-provided external Redis option typically offers the most redundant recovery options by automatically replicating data across zones or regions. For built-in or local Redis instances, recovery options are typically limited to the recovery options of the nodes that underlie the cluster that the Redis instance is deployed to.
• Multicluster: Cloud instances are especially well-suited to multicluster environments, where every cluster can point to the same backing cloud instance. For data plane workloads like external auth and rate limiting, multicluster availability is especially important. By default, the built-in Redis is not multicluster. Redis Enterprise also does not support cross-cluster operations as described in the Redis Enterprise docs.
• Performance at scale: Because the Redis instances are deployed to the same cluster as the workloads that use them, performance typically scales well. Cloud performance can vary depending on the strength of the network connectivity. Typically, you want to create the external Redis instance as near to your cluster infrastructure as possible, and in the same network. Be sure to review sizing guidance for each deployment option.

Built-in local Redis link

By default, Solo provides the option to deploy several built-in, local Redis instances for various components when you install Gloo Mesh Enterprise.

This option is convenient for quick setups, demonstrations, and small testing or preprod environments. To make the local Redis more highly available, you can configure persistent storage.

One current limitation is that this built-in option does not let you configure auth or TLS encryption. However, traffic is partly secured already, because the traffic stays within your cluster environment. This option can also be difficult to scale as your environment grows larger, or in multicluster use cases particularly for external auth, portal server, and rate limiter.

For setup steps, see Built-in Redis.

For sizing guidance, review the following suggestions.

Bring your own local Redis link

Instead of using the Solo-provided built-in local Redis instance, you can deploy your own. This option can be particuarly well-suited for an existing Redis deployment that you manage, such as a local instance of Redis Enterprise.

By using Redis Enterprise, you can set up replicated nodes in each workload cluster where the component runs, such as the gateway, external auth service, portal server, and rate limiter. This way, performance increases because the data is typically stored and accessed on the same network as the requesting client.

For setup steps, see Local Redis.

For sizing guidance, review the following suggestions.

Footnotes for the table:

• * Persistent data for large environments: For large environments with thousands of custom resources, enabling persistence can have an impact on performance. For example, copies of the snapshot and insights data must be written to the persistent volume, which might initially take a large amount of time.

Bring your own external Redis instance link

Instead of using the Solo-provided built-in local Redis instance, you can set up Gloo Mesh Enterprise to use your own external Redis instance in a cloud provider, such as AWS ElastiCache or Google Cloud Memorystore. This option is ideal for when you want to use features of a cloud-managed Redis service, such as for performance, compliance, or security reasons.

An external Redis instance offers advantages in scalability, management, and security. You might be able to scale more responsively to loads by adding more resources to your cloud instance. Cloud providers often help with routine maintenance tasks such as backups, patching, and monitoring. You can also enable security features such as creating the instance in the same VPC, enabling encryption at rest, TLS in transit, and password authentication.

Tradeoffs include cost, complexity, and latency. Especially at scale and in multicluster environments, a cloud instance might be more expensive than self-managed option. You also must be aware of potentially complex cloud provider networking and security settings. Finally, the network latency might not offer the performance that you need, especially if your apps are in a different region or network than the Redis instance.

For setup steps, see External Redis.

Footnotes for the table:

• * Redis size: How large the Redis instance should be, based on the size and deployment method for your environment. For local instances in production environments, use use dynamic, persistent storage. For external Redis instances, the sizing suggestions in the table are based off AWS ElastiCache node types and their corresponding AWS EC2 instance types. If you use a different Redis-compatible provider, try to use a comparable instance size.

Persistent storage link

When you use a local Redis instance, the data is not persistent. This means that if Redis restarts, such as due to an upgrade, the data is no longer available. The related Gloo components, such as the management server and agent relay process, must repopulate the data in Redis. This relay process can take time, particularly in large environments. Additionally, you might have safety features such as safe mode or safe start windows to prevent translation errors until the data becomes available. In such cases, you might want your local Redis data to persist to speed up the time until the data is available.

How persistent storage works link

You can set up persistent storage for the local Redis instances that back your Gloo components by using Kubernetes storage. To set up Kubernetes storage, you configure three main storage resources: storage classes, persistent volumes (PVs), and persistent volume claims (PVCs). You configure storage classes and PVs on your own. For example, your cloud provider might have pre-configured storage classes that you can choose from to create your PVs. Then during a Gloo installation or upgrade, you use the Gloo Helm chart to configure a PVC (or use an existing one) for the local Redis instance.

Persistent volumes (PVs) are a cluster-wide resource where the persisted data is stored outside the lifecycle of the Redis pod. You can provision the PV yourself or dynamically by using a PVC. Note that your infrastructure provider might limit your options for how to provision the PV.

Persistent volume claims (PVC) request storage for the local Redis instance that mounts the PVC as a volume. PVCs specify size, access modes, and specific storage class requirements. When a PVC is created, Kubernetes finds a PV that matches the claim’s requirements. If a suitable PV is available, the PV is bound to the PVC. If no suitable PV is found but a storage class is defined, Kubernetes can dynamically provision a new PV according to the specifications of the storage class and bind it to the PVC.

Storage classes define different types of storage that are offered by your cluster’s cloud provider or on-premise storage system. You might have different storage classes for different storage performance levels or backup policies that customize settings such as replication or IOPS. You set the storage class that the PVC requests in your Helm settings. If you do not define a storage classes, the default storage class is used.

To statically provision the PV and matching PVC, set the redis.deployment.persistence.persistentVolume.existingClaim value to true. Then, put the name of the PVC that you want the Redis deployment to mount in the redis.deployment.persistence.persistentVolume.name field.

To use dynamic provisioning, you can set up the details of the PVC in the redis.persistence.persistentVolume section of the Gloo Helm chart during installation or upgrade, as described in the setup guide.

Redis options for RDB and AOF link

When you enable persistence for local Redis, you must also set up the persistence mode. You can choose between two main persistence modes: Redis Database (RDB) or Append Only File (AOF). You can also use the modes together, but this use case is rare.

• RDB: RDB provides a point-in-time snapshot for a disaster recovery backup, but has trade-offs in the completeness of data versus AOF.
• AOF: AOF logs every write operation for a more complete recovery option, but has trade-offs in resource usage which can impact performance versus RDB.

For more information, see the Redis docs. For more details of how to configure these persistent modes, see the setup guide.

More details on stored data link

Review the following sections to learn more about what data is stored for the different uses of Redis to back up Gloo Mesh Enterprise components.

Data stored in snapshots link

The Gloo snapshot that the agent sends to the management server is stored in the management server’s backing storage. For an example of how to check the data that gets stored, see Review data in Redis.

• Discovered Kubernetes resources, such as Kubernetes services, deployments, replicasets, daemonsets, and statefulsets. The management server translates discovered resources into Istio resources and displays them in the Gloo UI. Note that you can use Istio discovery selectors to ignore certain Kubernetes resources. Ignored resources are not included in the snapshot that is sent from the agent to the management server.
• Gloo custom resources that you create. The management server translates Gloo resources and displays them in the Gloo UI.
• Istio resources, including:
  • Istio resources that, after initial server-agent setup, the management server automatically translates from your Gloo resources and writes back to the workload cluster. These resources are included in the snapshot to avoid accidentally deleting them from the workload cluster if an agent disconnects and reconnects, and to display them in the Gloo UI.
  • Any Istio resources that you manually created, so that they can be displayed in the Gloo UI.
• Internal resources that are computed in memory by each agent and pushed to the management server without being persisted in the workload cluster. Internal resources include:
  • Gateway resources, which contain information about gateway endpoints in the cluster.
  • IssuedCertificate and CertificateRequest resources, which are used in internal multi-step workflows that involve both the agent and the management server.
• External authentication information the UI, when enabled. This data includes user session information such as access and ID tokens.

Data stored for insights link

If you enable the insights engine, Gloo Mesh Enterprise automatically analyzes your Istio setup to check for health issues. This insight information is stored in Redis for the Gloo UI to read and display to you in the dashboard.

Data stored for the external auth service link

The external auth service stores data to use to authenticate to destinations and routes that are protected by an external auth policy and enforced by an external auth server.

Such data can include the following, depending on how you configure external auth.

• API keys that you create (such as through the developer portal), either as plain text or hashed if you configure a secret key to hash the values.
• Session data such as cookies.
• OIDC distributed claims.

For more information, see External authentication and authorization.

Data stored for the developer portal link

The portal server stores the following data in the backing database. For an example of how to check the data that gets stored, see Review data in Redis.

• API keys that end users can use to authenticate to destinations and routes that are protected by an external auth policy that the external auth server enforces.

To use Gloo Portal, you must configure both the portal server and external auth service to use the same backing storage. Review the following diagram for more information.

1. As the portal admin, you configure the backing storage database for both the external auth and portal servers.
2. As an end user, you use the developer portal to generate an API key. Then, the Gloo portal server writes this API key to the shared backing storage database, such as Redis.
3. As an end user, you include the API key in subsequent requests to the API products that you want to access. The Gloo ingress gateway receives your requests and checks the API’s route for external auth policies. The Gloo external auth server reads the API key value from the shared backing storage database, such as Redis. If the API key matches a valid value, then the Gloo gateway sends the request to the API product and you get back a successful response.

For more information, see Verify that external auth works with API keys in a backing storage database in the Gloo Mesh Gateway Portal docs.

Data stored for the rate limiter link

The rate limiter stores information for limiting requests in the backing Redis database, including the following information.

• The generated unique key for the request.
• The associated count against the rate limit.

For more information, see Rate limiting.