External Redis
Bring your own external Redis instance, such as a Redis API compatible service such as AWS ElastiCache.
About
Instead of using the built-in local RedisĀ®*
1 instances, you can achieve higher availability, disaster recovery, and enhanced control and security by bringing your own external Redis instances. For more information, see About Redis.
Step 1: Configure your Redis provider
Configure your external Redis instance in your cloud provider to work with Gloo Mesh Enterprise.
Using a Redis or an AWS ElastiCache instance in cluster mode is currently not supported.
Create a Redis instance external to your cluster environment, such as Redis Cloud or a Redis API-compatible cloud provider service like AWS ElastiCache.
Make sure that you can connect to your Redis instance from the Gloo management cluster. These steps can vary by provider, but the following network settings are common.
- Create the Redis instance in the same virtual private network (VPC) as the Kubernetes cluster where you install Gloo Mesh Enterprise.
- Create the Redis instance and the Kubernetes cluster in the same security group.
- Make sure that the security group allows inbound and outbound connections between the cluster and Redis instance.
- If the cluster is behind a firewall, you might need to add the public IP address of the Redis instance to your allowlist.
Configure requirements specific to your cloud provider.
AWS ElastiCache
When setting up your AWS ElastiCache instance, consider the following deployment options:
- Use a high availability (HA) setup consisting of at least 1 primary and 1 replica instance.
- Single-region, multi-zone HA with automatic failover by using a single, shared Redis instance with one primary and up to five replicas. For more information, see the Amazon ElastiCache docs.
- Multi-cluster, multi-region active/passive HA with one active management cluster and one passive management cluster in different regions. For more information, see the Solo blog and the Amazon ElastiCache Redis OSS with Global Datastores docs.
- Use non-clustered mode. Cluster mode is currently not supported.
- Set up multiple availability zones (multi-AZ) with auto-failover enabled.
- Set the
maxmemory-policy
directive tonoeviction
. For more information about eviction policies, see the Redis docs. - Use node types with TLS Offloading and Enhanced I/O Multiplexing.
- Although optional, enabling TLS is recommended.
- Disable the default user and enforce RBAC rules for your ElastiCache instance. For more information, see the AWS docs.
- Monitor performance metrics including CPU utilization, memory usage, network throughput, and cache hit ratio.
- Based on the size of your environment, increase the type of node instances. For more information, see Size and memory system requirements.
Step 2: Get your Gloo environment details
You can configure Redis during an initial installation or by performing an upgrade. Either way, you can configure the Redis details in your Helm values file.
Installation
If you have not installed Gloo Mesh Enterprise yet, prepare details for your installation.
Decide the Helm release namespace to use. The example uses the names suggested throughout the documentation,
gloo-platform
andgloo-mesh
.export GLOO_MESH_RELEASE_NAMESPACE=gloo-mesh
Create a Helm values file for your installation, such as
gloo-single.yaml
for a single cluster ormgmt-plane.yaml
anddata-plane.yaml
for a multicluster environment.
Upgrade
If you are upgrading Gloo Mesh Enterprise to set up Redis, get the details of your installation.
Save the details of your Gloo Mesh Enterprise installation, including the Helm chart namespace for the component that you want to set up the Redis instance for. In the following example, the release name is
gloo-platform
and the namespace isgloo-mesh
, but you can update your values accordingly.export GLOO_MESH_RELEASE_NAMESPACE=gloo-mesh
To get the values for your own release, run the following Helm command.
helm list --all-namespaces
NAME NAMESPACE REVISION UPDATED STATUS CHART gloo-platform gloo-mesh 1 2023-09-01 13:26:56.061102 -0400 EDT deployed gloo-platform-2.7.0-beta1
Get your Gloo Platform Helm values file. For more information, see Upgrade guide.
Single cluster:
helm get values gloo-platform -o yaml -n gloo-mesh > gloo-single.yaml open gloo-single.yaml
Multicluster:
helm get values gloo-platform -n gloo-mesh -o yaml --kube-context $MGMT_CONTEXT > mgmt-plane.yaml open mgmt-plane.yaml
helm get values gloo-platform -n gloo-mesh -o yaml --kube-context $REMOTE_CONTEXT > data-plane.yaml open data-plane.yaml
Step 3: Set up the Redis store
Through Helm, you can configure Redis to store the data for several Gloo Mesh Enterprise components, as described in About Redis. After, Gloo deploys a Redis instance for each one based on the details that you configure.
When you install or upgrade Gloo Mesh Enterprise, include the following Helm values to enable each component to use your own external Redis instance.
Step 4: Set up Redis authentication
Choose how to authenticate with your Redis cluster. Depending on your Redis provider, you might be required to use a specific authentication method, such as TLS certificates or a username and password.
TLS-encryption
For TLS-encrypted communication with Redis, the steps vary by provider.
Enable TLS encryption for your Redis instance. For more information, consult the documentation for your cloud provider, such as the Redis Cloud docs or the AWS ElastiCache docs.
If applicable, get the TLS certificates from your cloud provider.
Optional if you have TLS certificates: Create a Kubernetes secret to store the TLS credentials in the same namespace as the Gloo component that uses the Redis instance, such as
gloo-mesh
for the management server or UI. Run the following command from the local directory where your TLS certificates and key were created. Update the filenames of the certificate as necessary. Note: If you do not have certificates, such as if you use a cloud provider like AWS ElastiCache, skip this step.kubectl apply -f - <<EOF apiVersion: v1 kind: Secret type: Opaque metadata: name: "redis-certs" namespace: $GLOO_MESH_RELEASE_NAMESPACE data: ca.crt: $(cat ca.crt | base64 | tr -d '\n') EOF
Prepare the
redis.certs
section of the Helm values file that you previously retrieved with the details of the secret that you previously created.
Password authentication
For a username and password auth, complete the following steps.
Enable password auth for your Redis instance.
Save the following values as environment variables.
- Username: If your Redis instance does not have a username, you can omit this value.
- Password: Note the password string.
export REDIS_UN=default export REDIS_PW=<password>
Create a Kubernetes secret to store the credentials in the same namespace as the management server.
kubectl apply -f- <<EOF apiVersion: v1 kind: Secret type: Opaque metadata: name: redis-auth-secrets namespace: $GLOO_MESH_RELEASE_NAMESPACE stringData: # If Redis doesn't have an explicit username, omit the username field. username: "$REDIS_UN" password: "$REDIS_PW" EOF
Prepare the
redis.auth
section of the Helm values file that you previously retrieved.
Step 5: Configure the telemetry pipeline
If you enable the insights engine and bring your own Redis instance, you must also update the Gloo telemetry pipeline installation in your Helm values file with the details for your Redis instance.
Customize the telemetry collector with an extra exporter that sends data to your own Redis instance. The example includes required stream attributes. Update the following fields with the details of your Redis instance.
endpoint
: Provide the address to your own external Redis instance. For cloud instances, this value is the primary cloud endpoint that the Gloo component in your cluster can access, such asmy-redis.use2.cache.amazonaws.com:6379
tls
: Provide the details to use for the TLS connection. For cloud instance like AWS ElastiCache that uses a public CA so that you do not have a CA certificate, set the TLS object to empty{}
.username
: If you use Redis auth password security, include the password in this field. If you do not have a username, omit this field.password
: If you use Redis auth password security, include the password in this field.
telemetryCollectorCustomization: extraExporters: redisstream/ui: endpoint: <my-redis-instance-endpoint> expire: 30m stream: ui stream_attributes: resource_attributes: - cluster_name - source - component - k8s.pod.name - k8s.container.name tls: {} username: <username> password: <password>
Multicluster setup: Customize the telemetry gateway with an extra exporter that sends data to your own Redis instance. The example includes required stream attributes and an example for AWS ElastiCache. Update the following fields with the details of your Redis instance.
endpoint
: Provide the address to your own external Redis instance. For cloud instances, this value is the primary cloud endpoint that the Gloo component in your cluster can access, such asmy-redis.use2.cache.amazonaws.com:6379
tls
: Provide the details to use for the TLS connection. For cloud instance like AWS ElastiCache that uses a public CA so that you do not have a CA certificate, set the TLS object to empty{}
.username
: If you use Redis auth password security, include the password in this field. If you do not have a username, omit this field.password
: If you use Redis auth password security, include the password in this field.
telemetryGatewayCustomization: extraExporters: redisstream/basic: endpoint: <my-redis-instance-endpoint> tls: {} password: <redis-password> redisstream/ui: endpoint: <my-redis-instance-endpoint> expire: 30m stream: ui stream_attributes: resource_attributes: - cluster_name - source - component - k8s.pod.name - k8s.container.name tls: {} username: <username> password: <password>
Now, the Gloo telemetry pipeline is ready to use your own Redis instance to store logs about the insights. Later, you can set up Istio insights to display in the Gloo UI by configuring the logs/analyzer
pipeline in Gloo telemetry. For more information, see Add Istio insights.
Step 6: Configure API key details for the developer portal
If you use the developer portal, configure details related to storing API keys in the Redis instance for the external auth service. By default, the configuration that you set for redisStore.extAuthService
is also used by the portal server. This way, the portal server can write API keys that protect the APIs exposed in the developer portal, and the external auth service can read the same API keys to authenticate requests.
redisStore.extAuthService.client.secretKey
: Replace with a random string to use to hash the API keys that are stored in Redis. The example uses"ThisIsSecret"
.
redisStore:
extAuthService:
client:
secretKey: "ThisIsSecret"
Step 7: Configure other Redis settings
Further configure the default Redis values. For specific fields and more information, check the Helm reference for each component of the redisStore
section.
Common configuration sections include:
- Client connection details, such as redirects, retries, and max connections.
Step 8: Install or upgrade with Helm
You can continue to install or upgrade Gloo Mesh Enterprise to use your own Redis instance. Include the Helm values file that you prepared in the previous steps.
The following example commands are provided for quick reference. For more details, consult the install or upgrade guides.
Step 9: Verify the connection
To review the data that is stored in Redis, you can connect to the local Redis instance. If you bring your own Redis, you must use those connection values. For more information, see your Redis provider docs, such as Amazon ElastiCache or Google Cloud Memorystore.
Choose between using the Redis CLI or RedisInsights.
Redis CLI
You can use the Redis CLI that is built into your deployment to review the data in your database.
Install Redis in your cluster so that you can use the
redis-cli
to connect to your Redis instance.From your terminal, log in to the Redis pod and open a Redis CLI shell.
Run Redis CLI commands, such as
monitor
orkeys
. For more information, see the Redis docs.Example response of data populated by the management server:
keys * 1) "$CLUSTER_NAME~apps~v1~Deployment" 2) "reports#$CLUSTER_NAME#httpbin#$CLUSTER_NAME" 3) "$CLUSTER_NAME~install.istio.io~v1alpha1~IstioOperator" 4) "reports#$CLUSTER_NAME#gloo-mesh#globalcccfde00-e4a9-4867-b6c0-5df8601f6e1e" 5) "$CLUSTER_NAME~admissionregistration.k8s.io~v1~MutatingWebhookConfiguration" 6) "reports#idx" 7) "reports#$CLUSTER_NAME#store#$CLUSTER_NAME" 8) "gloo.mesh.key.set"
Example response of rate limit counters:
monitor OK 1681223733.198361 [0 10.xxx.x.x:xxxxx] "INCRBY" "tree|solo.io|generic_key^gloo-mesh.rl-server-config-gloo-mesh-$CLUSTER_NAME-rate-limiter|generic_key^counter|1718236800" "1"
Example response of API keys for the external auth service and portal server add-ons. Note that the API key name and value are hashed based on the
apiKeyStorage.secretKey
that you configured when setting up the add-ons.keys * 1) "1bbe3f25-5a2a-4797-8ac0-d59c7480377a" 2) "abcmHk2Iwcy3VL+NNswyrs3B4YIEx39BG84jzwcoWB8=" get "1bbe3f25-5a2a-4797-8ac0-d59c7480377a" "xksmHk2Iwcy3VL+NNswyrs3B4YIEx39BG84jzwcoWB8=" get "abcmHk2Iwcy3VL+NNswyrs3B4YIEx39BG84jzwcoWB8=" "{\"api_key\":\"abcmHk2Iwcy3VL+NNswyrs3B4YIEx39BG84jzwcoWB8=\",\"labels\":[\"first.last@example.com\"],\"metadata\":{\"config_id\":\"gloo-mesh.api-key-auth-default-cluster-name-mgt-ext-auth-service\",\"created-ts-unix\":\"1718742070\",\"name\":\"gold-api-key-name\",\"usagePlan\":\"gold\",\"username\":\"first.last@example.com\"},\"uuid\":\"1bbe3f25-5a2a-4797-8ac0-d59c7480377a\"}"
RedisInsights
RedisInsights is a desktop application that you can install for UI- and CLI-based interactions. For more information, see the Redis docs.
- Install RedisInsights. The steps vary depending on where you want to install the app.
- On your local machine: Download RedisInsights.
- In your Kubernetes cluster: See the Redis docs.
- Enable port forwarding on the Redis deployment. Update the namespace and deployment to match the Redis instance in your cluster.
kubectl port-forward -n gloo-mesh deploy/gloo-mesh-redis 6379
- Launch the RedisInsights app.
- Click + Add Redis Database and enter in your localhost and port details, which by default are
127.0.0.1:6379
. - Click the database that you just added.
- Use the UI or CLI to explore the data. For example, you can click the Browser tab (the key icon) and toggle the Key view to explore all of the keys in the database instance, as shown in the following screenshot.
*
Redis is a registered trademark of Redis Ltd. Any rights therein are reserved to Redis Ltd. Any use by Solo.io, Inc. is for referential purposes only and does not indicate any sponsorship, endorsement or affiliation between Redis and Solo.io. ↩︎