Portal server
Review how the Gloo portal server backs up data and your setup options to make the backing database more resilient.
What data gets stored
The portal server stores the following data in the backing database. For an example of how to check the data that gets stored, see Review data in Redis.
- API keys that end users can use to authenticate to destinations and routes that are protected by an external auth policy that the external auth server enforces.
How the portal and external auth servers share backing storage
To use Gloo Portal, you must configure both the portal server and external auth service to use the same backing storage. Review the following diagram for more information.
- As the portal admin, you configure the backing storage instance for both the external auth and portal servers. For more information, see Backing storage options.
- As an end user, you use the developer portal to generate an API key. Then, the Gloo portal server writes this API key to the shared backing storage database, such as Redis.
- As an end user, you include the API key in subsequent requests to the API products that you want to access. The Gloo ingress gateway receives your requests and checks the API’s route for external auth policies. The Gloo external auth server reads the API key value from the shared backing storage database, such as Redis. If the API key matches a valid value, then the Gloo gateway sends the request to the API product and you get back a successful response.
For steps to try out this process yourself, see Verify that external auth works with API keys in a backing storage database.
Backing storage options
Review the following options for setting up backing storage with Redis®*
1 instances.
Option | Use case | Benefits | Drawbacks |
---|---|---|---|
Default (no backing storage) | Local testing in single clusters |
|
|
Built-in local Redis | Proof of concept and staging environments in single clusters |
|
|
Bring your own Redis | Production-level, multicluster environments |
|
|
Shared or separate instances
In Gloo Mesh Gateway, many components use Redis to store data. Several of these components must share the same instance in order to function properly, as follows:
- Management server, UI, and insights engine (which is also used as part of the OTel pipeline).
- External auth service and developer portal server.
- Rate limiter, which is often shared with the external auth service and portal server, but does not have to be.
If you enable the default, Solo-provided, local Redis options, these Redis instances are created separately.
If you bring your own Redis, you can choose to share the Redis instances or not. For internal compliance or security purposes, you might want separate Redis instances for each of these use cases. As such, the following guide assumes a separate deployment, and uses separate secrets. However, you can also share a single external Redis instance and update the names accordingly.
Default (no backing storage)
By default, no backing storage is configured for the portal server. For testing purposes, you can manually create backing storage, such as saving an API key’s details in a Kubernetes secret. For example steps, see Require API key external auth for Gloo Portal.
Built-in local Redis
When you install Gloo Platform, a local Redis instance redis
is set up in the gloo-mesh
namespace of the workload cluster. The rate limiting server automatically reads and writes to this Redis instance. However, you must manually configure the Gloo external auth and Gloo portal servers to read from and write data to this Redis instance.
Using a local Redis instance is not a recommended practice for production. To achieve higher availability, disaster recovery and add other security features, bring your own Redis instance instead.
Include the same local Redis settings for the external auth and portal servers when you install or upgrade Gloo Platform. Review the following table to understand the matching settings. For more information, check the Helm reference.
extAuthService.extAuth setting glooPortalServer setting Description apiKeyStorage.name and apiKeyStorage.enabled apiKeyStorage.redis.enabled Use Redis as the backing storage for API keys. apiKeyStorage.secretKey apiKeyStorage.secretKey The string value that you want to use to hash API keys before they are stored in the Redis database. N/A apiKeyStorage.configPath The path to the Redis configuration file. apiKeyStorage.config.host apiKeyStorage.redis.address The host that the Redis instance is available on. When using the built-in local Redis, this value is set to the local redis
service in thegloo-mesh
namespace:redis.gloo-mesh:6379
.Continue with the single-cluster or multicluster install guides, or the upgrade guide for more instructions to finish your installation or upgrade.
Bring your own Redis
Instead of using the built-in local Redis instance, you can achieve higher availability, disaster recovery, and enhanced control and security by bringing your own Redis cluster.
When you install Gloo Mesh Gateway, you can choose to use add-ons such as rate limiter, external auth, or portal (for Gloo Mesh Gateway). These add-ons often share the same backing Redis instance. Even if you want to use the same Redis instance for all of these add-ons, you must configure the Redis details for each add-on separately in the Helm installation. This guide shows how to configure the details for the external auth service and portal server.
Keep in mind that by bringing your own Redis instance, you configure both the external auth service and portal server to read from and write to the same Redis database. Any settings for your Redis instance, such as the maximum number of retries, are shared between these components.
Step 1: Choose a Redis provider
You can choose between a local or externally hosted Redis instance.
- For a local instance: Install the Redis instance in the same cluster as the component that the Redis backs. For example, for the Gloo management server, install the Redis instance in the management cluster. For an add-on, install the Redis instance in the same workload as the rate limiter, external auth service, or portal server.
- For an external cloud instance: Create an instance outside your cluster environment, such as Redis Enterprise or a cloud provider service like AWS ElastiCache.
Using a Redis or AWS ElastiCache instance in cluster mode is currently not supported.
Step 2: Configure your Redis instance
To create or use an existing Redis instance, make sure that you can connect to your Redis instance from the Gloo management cluster. These steps can vary by provider, but the following guidelines are common.
- Local instances: The Gloo management server and Redis instances must be able to communicate. For example, you might deploy them in the same Kubernetes namespace.
- Cloud instances: Often, you must configure networking settings such as the following:
- Create the Kubernetes cluster and the Redis instance in the same virtual private network (VPC).
- Create the Kubernetes cluster and the Redis instance in the same security group.
- Make sure that the security group allows inbound and outbound connections between the cluster and Redis instance.
- If the cluster is behind a firewall, you might need to add the public IP address of the Redis cluster to your allowlist.
- Provider-specific considerations: Continue reading for requirements per Cloud provider.
AWS ElastiCache
When setting up your AWS ElastiCache instance, consider the following deployment options:
- Use a high availability (HA) setup consisting of at least 1 primary and 1 replica instance.
- Use non-clustered mode. Cluster mode is currently not supported.
- Set up multiple availability zones (multi-AZ) with auto-failover enabled.
- Set the
maxmemory-policy
directive tonoeviction
. For more information about eviction policies, see the Redis docs. - Use node types with TLS Offloading and Enhanced I/O Multiplexing.
- Although optional, enabling TLS is recommended.
- Monitor performance metrics including CPU utilization, memory usage, network throughput, and cache hit ratio.
- Based on the size of your environment, increase the type of node instances. For more information, see Size and memory system requirements.
Step 3: Get your Gloo environment details
You can configure Redis during an initial installation or by performing an upgrade. Either way, you can configure the Redis details in your Helm values file.
Installation
If you have not installed Gloo Mesh Gateway yet, prepare details for your installation.
Decide the Helm release namespace to use. The example uses the names suggested throughout the documentation,
gloo-platform
andgloo-mesh
.export GLOO_MESH_RELEASE_NAMESPACE=gloo-mesh
Create a Helm values file for your installation, such as
gloo-single.yaml
for a single cluster ormgmt-plane.yaml
anddata-plane.yaml
for a multicluster environment.
Upgrade
If you are upgrading Gloo Mesh Gateway to set up Redis, get the details of your installation.
Save the details of your Gloo Mesh Gateway installation, including the Helm chart namespace for the component that you want to set up the Redis instance for. In the following example, the release name is
gloo-platform
and the namespace isgloo-mesh
, but you can update your values accordingly.export GLOO_MESH_RELEASE_NAMESPACE=gloo-mesh
To get the values for your own release, run the following Helm command.
helm list --all-namespaces
NAME NAMESPACE REVISION UPDATED STATUS CHART gloo-platform gloo-mesh 1 2023-09-01 13:26:56.061102 -0400 EDT deployed gloo-platform-2.5.12
Get your Gloo Platform Helm values file. For more information, see Upgrade guide.
Single cluster:
helm get values gloo-platform -o yaml -n gloo-mesh > gloo-single.yaml open gloo-single.yaml
Multicluster:
helm get values gloo-platform -n gloo-mesh -o yaml --kube-context $MGMT_CONTEXT > mgmt-plane.yaml open mgmt-plane.yaml
helm get values gloo-platform -n gloo-mesh -o yaml --kube-context $REMOTE_CONTEXT > data-plane.yaml open data-plane.yaml
Step 4: Set up Redis authentication
Choose how to authenticate with your Redis cluster. Depending on your Redis provider, you might be required to use a specific authentication method, such as TLS certificates or a username and password.
TLS-encryption
For TLS-encrypted communication with Redis, the steps vary by provider.
Enable TLS encryption for your Redis instance.
- For local instances, you might create your own certificates such as with
openssl
orcert-manager
to make the public Certificate Authority (CA) certificate,ca.crt
. Then, you create a secret with this CA certificate so that the Gloo client can establish trust for the TLS connection to the Redis server instance. Note that server-side credentials for a mutual TLS (mTLS) connection are currently not supported. - For cloud instances, the provider often manages the certificates for you. For example, AWS ElastiCache encrypts in-transit communication with their own public CA. Note: Most up-to-date apps include CA certificates that trust the AWS CA by default. However, if after setup, you notice timeouts and
SSL_connect failed: certificate verify failed
messages in the logs, your app might not have the CA certificates. Depending on your app, you can update the CA certificates such as by logging into the pod and runningupdate-ca-certificates
. For an example, see the Bring your own Redis deployment example in the verification guide.
- For local instances, you might create your own certificates such as with
For local instances: Create a Kubernetes secret to store the TLS credentials in the same namespace as the add-on. Run the following command from the local directory where your TLS certificates and key were created. Update the filenames of the certificates as necessary. Note: If you do not have certificates, such as if you use a cloud provider like AWS ElastiCache, skip this step.
For the portal server, the secret must be namedportal-redis-certs
. For the ext auth service, you can use the same secret, or create another secret with the same credentials.kubectl apply -f - <<EOF apiVersion: v1 kind: Secret type: Opaque metadata: name: "portal-redis-certs" namespace: $GLOO_MESH_RELEASE_NAMESPACE labels: app.kubernetes.io/managed-by: Helm annotations: meta.helm.sh/release-name: $GLOO_MESH_RELEASE_NAME meta.helm.sh/release-namespace: $GLOO_MESH_RELEASE_NAMESPACE data: ca.crt: $(cat ca.crt | base64 | tr -d '\n') EOF
Prepare the
redis.certs
section of the Helm values file that you previously retrieved.
Password authentication
For a username and password auth, complete the following steps.
Enable password auth for your Redis instance.
Save the following values as environment variables.
- Username: If your Redis instance does not have a username, use
default
. - Password: Note the password string.
export REDIS_UN=default export REDIS_PW=<password>
- Username: If your Redis instance does not have a username, use
Create a Kubernetes secret to store the credentials in the same namespace as the add-on.
For the portal server, the secret must be named portal-redis-credentials
. For the ext auth service, you can use the same secret, or create another secret with the same credentials.
kubectl apply -f- <<EOF
apiVersion: v1
kind: Secret
type: Opaque
metadata:
name: portal-redis-credentials
namespace: $GLOO_MESH_ADDONS_NAMESPACE
labels:
app.kubernetes.io/managed-by: Helm
annotations:
meta.helm.sh/release-name: $GLOO_MESH_RELEASE_NAME
meta.helm.sh/release-namespace: $GLOO_MESH_ADDONS_NAMESPACE
stringData:
#If Redis doesn't have an explicit username, specify ‘default’.
username: "$REDIS_UN"
password: "$REDIS_PW"
EOF
Prepare the
redis.auth
section of the Helm values file that you previously retrieved.extAuthService: enabled: true extAuth: apiKeyStorage: name: redis enabled: true redis: auth: # Enable authentication for Redis with the username and password in the referenced secret. enabled: true # The name of the secret in the same namespace as the add-on. secretName: portal-redis-credentials # The key name in the data section of the Kubernetes secret # that has the password. passwordKey: password # The key name in the data section of the Kubernetes secret # that has the username. usernameKey: username glooPortalServer: enabled: true apiKeyStorage: redis: auth: # Enable authentication for Redis with the username and password in the referenced secret. enabled: true # The name of the secret in the same namespace as the add-on. # The secret must have the following name. secretName: portal-redis-credentials # The key name in the data section of the Kubernetes secret # that has the password. passwordKey: password # The key name in the data section of the Kubernetes secret # that has the username. usernameKey: username
Step 5: Configure other Redis settings
Complete the redis
section of your Helm values file. For more settings, see the Helm reference.
Enable the use of a Redis instance to store API keys for the external auth service and the portal server.
extAuthService: enabled: true extAuth: apiKeyStorage: name: redis enabled: true glooPortalServer: apiKeyStorage: redis: enabled: true
Configure a secret key to use to hash the API keys before they are stored in the backing Redis instance. The following example uses
ThisIsSecret
, but you can provide your own string value.Need to use a signing key for JWTs or other tokens used by the external auth service? Configure theextAuthService.extAuth.signingKey
orsigningKeyFile
. For more information, see the Helm reference docs.extAuthService: extAuth: apiKeyStorage: # Replace with a random string to use to generate hash values for API keys. secretKey: "ThisIsSecret" glooPortalServer: apiKeyStorage: redis: # Replace with the same random string to use to generate hash values for API keys. secretKey: "ThisIsSecret"
Include the address to use to connect to your Redis cluster.
- For local instances, this value is typically the Kubernetes service name that the add-on can access, such as
gloo-mesh-redis.gloo-mesh
. - For a cloud instance, this value is typically the primary endpoint, such as
master.my-redis-name.cache.amazonaws.com
for AWS ElastiCache or10.xxx.xx.xx
for Google Cloud Memorystore. - Include the port, which defaults to
6379
.
extAuthService: extAuth: apiKeyStorage: config: host: "master.my-redis-name.cache.amazonaws.com:6379" glooPortalServer: apiKeyStorage: redis: address: "master.my-redis-name.cache.amazonaws.com:6379"
- For local instances, this value is typically the Kubernetes service name that the add-on can access, such as
Optionally configure other settings. For more information and possible settings, see the Helm reference docs.
Step 6: Upgrade the Helm installation
Continue with the Upgrade guide to reinstall Gloo Mesh Gateway with your own Redis instance. Include the Helm values file that you prepared in the previous steps.
The following example Helm values file shows a basic installation with the external auth service and portal server sections filled out for an AWS ElastiCache Redis instance that uses TLS and auth.
extAuthService:
enabled: true
extAuth:
apiKeyStorage:
name: redis
enabled: true
config:
host: "master.my-redis-name.cache.amazonaws.com:6379"
db: 0
redis:
# Enable authentication for Redis with the username and password in the referenced secret.
auth:
enabled: true
secretName: portal-redis-credentials
# Enable TLS connections to the Redis instance.
certs:
enabled: true
# Enable using a TLS connection with the cloud Redis instance.
config:
socketType: tls
# Replace with a random string to use to generate hash values for API keys.
secretKey: "ThisIsSecret"
glooPortalServer:
enabled: true
apiKeyStorage:
redis:
enabled: true
address: "master.my-redis-name.cache.amazonaws.com:6379"
# Enable authentication for Redis with the username and password in the referenced secret.
auth:
enabled: true
# The secret must have the following secret name.
secretName: portal-redis-credentials
# Enable TLS connections to the Redis instance.
certs:
enabled: true
# Replace with a random string to use to generate hash values for API keys.
secretKey: "ThisIsSecret"
Verify the connection
To verify the connection, make sure that the external auth service and portal server are configured to use the same Redis instance. Then, verify the connection to the Redis instance.
Verify the Redis configuration of the add-ons
Verify that the external auth and portal server add-ons are configured to use the same Redis instance.
Verify that any necessary secrets are created.
kubectl get secrets -n gloo-mesh
Example output:
NAME TYPE DATA AGE ext-auth-service-api-key-secret-key Opaque 1 47s ext-auth-service-api-key-storage Opaque 1 47s ext-auth-service-signing-key Opaque 1 47s portal-storage-config Opaque 1 47s portal-redis-certs Opaque 1 47s portal-redis-credentials Opaque 1 47s
Log in to the portal server and external auth service pods, and verify that the host information is stored along the config path that you set.
Verify the Redis connection
To review the data that is stored in Redis, you can connect to the local Redis instance. If you bring your own Redis, you must use those connection values. For more information, see your Redis provider docs, such as Amazon ElastiCache or Google Cloud Memorystore.
Choose between using the Redis CLI or RedisInsights.
Redis CLI
You can use the Redis CLI that is built into your deployment to review the data in your database.
Install Redis in your cluster so that you can use the
redis-cli
to connect to your Redis instance.From your terminal, log in to the Redis pod and open a Redis CLI shell.
Run Redis CLI commands, such as
monitor
orkeys
. For more information, see the Redis docs.Example response of data populated by the management server:
keys * 1) "$CLUSTER_NAME~apps~v1~Deployment" 2) "reports#$CLUSTER_NAME#httpbin#$CLUSTER_NAME" 3) "$CLUSTER_NAME~install.istio.io~v1alpha1~IstioOperator" 4) "reports#$CLUSTER_NAME#gloo-mesh#globalcccfde00-e4a9-4867-b6c0-5df8601f6e1e" 5) "$CLUSTER_NAME~admissionregistration.k8s.io~v1~MutatingWebhookConfiguration" 6) "reports#idx" 7) "reports#$CLUSTER_NAME#store#$CLUSTER_NAME" 8) "gloo.mesh.key.set"
Example response of rate limit counters:
monitor OK 1681223733.198361 [0 10.xxx.x.x:xxxxx] "INCRBY" "tree|solo.io|generic_key^gloo-mesh.rl-server-config-gloo-mesh-$CLUSTER_NAME-rate-limiter|generic_key^counter|1718236800" "1"
Example response of API keys for the external auth service and portal server add-ons. Note that the API key name and value are hashed based on the
apiKeyStorage.secretKey
that you configured when setting up the add-ons.keys * 1) "1bbe3f25-5a2a-4797-8ac0-d59c7480377a" 2) "abcmHk2Iwcy3VL+NNswyrs3B4YIEx39BG84jzwcoWB8=" get "1bbe3f25-5a2a-4797-8ac0-d59c7480377a" "xksmHk2Iwcy3VL+NNswyrs3B4YIEx39BG84jzwcoWB8=" get "abcmHk2Iwcy3VL+NNswyrs3B4YIEx39BG84jzwcoWB8=" "{\"api_key\":\"abcmHk2Iwcy3VL+NNswyrs3B4YIEx39BG84jzwcoWB8=\",\"labels\":[\"first.last@example.com\"],\"metadata\":{\"config_id\":\"gloo-mesh.api-key-auth-default-cluster-name-mgt-ext-auth-service\",\"created-ts-unix\":\"1718742070\",\"name\":\"gold-api-key-name\",\"usagePlan\":\"gold\",\"username\":\"first.last@example.com\"},\"uuid\":\"1bbe3f25-5a2a-4797-8ac0-d59c7480377a\"}"
RedisInsights
RedisInsights is a desktop application that you can install for UI- and CLI-based interactions. For more information, see the Redis docs.
- Install RedisInsights. The steps vary depending on where you want to install the app.
- On your local machine: Download RedisInsights.
- In your Kubernetes cluster: See the Redis docs.
- Enable port forwarding on the Redis deployment. Update the namespace and deployment to match the Redis instance in your cluster.
kubectl port-forward -n gloo-mesh deploy/gloo-mesh-redis 6379
- Launch the RedisInsights app.
- Click + Add Redis Database and enter in your localhost and port details, which by default are
127.0.0.1:6379
. - Click the database that you just added.
- Use the UI or CLI to explore the data. For example, you can click the Browser tab (the key icon) and toggle the Key view to explore all of the keys in the database instance, as shown in the following screenshot.
*
Redis is a registered trademark of Redis Ltd. Any rights therein are reserved to Redis Ltd. Any use by Solo.io, Inc. is for referential purposes only and does not indicate any sponsorship, endorsement or affiliation between Redis and Solo.io. ↩︎