Set up a demo for testing

Set up a demo environment to try out Gloo Mesh Enterprise features, such as traffic and security policies.

Before you begin

Make sure that you have the following tools installed on your local system.

Step 1: Create your clusters

Use Kubernetes or OpenShift clusters in your cloud provider, or create local clusters with a project such as Kind or K3s.

  1. Create three clusters: one cluster to run the management components, and two clusters to run your workloads. For example, you might have three clusters that are named mgmt-cluster, cluster-1, and cluster-2.
  2. Save the cluster names and Kubernetes contexts as the following environment variables.
    export MGMT_CLUSTER=mgmt-cluster
    export REMOTE_CLUSTER1=cluster-1
    export REMOTE_CLUSTER2=cluster-2
    export MGMT_CONTEXT=mgmt-cluster
    export REMOTE_CONTEXT1=cluster-1
    export REMOTE_CONTEXT2=cluster-2

Step 2: Configure the locality labels for the nodes

Gloo Mesh uses Kubernetes labels on the nodes in your clusters to indicate locality for the services that run on the nodes. For more information, see the Kubernetes topology and Istio locality documentation.

Verify that your nodes have locality labels

Verify that your nodes have at least region and zone labels. If so, and you do not want to update the labels, you can skip the remaining steps.

kubectl get nodes --context $REMOTE_CONTEXT1 -o jsonpath='{.items[*].metadata.labels}'
kubectl get nodes --context $REMOTE_CONTEXT2 -o jsonpath='{.items[*].metadata.labels}'

Example output with region and zone labels:


Add locality labels to your nodes

If your nodes do not already have region and zone labels, you must add the labels. Depending on your cluster setup, you might add the same region label to each node, but a separate zone label per node. The values are not validated against your underlying infrastructure provider. The following example shows how you might label multizone clusters in two different regions, but you can adapt the steps for your actual setup.

  1. Label all the nodes in each cluster for the region. If your nodes have incorrect region labels, include the --overwrite flag in the command.
    kubectl label nodes --all --context $REMOTE_CONTEXT1
    kubectl label nodes --all --context $REMOTE_CONTEXT2
  2. List the nodes in each cluster. Note the name for each node.
    kubectl get nodes --context $REMOTE_CONTEXT1
    kubectl get nodes --context $REMOTE_CONTEXT2
  3. Label each node in each cluster for the zone. If your nodes have incorrect zone labels, include the --overwrite flag in the command.
    kubectl label node <cluster-1_node-1> --context $REMOTE_CONTEXT1
    kubectl label node <cluster-1_node-2> --context $REMOTE_CONTEXT1
    kubectl label node <cluster-1_node-3> --context $REMOTE_CONTEXT1
    kubectl label node <cluster-2_node-1> --context $REMOTE_CONTEXT2
    kubectl label node <cluster-2_node-2> --context $REMOTE_CONTEXT2
    kubectl label node <cluster-2_node-3> --context $REMOTE_CONTEXT2

Step 3: Install Gloo Mesh Enterprise

The following example installs Gloo Mesh Enterprise for testing purposes. For complete installation instructions, see Install Gloo Mesh.

  1. Set the Gloo Mesh Enterprise version as an environment variable.

    export GLOO_MESH_VERSION=2.1.0-beta8
  2. Install the meshctl CLI, such as with the following example command for Linux or macOS.

    curl -sL | v$GLOO_MESH_VERSION sh -
  3. Install Gloo Mesh management components. If you need to get a Gloo Mesh license key, contact an account representative.

    To unlock advanced north-south traffic management features, use a Gloo Mesh Gateway license key. For more information, see Gateway features.

    meshctl install --license $GLOO_MESH_LICENSE_KEY --kubecontext $MGMT_CONTEXT
  4. Register each workload cluster.

    meshctl cluster register \
      --kubecontext=$MGMT_CONTEXT \
      --remote-context=$REMOTE_CONTEXT1 \
      --version $GLOO_MESH_VERSION \
    meshctl cluster register \
      --kubecontext=$MGMT_CONTEXT \
      --remote-context=$REMOTE_CONTEXT2 \
      --version $GLOO_MESH_VERSION \
  5. Optional: To test rate limit or external auth policies, create a separate gloo-mesh-addons release.

  6. Optional: Verify the registration.

Step 4: Set up TLS secrets for HTTPS traffic

To test HTTPS traffic, you must create TLS secrets for the ingress gateway. The following steps use self-signed certificates for testing environments only. This setup includes two separate domain hostnames, in secret gw-ssl-1 and in secret gw-ssl-2. For more information, see Secure gateways.

Want to use different hosts? You can substitute any hosts that you want, or use wildcards (*) so that it works for any host. However, throughout the rest of the documentation, make sure to update the networking resources to match the host, such as in route tables and virtual gateways.

  1. Generate a root key to use for both your sample hostnames.
    openssl req -new -newkey rsa:4096 -x509 -sha256 -days 3650 -nodes -out gateway-root.crt -keyout gateway-root.key -subj /CN=root
  2. Create the following two configuration files.
    cat >> gw-ssl-1.conf <<EOF
    req_extensions = v3_req
    distinguished_name = req_distinguished_name
    [ v3_req ]
    basicConstraints = CA:FALSE
    keyUsage = nonRepudiation, digitalSignature, keyEncipherment
    extendedKeyUsage = clientAuth, serverAuth
    subjectAltName = @alt_names
    DNS =
    cat >> gw-ssl-2.conf <<EOF
    req_extensions = v3_req
    distinguished_name = req_distinguished_name
    [ v3_req ]
    basicConstraints = CA:FALSE
    keyUsage = nonRepudiation, digitalSignature, keyEncipherment
    extendedKeyUsage = clientAuth, serverAuth
    subjectAltName = @alt_names
    DNS =
  3. Create the TLS keys and certificates for each hostname.
    openssl genrsa -out gw-ssl-1.key 2048
    openssl req -new -key gw-ssl-1.key -out gw-ssl-1.csr -subj / -config gw-ssl-1.conf
    openssl x509 -req -days 3650 -CA gateway-root.crt -CAkey gateway-root.key -set_serial 0 -in gw-ssl-1.csr -out gw-ssl-1.crt -extensions v3_req -extfile gw-ssl-1.conf
    openssl genrsa -out gw-ssl-2.key 2048
    openssl req -new -key gw-ssl-2.key -out gw-ssl-2.csr -subj / -config gw-ssl-2.conf
    openssl x509 -req -days 3650 -CA gateway-root.crt -CAkey gateway-root.key -set_serial 0 -in gw-ssl-2.csr -out gw-ssl-2.crt -extensions v3_req -extfile gw-ssl-2.conf
  4. Create the TLS secrets for each hostname in each workload cluster. The secrets must be created in the same namespace as the ingress gateway.
    kubectl --context ${REMOTE_CONTEXT1} -n istio-system create secret generic gw-ssl-1-secret \
    --from-file=tls.key=gw-ssl-1.key \
    kubectl --context ${REMOTE_CONTEXT1} -n istio-system create secret generic gw-ssl-2-secret \
    --from-file=tls.key=gw-ssl-2.key \
    kubectl --context ${REMOTE_CONTEXT2} -n istio-system create secret generic gw-ssl-1-secret \
    --from-file=tls.key=gw-ssl-1.key \
    kubectl --context ${REMOTE_CONTEXT2} -n istio-system create secret generic gw-ssl-2-secret \
    --from-file=tls.key=gw-ssl-2.key \

Step 5: Install Istio and sample apps

The following steps install a simple profile of Istio. For production-level steps, see Install Istio.

You install two sample apps in your demo setup: Bookinfo and httpbin. These sample apps are used throughout the documentation to help test connectivity, such as in the policy guides.

  1. Complete Step 3 of the Getting Started to install Istio in each workload cluster. The steps vary depending on if your platform is Kubernetes or OpenShift.

  2. Create the bookinfo namespace in each workload cluster.

    kubectl create ns bookinfo --context $REMOTE_CONTEXT1
    kubectl create ns bookinfo --context $REMOTE_CONTEXT2
  3. Complete Step 2: Deploy Bookinfo across Clusters in the Multicluster federation and isolation with Bookinfo guide.

  4. Create an httpbin app.

    kubectl --context $REMOTE_CONTEXT1 create ns httpbin
    kubectl --context $REMOTE_CONTEXT1 label namespace httpbin istio-injection=enabled
    kubectl --context $REMOTE_CONTEXT1 -n httpbin apply -f
  5. Get the IP addresses for the Istio east-west and ingress gateways, as well as the HTTP and HTTPS addresses for ingress.

    kubectl get svc -A --context ${REMOTE_CONTEXT1} | grep gateway

    In the following example, the IP addresses are as follows:

    • istio-eastwestgateway:
    • istio-ingressgateway:
    • HTTP istio-ingressgateway:
    • HTTPS istio-ingressgateway:
    istio-system   istio-eastwestgateway   LoadBalancer   15021:32746/TCP,15443:31161/TCP
    istio-system   istio-ingressgateway    LoadBalancer    15021:30194/TCP,80:30727/TCP,443:32488/TCP,15443:31475/TCP 
  6. Set the Istio gateway IP addresses as environment variables.

    export EASTWEST_GW_IP=<istio-eastwestgateway-IP>
    export INGRESS_GW_IP=<istio-ingressgateway-IP>
    export INGRESS_GW_HTTP=<istio-ingressgateway-IP>:80
    export INGRESS_GW_HTTPS=<istio-ingressgateway-IP>:443
    echo $EASTWEST_GW_IP
    echo $INGRESS_GW_IP

Optional Step 6: Install Keycloak

You might want to test how to restrict access to your applications to authenticated users, such as with external auth or JWT policies. You can install Keycloak in your cluster as an OpenID Connect (OIDC) provider.

The following steps install Keycloak in your cluster, and configure two user credentials as follows.

Install and configure Keycloak:

  1. Create a namespace for your Keycloak deployment.

    kubectl --context ${MGMT_CONTEXT} create namespace keycloak
  2. Create the Keycloak deployment.

    kubectl --context ${MGMT_CONTEXT} -n keycloak apply -f
  3. Wait for the Keycloak rollout to finish.

    kubectl --context ${MGMT_CONTEXT} -n keycloak rollout status deploy/keycloak
  4. Set the Keycloak endpoint details from the load balancer service.

    export ENDPOINT_KEYCLOAK=$(kubectl --context ${MGMT_CONTEXT} -n keycloak get service keycloak -o jsonpath='{.status.loadBalancer.ingress[0].*}'):8080
    export HOST_KEYCLOAK=$(echo ${ENDPOINT_KEYCLOAK} | cut -d: -f1)
    export PORT_KEYCLOAK=$(echo ${ENDPOINT_KEYCLOAK} | cut -d: -f2)
    export KEYCLOAK_URL=http://${ENDPOINT_KEYCLOAK}/auth
    echo $KEYCLOAK_URL
  5. Set the Keycloak admin token. If you see a parsing error, try running the curl command by itself. You might notice that your network is blocking the requests, which might require updating the security settings so that the request can be processed.

    export KEYCLOAK_TOKEN=$(curl -d "client_id=admin-cli" -d "username=admin" -d "password=admin" -d "grant_type=password" "$KEYCLOAK_URL/realms/master/protocol/openid-connect/token" | jq -r .access_token)
  6. Use the admin token to configure Keycloak with the two users for testing purposes. If you get a 401 Unauthorized error, run the previous command and try again.

    # Create initial token to register the client
    read -r client token <<<$(curl -H "Authorization: Bearer ${KEYCLOAK_TOKEN}" -X POST -H "Content-Type: application/json" -d '{"expiration": 0, "count": 1}' $KEYCLOAK_URL/admin/realms/master/clients-initial-access | jq -r '[.id, .token] | @tsv')
    export KEYCLOAK_CLIENT=${client}
    # Register the client
    read -r id secret <<<$(curl -X POST -d "{ \"clientId\": \"${KEYCLOAK_CLIENT}\" }" -H "Content-Type:application/json" -H "Authorization: bearer ${token}" ${KEYCLOAK_URL}/realms/master/clients-registrations/default| jq -r '[.id, .secret] | @tsv')
    export KEYCLOAK_SECRET=${secret}
    # Add allowed redirect URIs
    curl -H "Authorization: Bearer ${KEYCLOAK_TOKEN}" -X PUT -H "Content-Type: application/json" -d '{"serviceAccountsEnabled": true, "directAccessGrantsEnabled": true, "authorizationServicesEnabled": true, "redirectUris": ["'https://${ENDPOINT_HTTPS_GW_CLUSTER1}'/callback"]}' $KEYCLOAK_URL/admin/realms/master/clients/${id}
    # Add the group attribute in the JWT token returned by Keycloak
    curl -H "Authorization: Bearer ${KEYCLOAK_TOKEN}" -X POST -H "Content-Type: application/json" -d '{"name": "group", "protocol": "openid-connect", "protocolMapper": "oidc-usermodel-attribute-mapper", "config": {"": "group", "jsonType.label": "String", "user.attribute": "group", "id.token.claim": "true", "access.token.claim": "true"}}' $KEYCLOAK_URL/admin/realms/master/clients/${id}/protocol-mappers/models
    # Create first user
    curl -H "Authorization: Bearer ${KEYCLOAK_TOKEN}" -X POST -H "Content-Type: application/json" -d '{"username": "user1", "email": "", "enabled": true, "attributes": {"group": "users"}, "credentials": [{"type": "password", "value": "password", "temporary": false}]}' $KEYCLOAK_URL/admin/realms/master/users
    # Create second user
    curl -H "Authorization: Bearer ${KEYCLOAK_TOKEN}" -X POST -H "Content-Type: application/json" -d '{"username": "user2", "email": "", "enabled": true, "attributes": {"group": "users"}, "credentials": [{"type": "password", "value": "password", "temporary": false}]}' $KEYCLOAK_URL/admin/realms/master/users

What's next?

Now that you have Gloo Mesh Enterprise up and running, check out some of the following resources to learn more about Gloo Mesh or try other Gloo Mesh features.