Install in a multicluster setup

Install the Gloo Mesh Gateway management plane and data plane separately across multiple clusters.

Overview

In a multicluster setup, you install the Gloo management plane and gateway proxy in separate clusters.

• Gloo management plane: When you install the Gloo management plane in a dedicated management cluster, a deployment named gloo-mesh-mgmt-server is created to translate and implement your Gloo configurations.
• Data plane: Set up one or more workload clusters that are registered with and managed by the Gloo management plane in the management cluster. A deployment named gloo-mesh-agent is created to run the Gloo agent in each workload cluster. Additionally, you use the Gloo management plane to install an ingress gateway proxy in each workload cluster, as part of the Istio lifecycle management. By using Gloo-managed installations, you no longer need to manually install and manage the istiod control plane and gateway proxy in each workload cluster. Instead, you provide the Istio configuration in your gloo-platform Helm chart, and Gloo translates this configuration into managed istiod control plane and gateway proxies in the clusters.

Before you begin

1. Install the following command-line (CLI) tools.

   • helm, the Kubernetes package manager.
   • kubectl, the Kubernetes command line tool. Download the kubectl version that is within one minor version of the Kubernetes clusters you plan to use.
   • meshctl, the Solo command line tool.

     curl -sL https://run.solo.io/meshctl/install | GLOO_MESH_VERSION=v2.11.5 sh -
     export PATH=$HOME/.gloo-mesh/bin:$PATH

2. Set your Gloo Mesh Gateway license key as an environment variable. If you do not have one, contact an account representative. If you prefer to specify license keys in a secret instead, see Licensing. To check your license’s validity, you can run meshctl license check --key $(echo ${GLOO_MESH_GATEWAY_LICENSE_KEY} | base64 -w0).

   export GLOO_MESH_GATEWAY_LICENSE_KEY=<license_key>

3. Set the Gloo Mesh Gateway version. This example uses the latest version. You can find other versions in the Changelog documentation. Append -fips for a FIPS-compliant image, such as 2.11.5-fips. Do not include v before the version number.

   export GLOO_VERSION=2.11.5

4. Create or use at least two existing Kubernetes clusters. The instructions in this guide assume one management cluster and two workload clusters. The cluster name must be alphanumeric with no special characters except a hyphen (-), lowercase, and begin with a letter (not a number) to follow the Kubernetes DNS label standard.

5. For quick installations, such as for testing environments, you can install with meshctl. To customize your installation in detail, such as for production environments, install with Helm.

Install with meshctl

Quickly install Gloo Mesh Gateway by using meshctl, such as for testing purposes.

Management plane

Deploy the Gloo management plane into a dedicated management cluster.

1. Install the Gloo management plane in your management cluster. This command uses a basic profile to create a gloo-mesh namespace and install the management plane components, such as the management server and Prometheus server, in your management cluster.

   info

   meshctl install creates a self-signed certificate authority for mTLS if you do not supply your own certificates. If you prefer to set up Gloo Mesh Gateway without secure communication for quick demonstrations, include the –set common.insecure=true flag. Note that using the default self-signed CAs or using insecure mode are not suitable for production environments.

   meshctl install --profiles mgmt-server \
     --kubecontext ${MGMT_CONTEXT} \
     --set common.cluster=${MGMT_CLUSTER} \
     --set licensing.glooGatewayLicenseKey=${GLOO_MESH_GATEWAY_LICENSE_KEY}

   Note:

   • Need to use OpenShift routes instead of load balancer service types? Follow the OpenShift steps in the Helm section instead.

   1. Elevate the permission for the gloo-mesh service account. The telemetry collector agent requires the hostmount-anyuid permission to read the varlogpods, varlibdockercontainers, and cilium-run volumes.

      oc adm policy add-scc-to-group hostmount-anyuid system:serviceaccounts:gloo-mesh --context ${MGMT_CONTEXT}
   2. Install the Gloo management plane in the management cluster.

      meshctl install --profiles mgmt-server-openshift \
        --kubecontext ${MGMT_CONTEXT} \
        --set common.cluster=${MGMT_CLUSTER} \
        --set licensing.glooGatewayLicenseKey=${GLOO_MESH_GATEWAY_LICENSE_KEY}

2. Verify that the management plane pods have a status of Running.

   kubectl get pods -n gloo-mesh --context $MGMT_CONTEXT

   Example output:

   NAME                                      READY   STATUS    RESTARTS   AGE
   gloo-mesh-mgmt-server-56c495796b-cx687    1/1     Running   0          30s
   gloo-mesh-redis-8455d49c86-f8qhw          1/1     Running   0          30s
   gloo-mesh-ui-65b6b6df5f-bf4vp             3/3     Running   0          30s
   gloo-telemetry-collector-agent-7rzfb      1/1     Running   0          30s
   gloo-telemetry-collector-agent-mf5rw      1/1     Running   0          30s
   gloo-telemetry-gateway-6547f479d5-r4zm6   1/1     Running   0          30s
   prometheus-server-57cd8c74d4-2bc7f        2/2     Running   0          30s
3. Save the external address and port that your cloud provider assigned to the Gloo OpenTelemetry (OTel) gateway service. The OTel collector agents in each workload cluster send metrics to this address.

   export TELEMETRY_GATEWAY_IP=$(kubectl get svc -n gloo-mesh gloo-telemetry-gateway --context $MGMT_CONTEXT -o jsonpath="{.status.loadBalancer.ingress[0]['hostname','ip']}")
   export TELEMETRY_GATEWAY_PORT=$(kubectl get svc -n gloo-mesh gloo-telemetry-gateway --context $MGMT_CONTEXT -o jsonpath='{.spec.ports[?(@.name=="otlp")].port}')
   export TELEMETRY_GATEWAY_ADDRESS=${TELEMETRY_GATEWAY_IP}:${TELEMETRY_GATEWAY_PORT}
   echo $TELEMETRY_GATEWAY_ADDRESS

   1. Expose the telemetry gateway by using an OpenShift route. Your route might look like the following.

      oc apply --context $MGMT_CONTEXT -n gloo-mesh -f- <<EOF
      kind: Route
      apiVersion: route.openshift.io/v1
      metadata:
        name: gloo-telemetry-gateway
        namespace: gloo-mesh
      spec:
        host: gloo-telemetry-gateway.<your_domain>.com
        to:
          kind: Service
          name: gloo-telemetry-gateway
          weight: 100
        port:
          targetPort: otlp
        tls:
          termination: passthrough
          insecureEdgeTerminationPolicy: Redirect
        wildcardPolicy: None
      EOF
   2. Save the telemetry gateway’s address, which consists of the route host and the route port. Note that the port is the route’s own port, such as 443, and not the otlp port of the telemetry gateway that the route points to.

      export TELEMETRY_GATEWAY_HOST=$(oc get route -n gloo-mesh gloo-telemetry-gateway --context $MGMT_CONTEXT -o jsonpath='{.status.ingress[0].host}')
      export TELEMETRY_GATEWAY_ADDRESS=${TELEMETRY_GATEWAY_HOST}:443
      echo $TELEMETRY_GATEWAY_ADDRESS

4. Create a workspace that selects all clusters and namespaces by default, and workspace settings that enable communication across clusters. Gloo workspaces let you organize team resources across Kubernetes namespaces and clusters. In this example, you create a global workspace that imports and exports all resources and namespaces, and a workspace settings resource in the gloo-mesh-config namespace. Later, as your teams grow, you can create a workspace for each team, to enforce service isolation, set up federation, and even share resources by importing and exporting.

   kubectl apply --context $MGMT_CONTEXT -f- <<EOF
   apiVersion: admin.gloo.solo.io/v2
   kind: Workspace
   metadata:
     name: $MGMT_CLUSTER
     namespace: gloo-mesh
   spec:
     workloadClusters:
       - name: '*'
         namespaces:
           - name: '*'
   ---
   apiVersion: v1
   kind: Namespace
   metadata:
     name: gloo-mesh-config
   ---
   apiVersion: admin.gloo.solo.io/v2
   kind: WorkspaceSettings
   metadata:
     name: $MGMT_CLUSTER
     namespace: gloo-mesh-config
   spec:
     options:
       serviceIsolation:
         enabled: false
       federation:
         enabled: false
         serviceSelector:
         - {}
       eastWestGateways:
       - selector:
           labels:
             istio: eastwestgateway
   EOF

Data plane

Register each workload cluster with the Gloo management plane by deploying Gloo data plane components. A deployment named gloo-mesh-agent runs the Gloo agent in each workload cluster.

1. Register both workload clusters with the management server. These commands use basic profiles to install the Gloo agent, rate limit server, and external auth server in each workload cluster.

   meshctl cluster register $REMOTE_CLUSTER1 \
   --kubecontext $MGMT_CONTEXT \
   --remote-context $REMOTE_CONTEXT1 \
   --profiles agent,ratelimit,extauth \
   --telemetry-server-address $TELEMETRY_GATEWAY_ADDRESS
   
   meshctl cluster register $REMOTE_CLUSTER2 \
   --kubecontext $MGMT_CONTEXT \
   --remote-context $REMOTE_CONTEXT2 \
   --profiles agent,ratelimit,extauth \
   --telemetry-server-address $TELEMETRY_GATEWAY_ADDRESS

   1. Elevate the permissions for the gloo-mesh service account. The telemetry collector agent requires the hostmount-anyuid permission to read the varlogpods, varlibdockercontainers, and cilium-run volumes.

      oc adm policy add-scc-to-group hostmount-anyuid system:serviceaccounts:gloo-mesh --context $REMOTE_CONTEXT1
      oc adm policy add-scc-to-group hostmount-anyuid system:serviceaccounts:gloo-mesh --context $REMOTE_CONTEXT2
   2. Register the workload clusters.

      meshctl cluster register $REMOTE_CLUSTER1 \
       --kubecontext $MGMT_CONTEXT \
       --remote-context $REMOTE_CONTEXT1 \
       --profiles agent-openshift,ratelimit,extauth \
       --version $GLOO_VERSION \
       --telemetry-server-address $TELEMETRY_GATEWAY_ADDRESS
      
      meshctl cluster register $REMOTE_CLUSTER2 \
       --kubecontext $MGMT_CONTEXT \
       --remote-context $REMOTE_CONTEXT2 \
       --profiles agent-openshift,ratelimit,extauth \
       --version $GLOO_VERSION \
       --telemetry-server-address $TELEMETRY_GATEWAY_ADDRESS

   Note: In OpenShift 4.11 and later, you might see warnings for the pods and containers which violate the OpenShift PodSecurity "restricted:v1.24" profile, due to the elevated permissions required by Istio. You can ignore these warnings. For more info, see this article.

2. Verify that the Gloo data plane components in each workload cluster are healthy. If not, try debugging the agent.

   kubectl get pods -n gloo-mesh --context $REMOTE_CONTEXT1
   kubectl get pods -n gloo-mesh --context $REMOTE_CONTEXT2

   Example output:

   NAME                                   READY   STATUS    RESTARTS   AGE
   extauth-585565448c-smz4j               1/1     Running   0          90s
   gloo-mesh-agent-8ffc775c4-tk2z5        2/2     Running   0          90s
   gloo-telemetry-collector-agent-g8p7x   1/1     Running   0          90s
   gloo-telemetry-collector-agent-mp2wd   1/1     Running   0          90s
   rate-limit-8db4b7b5d-hq7g8             1/1     Running   0          90s

3. Verify that your Gloo Mesh Gateway setup is correctly installed. If not, try debugging the relay connection. Note that this check might take a few seconds to verify that:

   • Your Solo product licenses are valid and current.
   • The Gloo CRDs are installed at the correct version.
   • The management plane pods in the management cluster are running and healthy.
   • The agents in the workload clusters are successfully identified by the management server.
   • Any Istio installation versions are compatible with the installed Gloo version.

   meshctl check --kubecontext $MGMT_CONTEXT

   Example output:

   🟢 License status
   
   INFO  gloo-gateway enterprise license expiration is 25 Aug 24 10:38 CDT
   
   🟢 CRD version check
   
   🟢 Gloo deployment status
   
   Namespace | Name                           | Ready | Status
   gloo-mesh | gloo-mesh-mgmt-server          | 1/1   | Healthy
   gloo-mesh | gloo-mesh-redis                | 1/1   | Healthy
   gloo-mesh | gloo-mesh-ui                   | 1/1   | Healthy
   gloo-mesh | gloo-telemetry-gateway         | 1/1   | Healthy
   gloo-mesh | prometheus-server              | 1/1   | Healthy
   gloo-mesh | gloo-telemetry-collector-agent | 2/2   | Healthy
   
   🟢 Mgmt server connectivity to workload agents
   
   Cluster  | Registered | Connected Pod                                   
   cluster1 | true       | gloo-mesh/gloo-mesh-mgmt-server-65bd557b95-v8qq6
   cluster2 | true       | gloo-mesh/gloo-mesh-mgmt-server-65bd557b95-v8qq6
   
   Connected Pod                                    | Clusters
   gloo-mesh/gloo-mesh-mgmt-server-65bd557b95-v8qq6 | 2
   
   🟢 Istio compatibility check
   
   All Istio versions found are compatible

Gateway proxies

To install an Istio ingress and east-west gateway in each workload cluster, follow the steps in Deploy gateways.

Install with Helm

Customize your Gloo Mesh Gateway setup by installing with the Gloo Platform Helm chart. For more information, see the Gloo Helm chart overview.

Management plane

Deploy the Gloo management plane into a dedicated management cluster.

1. Production installations: Review Best practices for production to prepare your optional security measures. For example, before you begin your Gloo Mesh Gateway installation, you can provide your own certificates to secure the management server and agent connection, and set up secure access to the Gloo UI.

2. Install helm, the Kubernetes package manager.

3. Save the name and kubeconfig context for your management cluster in environment variables.

   export MGMT_CLUSTER=<management-cluster-name>
   export MGMT_CONTEXT=<management-cluster-context>

4. Add and update the Helm repository for Gloo.

   helm repo add gloo-platform https://storage.googleapis.com/gloo-platform/helm-charts
   helm repo update

5. Install the Gloo CRDs.

   helm upgrade -i gloo-platform-crds gloo-platform/gloo-platform-crds \
      --namespace=gloo-mesh \
      --create-namespace \
      --version=$GLOO_VERSION \
      --kube-context $MGMT_CONTEXT
6. Prepare a Helm values file to provide your customizations. To get started, you can use the minimum settings in the following profile as a basis. These settings enable all components that are required for a Gloo management plane installation.

   curl https://storage.googleapis.com/gloo-platform/helm-profiles/$GLOO_VERSION/mgmt-server.yaml > mgmt-plane.yaml
   open mgmt-plane.yaml

   curl https://storage.googleapis.com/gloo-platform/helm-profiles/$GLOO_VERSION/mgmt-server-openshift.yaml > mgmt-plane.yaml
   open mgmt-plane.yaml

7. Decide how you want to secure the relay connection between the management server and agents. In test and POC environments, you can use self-signed certificates to secure the connection. If you plan to use Gloo Mesh Gateway in production, bringing your own certificates instead is recommended. For more information, see Setup options.

   Use your preferred PKI provider to generate your root CA and intermediate CA credentials. You then store the intermediate CA credentials in your cluster so that you can leverage Gloo Mesh Gateway’s built-in capability to automatically issue and sign client TLS certificates for agents. For more information, see the Bring your own CAs with automatic client TLS certificate rotation.

   1. Create your root CA, intermediate CA, server, and telemetry gateway credentials, as well as relay identity token, and store them as the following Kubernetes secrets in the gloo-mesh namespace on the management cluster:

      • relay-root-tls-secret that holds the root CA certificate
      • relay-tls-signing-secret that holds the intermediate CA credentials
      • relay-server-tls-secret that holds the server key, TLS certificate and certificate chain information for the management server
      • relay-identity-token-secret that holds the relay identity token value
      • telemetry-root-secret that holds the root CA certificate for the certificate chain
      • gloo-telemetry-gateway-tls-secret that holds the key, TLS certificate and certificate chain for the telemetry gateway

      You can use your preferred PKI provider to create the TLS certificates or follow the steps in BYO server certificate with managed client certificate to create these TLS credentials with OpenSSl.

   2. In your Helm values file for the management server, add the following values.

      
      glooMgmtServer:
        enabled: true
        relay:
          disableCaCertGeneration: true
          signingTlsSecret:
            name: relay-tls-signing-secret
            namespace: gloo-mesh
          tlsSecret:
            name: relay-server-tls-secret
            namespace: gloo-mesh
          tokenSecret:
            key: token
            name: relay-identity-token-secret
            namespace: gloo-mesh
      telemetryCollector: 
        enabled: true
        extraVolumes: 
          - name: root-ca
            secret:
              defaultMode: 420
              optional: true
              secretName: telemetry-root-secret
          - configMap:
              items:
                - key: relay
                  path: relay.yaml
              name: gloo-telemetry-collector-config
            name: telemetry-configmap
          - hostPath:
              path: /var/run/cilium
              type: DirectoryOrCreate
            name: cilium-run
      telemetryGateway:
        enabled: true
        extraVolumes:
        - name: tls-keys
          secret:
            secretName:  gloo-telemetry-gateway-tls-secret
            defaultMode: 420
        - name: telemetry-configmap
          configMap:
            name: gloo-telemetry-gateway-config
            items:
              - key: relay
                path: relay.yaml
      telemetryGatewayCustomization:
        disableCertGeneration: true

      Helm value	Description
      glooMgmtServer.relay.disableCaCertGeneration	Disable the generation of self-signed certificates to secure the relay connection between the management server and agent.
      glooMgmtServer.relay.signingTlsSecret	Add the name and namespace of the Kubernetes secret that holds the intermediate CA credentials that you created earlier.
      glooMgmtServer.relay.tlsSecret	Add the name and namespace of the Kubernetes secret that holds the server TLS certificate for the management server that you created earlier.
      glooMgmtServer.relay.tokenSecret	Add the name, namespace, and key of the Kubernetes secret that holds the relay identity token that you created earlier.
      telemetryGateway.extraVolumes	Add the gloo-telemetry-gateway-tls-secret-custom Kubernetes secret that you created earlier to the tls-keys volume. Make sure that you also add the other volumes to your telemetry gateway configuration.
      telemetryCollector.extraVolumes	Add the telemetry-root-secret Kubernetes secret that you created earlier to the root-ca volume. Make sure that you also add the other volumes to your telemetry collector configuration.

   Use your preferred PKI provider to create the root CA, server TLS, client TLS, and telemetry gateway certificates. Then, store these certificates in the cluster so that the agent uses a client TLS certificate when establishing the first connection with the management server. No relay identity tokens are used. With this approach, you cannot use Gloo Mesh Gateway’s built-in client TLS certificate rotation capabilities. Instead, you must set up your own processes to monitor the expiration of your certificates and to rotate them before they expire. For more information, see Bring your own CAs and client TLS certificates.

   3. Create your root CA, server, client, and telemetry gateway TLS certificates. You can use your preferred PKI provider to do that or follow the steps in Create certificates with OpenSSL to create custom TLS certificates by using OpenSSL. Then, you store the following information in a Kubernetes secret in the gloo-mesh namespace on the management cluster.
      • relay-server-tls-secret that holds the server key, TLS certificate and certificate chain information for the management server
      • gloo-telemetry-gateway-tls-secret that holds the key, TLS certificate and certificate chain for the telemetry gateway
      • telemetry-root-secret that holds the root CA certificate for the certificate chain
   4. In your Helm values file for the management server, add the following values.

      
      glooMgmtServer:
        enabled: true
        relay:
          disableCa: true
          disableCaCertGeneration: true
          disableTokenGeneration: true
          tlsSecret:
            name: relay-server-tls-secret
            namespace: gloo-mesh
          tokenSecret:
            key: null
            name: null
            namespace: null
      telemetryCollector: 
        enabled: true
        extraVolumes: 
          - name: root-ca
            secret:
              defaultMode: 420
              optional: true
              secretName: telemetry-root-secret
          - configMap:
              items:
                - key: relay
                  path: relay.yaml
              name: gloo-telemetry-collector-config
            name: telemetry-configmap
          - hostPath:
              path: /var/run/cilium
              type: DirectoryOrCreate
            name: cilium-run
      telemetryGateway:
        enabled: true
        extraVolumes:
        - name: tls-keys
          secret:
            secretName:  gloo-telemetry-gateway-tls-secret
            defaultMode: 420
        - name: telemetry-configmap
          configMap:
            name: gloo-telemetry-gateway-config
            items:
              - key: relay
                path: relay.yaml
      telemetryGatewayCustomization:
        disableCertGeneration: true

      Helm value	Description
      relay.disableCa	Disable the generation of self-signed root and intermediate CA certificates and the use of identity tokens to establish initial trust between the management server and agent.
      relay.disableCaCertGeneration	Disable the generation of self-signed certificates to secure the relay connection between the management server and agent.
      relay.disableTokenGeneration	Disable the generation of relay identity tokens.
      relay.tlsSecret	Add the name and namespace of the Kubernetes secret that holds the server TLS certificate for the management server that you created earlier.
      relay.tokenSecret	Set all values to null to instruct the management server to not use identity tokens to establish initial trust with agents.
      telemetryGateway.extraVolumes	Add the gloo-telemetry-gateway-tls-secret Kubernetes secret that you created earlier to the tls-keys volume. Make sure that you also add the other volumes to your telemetry gateway configuration.
      telemetryCollector.extraVolumes	Add the telemetry-root-secret Kubernetes secret that you created earlier to the root-ca volume. Make sure that you also add the other volumes to your telemetry collector configuration.

   report

   Do not use self-signed certs for production. This setup is recommended for testing purposes only.

   You can use Gloo Mesh Gateway to generate self-signed certificates and keys for the root and intermediate CA. These credentials are used to derive the server TLS certificate for the management server and client TLS certificate for the agent. Note that this setup is not recommended for production, but can be used for testing purposes. For more information, see Self-signed CAs with automatic client certificate rotation.
   
   

   In your Helm values file for the management server, add the following values. Note that mTLS is the default mode in Gloo Mesh Gateway and does not require any additional configuration on the management server.

   
   glooMgmtServer:
     enabled: true

   Use your preferred PKI provider to create the server TLS certificate for the management server. For more information, see Bring your own server TLS certificate.

   5. Create your root CA credentials and derive a server TLS certificate. To create the TLS certificate, you can use your preferred PKI provider or follow the steps in BYO server certificate to create a custom server TLS certificate by using OpenSSL. Make sure that you have the following Kubernetes secrets in the management cluster:

      • relay-server-tls-secret-custom: The key and TLS certificate for the management server, and root CA certificate information for the certificate chain. Note that you can use a different name, but you cannot use relay-server-tls-secret as this name is reserved by the management server when creating self-signed root CAs and server TLS certificates.
      • gloo-telemetry-gateway-tls-secret-custom: The key and TLS certificate for the telemetry gateway, and root CA certificate information for the certificate chain. It is recommended to use the same credentials that you use for the management server. Note that you can use a different name, but you cannot use gloo-telemetry-gateway-tls-secret as this name is reserved by the management server when creating self-signed certificates.
      • telemetry-root-secret: The root CA certificate for the certificate chain.

   6. In your Helm values file for the management server, add the following values.

      
      glooMgmtServer:
        enabled: true
        relay:
          tlsSecret:
            name: relay-server-tls-secret-custom
        extraEnvs:
          RELAY_DISABLE_CLIENT_CERTIFICATE_AUTHENTICATION:
            value: "true"  
          RELAY_TOKEN: 
            value: "My token"
      telemetryCollector: 
        enabled: true
        extraVolumes: 
          - name: root-ca
            secret:
              defaultMode: 420
              optional: true
              secretName: telemetry-root-secret
          - configMap:
              items:
                - key: relay
                  path: relay.yaml
              name: gloo-telemetry-collector-config
            name: telemetry-configmap
          - hostPath:
              path: /var/run/cilium
              type: DirectoryOrCreate
            name: cilium-run
      telemetryGateway:
        enabled: true
        extraVolumes:
        - name: tls-keys
          secret:
            secretName:  gloo-telemetry-gateway-tls-secret-custom
            defaultMode: 420
        - name: telemetry-configmap
          configMap:
            name: gloo-telemetry-gateway-config
            items:
              - key: relay
                path: relay.yaml
      telemetryGatewayCustomization:
        disableCertGeneration: true

      Helm value	Description
      relay.tlsSecret.name	Add the name of the Kubernetes secret with the custom server TLS secret that you created earlier.
      RELAY_TOKEN	Specify the relay token that the management server and agent use to establish initial trust. When you install Gloo Mesh Gateway and set RELAY_DISABLE_CLIENT_CERTIFICATE_AUTHENTICATION to true, the connection between the management server and agent is automatically secured by using simple, server-side TLS. In a simple TLS setup, only the management server presents a certificate to authenticate its identity. The identity of the agent is not verified. To ensure that only trusted agents connect to the management server, the relay identity token is used. The relay identity token can be any string value and is stored in the relay-identity-token-secret Kubernetes secret on the management cluster. You must set the same value in glooAgent.extraEnvs.RELAY_TOKEN.value when you install the agent to allow the agent to connect to the management server.
      RELAY_DISABLE_CLIENT_CERTIFICATE_AUTHENTICATION	Set this value to true to not require a client TLS certificate from the agent to prove the agent’s identity and establish the connection with the management server. This setting is required when you want to use simple TLS to secure the connection between the management server and agent.
      telemetryGateway.extraVolumes	Add the gloo-telemetry-gateway-tls-secret-custom Kubernetes secret that you created earlier to the tls-keys volume. Make sure that you also add the other volumes to your telemetry gateway configuration.
      telemetryCollector.extraVolumes	Add the telemetry-root-secret Kubernetes secret that you created earlier to the root-ca volume. Make sure that you also add the other volumes to your telemetry collector configuration.

   report

   Do not use self-signed certs for production. This setup is recommended for testing purposes only.

   Use Gloo Mesh Gateway to create a self-signed server TLS certificate that the management server presents when workload cluster agents connect. This setup is not recommended for production, but can be used for testing purposes. For more information, see Self-signed server TLS certificate.
   
   In your Helm values file for the management server, add the following values.

   
   glooMgmtServer:
     enabled: true
     extraEnvs:
       RELAY_DISABLE_CLIENT_CERTIFICATE_AUTHENTICATION:
         value: "true"  
       RELAY_TOKEN: 
         value: "My token"

   Helm value	Description
   RELAY_TOKEN	Specify the relay token that the management server and agent use to establish initial trust. When you install Gloo Mesh Gateway and set RELAY_DISABLE_CLIENT_CERTIFICATE_AUTHENTICATION to true, the connection between the management server and agent is automatically secured by using simple, server-side TLS. In a simple TLS setup, only the management server presents a certificate to authenticate its identity. The identity of the agent is not verified. To ensure that only trusted agents connect to the management server, the relay identity token is used. The relay identity token can be any string value and is stored in the relay-identity-token-secret Kubernetes secret on the management cluster. You must set the same value in glooAgent.extraEnvs.RELAY_TOKEN.value when installing Gloo Mesh Gateway in a workload cluster to allow agents to connect to the management server.
   RELAY_DISABLE_CLIENT_CERTIFICATE_AUTHENTICATION	Set this value to true to not require a client TLS certificate from the agent to prove the agent’s identity and establish the connection with the management server. This setting is required when you want to use simple TLS to secure the connection between the management server and agent.

8. Edit the file to provide your own details for settings that are recommended for production deployments, such as the following settings.

   check_circle

   For more information about the settings you can configure:

   • See Best practices for production.
   • See all possible fields for the Helm chart by running helm show values gloo-platform/gloo-platform --version v2.11.5 > all-values.yaml. You can also see these fields in the Helm values documentation.

   | Field | Description | |——-|————||glooInsightsEngine.enabled|Enable the Gloo insights engine, which is recommended to help you improve the security and observability of your environment by creating actionable Istio insights.| |glooMgmtServer.resources.limits|Add resource limits for the gloo-mesh-mgmt-server pod, such as cpu: 1000m and memory: 1Gi.| |glooMgmtServer.safeMode
glooMgmtServer.safeStartWindow|Configure how you want the Gloo management server to handle translation after a Redis restart. For available options, see Redis safe mode options. | |glooMgmtServer.serviceOverrides.metadata.annotations|Add annotations for the management server load balancer as needed, such as AWS-specific load balancer annotations. For more information, see Deployment and service overrides.| |glooUi.auth|Set up OIDC authorization for the Gloo UI. For more information, see UI authentication. | |prometheus.enabled|Disable the default Prometheus instance as needed to provide your own. Otherwise, you can keep the default Prometheus server enabled, and deploy a production-level server to scrape metrics from the server. For more information on each option, see Best practices for collecting metrics in production. | | redis | Disable the default Redis deployment and provide your own backing database as needed. For more information, see Backing databases. | | OpenShift: glooMgmtServer.serviceType and telemetryGateway.service.type | In some OpenShift setups, you might not use load balancer service types. You can set these two deployment service types to ClusterIP, and expose them by using OpenShift routes after installation. |

9. Use the customizations in your Helm values file to install the Gloo management plane components in your management cluster.

   helm upgrade -i gloo-platform gloo-platform/gloo-platform \
      --kube-context $MGMT_CONTEXT \
      -n gloo-mesh \
      --version $GLOO_VERSION \
      --values mgmt-plane.yaml \
      --set common.cluster=$MGMT_CLUSTER \
      --set licensing.glooGatewayLicenseKey=$GLOO_MESH_GATEWAY_LICENSE_KEY

   5. Elevate the permissions of the gloo-mesh service account that will be created.

      oc adm policy add-scc-to-group anyuid system:serviceaccounts:gloo-mesh --context $MGMT_CONTEXT

   6. Install the Gloo management plane.

      helm upgrade -i gloo-platform gloo-platform/gloo-platform \
         --kube-context $MGMT_CONTEXT \
         -n gloo-mesh \
         --version $GLOO_VERSION \
         --values mgmt-plane.yaml \
         --set common.cluster=$MGMT_CLUSTER \
         --set licensing.glooGatewayLicenseKey=$GLOO_MESH_GATEWAY_LICENSE_KEY

10. Verify that the management plane pods have a status of Running.

    kubectl get pods -n gloo-mesh --context $MGMT_CONTEXT

    Example output:

    NAME                                      READY   STATUS    RESTARTS   AGE
    gloo-mesh-mgmt-server-56c495796b-cx687    1/1     Running   0          30s
    gloo-mesh-redis-8455d49c86-f8qhw          1/1     Running   0          30s
    gloo-mesh-ui-65b6b6df5f-bf4vp             3/3     Running   0          30s
    gloo-telemetry-collector-agent-7rzfb      1/1     Running   0          30s
    gloo-telemetry-collector-agent-mf5rw      1/1     Running   0          30s
    gloo-telemetry-gateway-6547f479d5-r4zm6   1/1     Running   0          30s
    prometheus-server-57cd8c74d4-2bc7f        2/2     Running   0          30s
11. Save the external address and port that your cloud provider assigned to the gloo-mesh-mgmt-server service. The gloo-mesh-agent agent in each workload cluster accesses this address via a secure connection.

    export MGMT_SERVER_NETWORKING_DOMAIN=$(kubectl get svc -n gloo-mesh gloo-mesh-mgmt-server --context $MGMT_CONTEXT -o jsonpath="{.status.loadBalancer.ingress[0]['hostname','ip']}")
    export MGMT_SERVER_NETWORKING_PORT=$(kubectl get svc -n gloo-mesh gloo-mesh-mgmt-server --context $MGMT_CONTEXT -o jsonpath='{.spec.ports[?(@.name=="grpc")].port}')
    export MGMT_SERVER_NETWORKING_ADDRESS=${MGMT_SERVER_NETWORKING_DOMAIN}:${MGMT_SERVER_NETWORKING_PORT}
    echo $MGMT_SERVER_NETWORKING_ADDRESS

    1. Expose the management server by using an OpenShift route. Your route might look like the following.

       oc apply --context $MGMT_CONTEXT -n gloo-mesh -f- <<EOF
       kind: Route
       apiVersion: route.openshift.io/v1
       metadata:
          name: gloo-mesh-mgmt-server
          namespace: gloo-mesh
          annotations:
             # Needed for the different agents to connect to different replica instances of the management server deployment
             haproxy.router.openshift.io/balance: roundrobin
       spec:
          host: gloo-mesh-mgmt-server.<your_domain>.com
          to:
             kind: Service
             name: gloo-mesh-mgmt-server
             weight: 100
          port:
             targetPort: grpc
          tls:
             termination: passthrough
             insecureEdgeTerminationPolicy: Redirect
          wildcardPolicy: None
       EOF
    2. Save the management server’s address, which consists of the route host and the route port. Note that the port is the route’s own port, such as 443, and not the grpc port of the management server that the route points to.

       export MGMT_SERVER_NETWORKING_DOMAIN=$(oc get route -n gloo-mesh gloo-mesh-mgmt-server --context $MGMT_CONTEXT -o jsonpath='{.status.ingress[0].host}')
       export MGMT_SERVER_NETWORKING_ADDRESS=${MGMT_SERVER_NETWORKING_DOMAIN}:443
       echo $MGMT_SERVER_NETWORKING_ADDRESS
12. Save the external address and port that your cloud provider assigned to the Gloo OpenTelemetry (OTel) gateway service. The OTel collector agents in each workload cluster send metrics to this address.

    export TELEMETRY_GATEWAY_IP=$(kubectl get svc -n gloo-mesh gloo-telemetry-gateway --context $MGMT_CONTEXT -o jsonpath="{.status.loadBalancer.ingress[0]['hostname','ip']}")
    export TELEMETRY_GATEWAY_PORT=$(kubectl get svc -n gloo-mesh gloo-telemetry-gateway --context $MGMT_CONTEXT -o jsonpath='{.spec.ports[?(@.name=="otlp")].port}')
    export TELEMETRY_GATEWAY_ADDRESS=${TELEMETRY_GATEWAY_IP}:${TELEMETRY_GATEWAY_PORT}
    echo $TELEMETRY_GATEWAY_ADDRESS

    1. Expose the telemetry gateway by using an OpenShift route. Your route might look like the following.

       oc apply --context $MGMT_CONTEXT -n gloo-mesh -f- <<EOF
       kind: Route
       apiVersion: route.openshift.io/v1
       metadata:
         name: gloo-telemetry-gateway
         namespace: gloo-mesh
       spec:
         host: gloo-telemetry-gateway.<your_domain>.com
         to:
           kind: Service
           name: gloo-telemetry-gateway
           weight: 100
         port:
           targetPort: otlp
         tls:
           termination: passthrough
           insecureEdgeTerminationPolicy: Redirect
         wildcardPolicy: None
       EOF
    2. Save the telemetry gateway’s address, which consists of the route host and the route port. Note that the port is the route’s own port, such as 443, and not the otlp port of the telemetry gateway that the route points to.

       export TELEMETRY_GATEWAY_HOST=$(oc get route -n gloo-mesh gloo-telemetry-gateway --context $MGMT_CONTEXT -o jsonpath='{.status.ingress[0].host}')
       export TELEMETRY_GATEWAY_ADDRESS=${TELEMETRY_GATEWAY_HOST}:443
       echo $TELEMETRY_GATEWAY_ADDRESS

Data plane

Register each workload cluster with the Gloo management plane by deploying Gloo data plane components. A deployment named gloo-mesh-agent runs the Gloo agent in each workload cluster.

1. For the workload cluster that you want to register with Gloo, set the following environment variables. You update these variables each time you follow these steps to register another workload cluster.

   export REMOTE_CLUSTER=<workload_cluster_name>
   export REMOTE_CONTEXT=<workload_cluster_context>

2. In the management cluster, create a KubernetesCluster resource to represent the workload cluster and store relevant data, such as the workload cluster’s local domain.

   kubectl apply --context $MGMT_CONTEXT -f- <<EOF
   apiVersion: admin.gloo.solo.io/v2
   kind: KubernetesCluster
   metadata:
      name: ${REMOTE_CLUSTER}
      namespace: gloo-mesh
   spec:
      clusterDomain: cluster.local
   EOF

3. In your workload cluster, apply the Gloo CRDs.

helm upgrade -i gloo-platform-crds gloo-platform/gloo-platform-crds \
   --namespace=gloo-mesh \
   --create-namespace \
   --version=$GLOO_VERSION \
   --kube-context $REMOTE_CONTEXT \
   --set installIstioOperator=false