Navigation :

Set up Gloo Mesh

Start with setting up Gloo Mesh Enterprise in three clusters.

The following figure depicts the multi-mesh architecture created by this quick-start guide.

Figure of a three-cluster Gloo Mesh quick-start architecture.

Before you begin

Install meshctl, the Gloo command line tool for bootstrapping Gloo Platform, registering clusters, describing configured resources, and more. Be sure to download version 2.4.0-beta1, which uses the latest Gloo Mesh installation values.
```
curl -sL https://run.solo.io/meshctl/install | GLOO_MESH_VERSION=v2.4.0-beta1 sh -
export PATH=$HOME/.gloo-mesh/bin:$PATH
```
Save the names of three clusters as environment variables. In this guide, the cluster names mgmt, cluster1, and cluster2 are used. The mgmt cluster serves as the management cluster, and cluster1 and cluster2 serve as the workload clusters in this setup. If your clusters have different names, specify those names instead. Note: The cluster name must be alphanumeric with no special characters except a hyphen (-), lowercase, and begin with a letter (not a number).
```
export MGMT_CLUSTER=mgmt
export REMOTE_CLUSTER1=cluster1
export REMOTE_CLUSTER2=cluster2
```
Save the kubeconfig contexts for your clusters. Run kubectl config get-contexts, look for your cluster in the CLUSTER column, and get the context name in the NAME column. Note: Do not use context names with underscores. The context name is used as a SAN specification in the generated certificate that connects workload clusters to the management cluster, and underscores in SAN are not FQDN compliant. You can rename a context by running kubectl config rename-context "<oldcontext>" <newcontext>.
```
export MGMT_CONTEXT=<management-cluster-context>
export REMOTE_CONTEXT1=<remote-cluster1-context>
export REMOTE_CONTEXT2=<remote-cluster2-context>
```

Install Gloo Mesh in the management cluster

Install Gloo Mesh in your management cluster. This command uses a basic profile to create a gloo-mesh namespace and install the control plane components, such as the management server and Prometheus server, in your management cluster. If you do not have a license key, contact an account representative.
- Kubernetes
- OpenShift
```
meshctl install --profiles mgmt-server \
  --kubecontext $MGMT_CONTEXT \
  --set common.cluster=$MGMT_CLUSTER \
  --set licensing.glooMeshLicenseKey=$GLOO_MESH_LICENSE_KEY
```
Note:
- Need to use OpenShift routes instead of load balancer service types? Follow the OpenShift steps in the full multicluster setup guide instead.
- After you run the following command in OpenShift 4.11 and later, you might see warnings for the pods and containers which violate the OpenShift PodSecurity "restricted:v1.24" profile, due to the elevated permissions required by Istio. You can ignore these warnings. For more info, see this article.
```
meshctl install --profiles mgmt-server \
  --kubecontext $MGMT_CONTEXT \
  --set common.cluster=$MGMT_CLUSTER \
  --set glooMgmtServer.floatingUserId=true \
  --set glooUi.floatingUserId=true \
  --set licensing.glooMeshLicenseKey=$GLOO_MESH_LICENSE_KEY \
  --set prometheus.server.securityContext=false \
  --set redis.deployment.floatingUserId=true
```

Verify that the control plane pods are running.

kubectl get pods -n gloo-mesh --context $MGMT_CONTEXT

Example output:

NAME                                      READY   STATUS    RESTARTS   AGE
gloo-mesh-mgmt-server-56c495796b-cx687    1/1     Running   0          30s
gloo-mesh-redis-8455d49c86-f8qhw          1/1     Running   0          30s
gloo-mesh-ui-65b6b6df5f-bf4vp             3/3     Running   0          30s
gloo-telemetry-gateway-6547f479d5-r4zm6   1/1     Running   0          30s
prometheus-server-57cd8c74d4-2bc7f        2/2     Running   0          30s

Save the external address and port that were assigned by your cloud provider to the Gloo OpenTelemetry (OTel) gateway load balancer service. The OTel collector agents in each workload cluster send metrics to this address.

LoadBalancer IP address
LoadBalancer hostname

export TELEMETRY_GATEWAY_IP=$(kubectl get svc -n gloo-mesh gloo-telemetry-gateway --context $MGMT_CONTEXT -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
export TELEMETRY_GATEWAY_PORT=$(kubectl -n gloo-mesh get service gloo-telemetry-gateway --context $MGMT_CONTEXT -o jsonpath='{.spec.ports[?(@.name=="otlp")].port}')
export TELEMETRY_GATEWAY_ADDRESS=${TELEMETRY_GATEWAY_IP}:${TELEMETRY_GATEWAY_PORT}
echo $TELEMETRY_GATEWAY_ADDRESS

export TELEMETRY_GATEWAY_HOSTNAME=$(kubectl get svc -n gloo-mesh gloo-telemetry-gateway --context $MGMT_CONTEXT -o jsonpath='{.status.loadBalancer.ingress[0].hostname}')
export TELEMETRY_GATEWAY_PORT=$(kubectl -n gloo-mesh get service gloo-telemetry-gateway --context $MGMT_CONTEXT -o jsonpath='{.spec.ports[?(@.name=="otlp")].port}')
export TELEMETRY_GATEWAY_ADDRESS=${TELEMETRY_GATEWAY_HOSTNAME}:${TELEMETRY_GATEWAY_PORT}
echo $TELEMETRY_GATEWAY_ADDRESS

Create a workspace that selects all clusters and namespaces by default. Gloo workspaces let you organize team resources across Kubernetes namespaces and clusters. In this example, you create a single workspace for everything. Later, as your teams grow, you can create a workspace for each team, to enforce service isolation, set up federation, and even share resources by importing and exporting.
```
kubectl apply --context $MGMT_CONTEXT -f- <<EOF
apiVersion: admin.gloo.solo.io/v2
kind: Workspace
metadata:
  name: $MGMT_CLUSTER
  namespace: gloo-mesh
spec:
  workloadClusters:
    - name: '*'
      namespaces:
        - name: '*'
EOF
```

Create a workspace settings resource for the workspace that enables federation across clusters and selects the Istio east-west gateway.

kubectl apply --context $MGMT_CONTEXT -f- <<EOF
apiVersion: admin.gloo.solo.io/v2
kind: WorkspaceSettings
metadata:
  name: $MGMT_CLUSTER
  namespace: gloo-mesh
spec:
  options:
    serviceIsolation:
      enabled: false
    federation:
      enabled: false
      serviceSelector:
      - {}
    eastWestGateways:
    - selector:
        labels:
          istio: eastwestgateway
EOF

Register workload clusters

Prepare the gloo-mesh-addons namespace.

Kubernetes
OpenShift

kubectl create ns gloo-mesh-addons --context $REMOTE_CONTEXT1
kubectl create ns gloo-mesh-addons --context $REMOTE_CONTEXT2

Elevate the permissions of the gloo-mesh-addons service account that will be created.

oc adm policy add-scc-to-group anyuid system:serviceaccounts:gloo-mesh-addons --context $REMOTE_CONTEXT1
oc adm policy add-scc-to-group anyuid system:serviceaccounts:gloo-mesh-addons --context $REMOTE_CONTEXT2

Create the gloo-mesh-addons project, and create a NetworkAttachmentDefinition custom resource for the project.

kubectl create ns gloo-mesh-addons --context $REMOTE_CONTEXT1

cat <<EOF | oc -n gloo-mesh-addons create --context $REMOTE_CONTEXT1 -f -
apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: istio-cni
EOF

kubectl create ns gloo-mesh-addons --context $REMOTE_CONTEXT2

cat <<EOF | oc -n gloo-mesh-addons create --context $REMOTE_CONTEXT2 -f -
apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: istio-cni
EOF

Register both workload clusters with the management server. These commands install the Gloo agent, rate limit server, and external auth server in each workload cluster.

Kubernetes
OpenShift

meshctl cluster register $REMOTE_CLUSTER1 \
  --kubecontext $MGMT_CONTEXT \
  --remote-context $REMOTE_CONTEXT1 \
  --profiles agent,ratelimit,extauth \
  --telemetry-server-address $TELEMETRY_GATEWAY_ADDRESS

meshctl cluster register $REMOTE_CLUSTER2 \
  --kubecontext $MGMT_CONTEXT \
  --remote-context $REMOTE_CONTEXT2 \
  --profiles agent,ratelimit,extauth \
  --telemetry-server-address $TELEMETRY_GATEWAY_ADDRESS

meshctl cluster register $REMOTE_CLUSTER1 \
  --kubecontext $MGMT_CONTEXT \
  --remote-context $REMOTE_CONTEXT1 \
  --profiles agent,ratelimit,extauth \
  --set glooAgent.floatingUserId=true \
  --version $GLOO_VERSION \
  --telemetry-server-address $TELEMETRY_GATEWAY_ADDRESS

meshctl cluster register $REMOTE_CLUSTER2 \
  --kubecontext $MGMT_CONTEXT \
  --remote-context $REMOTE_CONTEXT2 \
  --profiles agent,ratelimit,extauth \
  --set glooAgent.floatingUserId=true \
  --version $GLOO_VERSION \
  --telemetry-server-address $TELEMETRY_GATEWAY_ADDRESS

Note: In OpenShift 4.11 and later, you might see warnings for the pods and containers which violate the OpenShift PodSecurity "restricted:v1.24" profile, due to the elevated permissions required by Istio. You can ignore these warnings. For more info, see this article.

Verify that the Gloo data plane components are healthy.

meshctl check --kubecontext $REMOTE_CONTEXT1
meshctl check --kubecontext $REMOTE_CONTEXT2

Example output:

🟢 CRD Version check


🟢 Gloo Platform Deployment Status

Namespace        | Name             | Ready | Status 
gloo-mesh        | gloo-mesh-agent  | 1/1   | Healthy
gloo-mesh-addons | ext-auth-service | 1/1   | Healthy
gloo-mesh-addons | rate-limiter     | 1/1   | Healthy
gloo-mesh-addons | redis            | 1/1   | Healthy

Verify that your Gloo Mesh setup is correctly installed. This check might take a few seconds to verify that:

Your Gloo Platform product licenses are valid and current.
The Gloo Platform CRDs are installed at the correct version.
The control plane pods in the management cluster are running and healthy.
The agents in the workload clusters are successfully identified by the control plane.

meshctl check --kubecontext $MGMT_CONTEXT

Example output:

🟢 License status

 INFO  gloo-mesh enterprise license expiration is 25 Aug 23 10:38 CDT
 INFO  No GraphQL license module found for any product

🟢 CRD version check

🟢 Gloo Platform deployment status

Namespace | Name                           | Ready | Status 
gloo-mesh | gloo-mesh-mgmt-server          | 1/1   | Healthy
gloo-mesh | gloo-mesh-redis                | 1/1   | Healthy
gloo-mesh | gloo-mesh-ui                   | 1/1   | Healthy
gloo-mesh | gloo-telemetry-gateway         | 1/1   | Healthy
gloo-mesh | prometheus-server              | 1/1   | Healthy
gloo-mesh | gloo-telemetry-collector-agent | 3/3   | Healthy

🟢 Mgmt server connectivity to workload agents

Cluster  | Registered | Connected Pod                                   
cluster1 | true       | gloo-mesh/gloo-mesh-mgmt-server-65bd557b95-v8qq6
cluster2 | true       | gloo-mesh/gloo-mesh-mgmt-server-65bd557b95-v8qq6

In each workload cluster, follow the steps in Install Istio by using the Istio Lifecycle Manager to deploy managed Istio service meshes.

Deploy sample apps to try out the routing capabilities and traffic policies in Gloo Mesh.

Understand what happened

Find out more information about the Gloo Mesh environment that you set up in this guide.

Gloo Mesh installation: This quick start guide used meshctl to install a minimum deployment of Gloo Mesh Enterprise for testing purposes, and some optional components are not installed. For example, self-signed certificates are used to secure communication between the management and workload clusters. To learn more about production-level installation options, including advanced configuration options available in the Gloo Mesh Enterprise Helm chart, see the Setup guide.

Relay architecture: When you installed the Gloo Mesh control plane in the management cluster, a deployment named gloo-mesh-mgmt-server was created to translate and implement your Gloo configurations and act as the relay server. When you registered the workload clusters to be managed by the control plane, a deployment named gloo-mesh-agent was created on each workload cluster to run a relay agent. All communication is outbound from the relay agents on the workload clusters to the relay server on the management cluster. For more information about server-agent communication, see the relay architecture page. Additionally, default, self-signed certificates were used to secure communication between the control and data planes. For more information about the certificate architecture, see Default Gloo Mesh-managed certificates.

Workload cluster registration: Cluster registration creates a KubernetesCluster custom resource on the management cluster to represent the workload cluster and store relevant data, such as the workload cluster's local domain (“cluster.local”). To learn more about cluster registration and how to register clusters with Helm rather than meshctl, review the cluster registration guide.

Istio installation: The Istio profiles in this getting started guide were provided within the Gloo Mesh installation Helm chart. However, Gloo Mesh can discover Istio service meshes regardless of their installation options. To manually install Istio, see the advanced configuration guides. Additionally, note that although ingress gateways were deployed to workload clusters, you must have a Gloo Gateway license to route north-south (ingress) traffic to workloads in your Gloo environment.

Gloo workspace: Gloo workspaces let you organize team resources across Kubernetes namespaces and clusters. In this example, a single workspace is created for everything. Later, as your teams grow, you can create a workspace for each team, to enforce service isolation, set up federation, and even share resources by importing and exporting. You can also change the default workspace by following the Workspace setup guide.

Set up Gloo Mesh

Before you begin

Install Gloo Mesh in the management cluster

Register workload clusters

Next

Understand what happened