Set up Gloo Mesh
Quickly set up Gloo Mesh Enterprise in three clusters.
Gloo Mesh Enterprise is a multicluster and multimesh management plane that is based on hardened, open-source projects like Envoy and Istio. With Gloo Mesh, you can unify the configuration, operation, and visibility of service-to-service connectivity across your distributed applications. These apps can run in different virtual machines (VMs) or Kubernetes clusters on premises or in various cloud providers, and even in different service meshes.
You can follow this guide to quickly get started with Gloo Mesh Enterprise. To learn more about the benefits and architecture, see About. To customize your installation with Helm instead, see the advanced installation guide.
The following figure depicts the multi-mesh architecture created by this quick-start guide.
Before you begin
Install the following command-line (CLI) tools.
kubectl
, the Kubernetes command line tool. Download thekubectl
version that is within one minor version of the Kubernetes clusters you plan to use.meshctl
, the Solo command line tool.curl -sL https://run.solo.io/meshctl/install | GLOO_MESH_VERSION=v2.5.11 sh - export PATH=$HOME/.gloo-mesh/bin:$PATH
Create or use at least two existing Kubernetes clusters. The instructions in this guide assume one management cluster and two workload clusters.
- The cluster name must be alphanumeric with no special characters except a hyphen (-), lowercase, and begin with a letter (not a number).
Set the names of your clusters from your infrastructure provider. If your clusters have different names, specify those names instead.
export MGMT_CLUSTER=mgmt export REMOTE_CLUSTER1=cluster1 export REMOTE_CLUSTER2=cluster2
- Save the kubeconfig contexts for your clusters. Run
kubectl config get-contexts
, look for your cluster in theCLUSTER
column, and get the context name in theNAME
column. Note: Do not use context names with underscores. The generated certificate that connects workload clusters to the management cluster uses the context name as a SAN specification, and underscores in SAN are not FQDN compliant. You can rename a context by runningkubectl config rename-context "<oldcontext>" <newcontext>
.export MGMT_CONTEXT=<management-cluster-context> export REMOTE_CONTEXT1=<remote-cluster1-context> export REMOTE_CONTEXT2=<remote-cluster2-context>
Set your Gloo Mesh Enterprise license key as an environment variable. If you do not have one, contact an account representative. If you prefer to specify license keys in a secret instead, see Licensing. To check your license’s validity, you can run
meshctl license check --key $(echo ${GLOO_MESH_LICENSE_KEY} | base64 -w0)
.export GLOO_MESH_LICENSE_KEY=<license_key>
If you plan to deploy an ingress gateway to manage ingress traffic to mesh workloads and you want to apply policies, such as rate limits, external authentication, or Web Application Firewalls to that gateway, you must also have a Gloo Mesh Gateway license. Without a Gloo Mesh Gateway license, you can only set up simple routing rules to match and forward traffic to mesh workloads. Save the Gloo Mesh Gateway license key as an additional environment variable.
export GLOO_MESH_GATEWAY_LICENSE_KEY=<license-key>
Install the Gloo management plane
Deploy the Gloo management plane into a dedicated management cluster.
Install the Gloo management plane in your management cluster. This command uses a basic profile to create a
gloo-mesh
namespace and install the management plane components, such as the management server and Prometheus server, in your management cluster.To install Gloo Mesh Enterprise with an additional Gloo Mesh Gateway license so that you can apply Gloo policies to the ingress gateway, add the--set licensing.glooGatewayLicenseKey=$GLOO_MESH_GATEWAY_LICENSE_KEY
option to themeshctl install
command.Verify that the management plane pods have a status of
Running
.kubectl get pods -n gloo-mesh --context $MGMT_CONTEXT
Example output:
NAME READY STATUS RESTARTS AGE gloo-mesh-mgmt-server-56c495796b-cx687 1/1 Running 0 30s gloo-mesh-redis-8455d49c86-f8qhw 1/1 Running 0 30s gloo-mesh-ui-65b6b6df5f-bf4vp 3/3 Running 0 30s gloo-telemetry-collector-agent-7rzfb 1/1 Running 0 30s gloo-telemetry-gateway-6547f479d5-r4zm6 1/1 Running 0 30s prometheus-server-57cd8c74d4-2bc7f 2/2 Running 0 30s
Save the external address and port that your cloud provider assigned to the Gloo OpenTelemetry (OTel) gateway service. The OTel collector agents in each workload cluster send metrics to this address.
Create a workspace that selects all clusters and namespaces by default, and workspace settings that enable communication across clusters. Gloo workspaces let you organize team resources across Kubernetes namespaces and clusters. In this example, you create a global workspace that imports and exports all resources and namespaces, and a workspace settings resource in the
gloo-mesh-config
namespace. Later, as your teams grow, you can create a workspace for each team, to enforce service isolation, set up federation, and even share resources by importing and exporting.kubectl apply --context $MGMT_CONTEXT -f- <<EOF apiVersion: admin.gloo.solo.io/v2 kind: Workspace metadata: name: $MGMT_CLUSTER namespace: gloo-mesh spec: workloadClusters: - name: '*' namespaces: - name: '*' --- apiVersion: v1 kind: Namespace metadata: name: gloo-mesh-config --- apiVersion: admin.gloo.solo.io/v2 kind: WorkspaceSettings metadata: name: $MGMT_CLUSTER namespace: gloo-mesh-config spec: options: serviceIsolation: enabled: false federation: enabled: false serviceSelector: - {} eastWestGateways: - selector: labels: istio: eastwestgateway EOF
Install the Gloo data plane
Register each workload cluster with the Gloo management plane by deploying Gloo data plane components. A deployment named gloo-mesh-agent
runs the Gloo agent in each workload cluster.
Register both workload clusters with the management server. These commands use basic profiles to install the Gloo agent, rate limit server, and external auth server in each workload cluster.
Verify that the Gloo data plane components in each workload cluster are healthy. If not, try debugging the agent.
meshctl check --kubecontext $REMOTE_CONTEXT1 meshctl check --kubecontext $REMOTE_CONTEXT2
Example output:
🟢 Gloo deployment status Namespace | Name | Ready | Status gloo-mesh | ext-auth-service | 1/1 | Healthy gloo-mesh | gloo-mesh-agent | 1/1 | Healthy gloo-mesh | gloo-telemetry-collector-agent | 3/3 | Healthy gloo-mesh | rate-limiter | 1/1 | Healthy
Verify that your Gloo Mesh Enterprise setup is correctly installed. If not, try debugging the relay connection. Note that this check might take a few seconds to verify that:
- Your Gloo product licenses are valid and current.
- The Gloo CRDs are installed at the correct version.
- The management plane pods in the management cluster are running and healthy.
- The agents in the workload clusters are successfully identified by the management server.
meshctl check --kubecontext $MGMT_CONTEXT
Example output:
🟢 License status INFO gloo-mesh enterprise license expiration is 25 Aug 24 10:38 CDT INFO No GraphQL license module found for any product 🟢 CRD version check 🟢 Gloo deployment status Namespace | Name | Ready | Status gloo-mesh | gloo-mesh-mgmt-server | 1/1 | Healthy gloo-mesh | gloo-mesh-redis | 1/1 | Healthy gloo-mesh | gloo-mesh-ui | 1/1 | Healthy gloo-mesh | gloo-telemetry-collector-agent | 3/3 | Healthy gloo-mesh | gloo-telemetry-gateway | 1/1 | Healthy gloo-mesh | prometheus-server | 1/1 | Healthy 🟢 Mgmt server connectivity to workload agents Cluster | Registered | Connected Pod cluster1 | true | gloo-mesh/gloo-mesh-mgmt-server-65bd557b95-v8qq6 cluster2 | true | gloo-mesh/gloo-mesh-mgmt-server-65bd557b95-v8qq6 Connected Pod | Clusters gloo-mesh/gloo-mesh-mgmt-server-65bd557b95-v8qq6 | 2
Deploy managed Istio
Use Gloo Mesh Enterprise to quickly install and manage service meshes in each workload cluster for you.
Create the
istiod
control planes in your workload clusters.To allow multicluster routing, create a
GatewayLifecycleManager
resource to deploy and manage an east-west gateway.kubectl apply --context $MGMT_CONTEXT -f- <<EOF apiVersion: admin.gloo.solo.io/v2 kind: GatewayLifecycleManager metadata: name: istio-eastwestgateway namespace: gloo-mesh spec: installations: - clusters: - activeGateway: true name: $REMOTE_CLUSTER1 - activeGateway: true name: $REMOTE_CLUSTER2 gatewayRevision: auto istioOperatorSpec: components: ingressGateways: - enabled: true k8s: service: ports: - port: 15021 targetPort: 15021 name: status-port - port: 15443 targetPort: 15443 name: tls selector: istio: eastwestgateway type: LoadBalancer label: istio: eastwestgateway app: istio-eastwestgateway name: istio-eastwestgateway namespace: gloo-mesh-gateways namespace: istio-system profile: empty EOF
Optional: If you want to allow traffic from outside the cluster to enter your mesh, create another
GatewayLifecycleManager
resource to deploy and manage an ingress gateway. The ingress gateway allows you to specify basic routing rules for how to match and forward incoming traffic to a workload in the mesh. However, to also apply policies, such as rate limits, external authentication, or a Web Application Firewall to the gateway, you must have a Gloo Mesh Gateway license. For more information about Gloo Mesh Gateway, see the docs. If you want a service mesh-only environment without ingress, you can skip this step.Note that you can optionally add cloud provider-specific annotations for the ingress gateway load balancer; for example, you can uncomment the AWSserviceAnnotations
in theistio-ingressgateway
gateway lifecycle manager.kubectl apply --context $MGMT_CONTEXT -f- <<EOF apiVersion: admin.gloo.solo.io/v2 kind: GatewayLifecycleManager metadata: name: istio-ingressgateway namespace: gloo-mesh spec: installations: - clusters: - activeGateway: true name: $REMOTE_CLUSTER1 - activeGateway: true name: $REMOTE_CLUSTER2 gatewayRevision: auto istioOperatorSpec: components: ingressGateways: - enabled: true k8s: service: ports: - name: status-port port: 15021 targetPort: 15021 - name: http2 port: 80 targetPort: 8080 - name: https port: 443 targetPort: 8443 - name: tls port: 15443 targetPort: 15443 selector: istio: ingressgateway type: LoadBalancer #serviceAnnotations: # service.beta.kubernetes.io/aws-load-balancer-backend-protocol: ssl # service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: "true" # service.beta.kubernetes.io/aws-load-balancer-nlb-target-type: instance # service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing # service.beta.kubernetes.io/aws-load-balancer-ssl-cert: "arn:aws:acm:<cert>" # service.beta.kubernetes.io/aws-load-balancer-type: external label: istio: ingressgateway app: istio-ingressgateway name: istio-ingressgateway namespace: gloo-mesh-gateways namespace: istio-system profile: empty EOF
Verify that the namespaces for your Istio installations are created in each workload cluster.
kubectl get ns --context $REMOTE_CONTEXT1 kubectl get ns --context $REMOTE_CONTEXT2
For example, the
gm-iop-1-20
,gloo-mesh-gateways
, andistio-system
namespaces are created:NAME STATUS AGE default Active 56m gloo-mesh Active 36m gm-iop-1-20 Active 91s gloo-mesh-gateways Active 90s istio-system Active 91s ...
Verify that Gloo Mesh successfully discovered the Istio service meshes. Gloo creates internal
mesh
resources to represent the state of the Istio service mesh.kubectl get mesh -n gloo-mesh --context $REMOTE_CONTEXT1 kubectl get mesh -n gloo-mesh --context $REMOTE_CONTEXT2
Next
Deploy sample apps to try out the routing capabilities and traffic policies in Gloo Mesh.
Understand what happened
Find out more information about the Gloo Mesh environment that you set up in this guide.
Gloo Mesh installation: This quick start guide used meshctl
to install a minimum deployment of Gloo Mesh Enterprise for testing purposes, and some optional components are not installed. For example, self-signed certificates are used to secure communication between the management and workload clusters. To learn more about production-level installation options, including advanced configuration options available in the Gloo Mesh Enterprise Helm chart, see the Setup guide.
Relay architecture: When you installed the Gloo Mesh management plane in the management cluster, a deployment named gloo-mesh-mgmt-server
was created to translate and implement your Gloo configurations and act as the relay server. When you registered the workload clusters to be managed by the management plane, a deployment named gloo-mesh-agent
was created on each workload cluster to run a relay agent. All communication is outbound from the relay agents on the workload clusters to the relay server on the management cluster. For more information about server-agent communication, see the relay architecture page. Additionally, default, self-signed certificates were used to secure communication between the control and data planes. For more information about the certificate architecture, see Default Gloo Mesh-managed certificates.
Workload cluster registration: Cluster registration creates a KubernetesCluster
custom resource on the management cluster to represent the workload cluster and store relevant data, such as the workload cluster’s local domain (“cluster.local”). To learn more about cluster registration and how to register clusters with Helm rather than meshctl
, review the cluster registration guide.
Istio installation: The Istio profiles in this getting started guide were provided with IstioLifecycleManager
and GatewayLifecycleManager
custom resources. However, Gloo Mesh can discover Istio service meshes regardless of their installation options. For more information about service mesh lifecycle management with Gloo, check out Service mesh lifecycle and Solo distributions of Istio.
Gloo workspace: Gloo workspaces let you organize team resources across Kubernetes namespaces and clusters. In this example, a single workspace is created for everything. Later, as your teams grow, you can create a workspace for each team, to enforce service isolation, set up federation, and even share resources by importing and exporting. You can also change the default workspace by following the Workspace setup guide.