In this guide, you use the Gloo Operator to deploy an ambient mesh to each workload cluster, create an east-west gateway in each cluster, and link the istiod control planes across cluster networks by using peering gateways. Then, you can deploy the Bookinfo sample app to the ambient mesh in each cluster, and make select services available across the multicluster mesh. Incoming requests can then be routed from an ingress gateway, such as Gloo Gateway, to services in your mesh across all clusters.

Figure: Multicluster ambient mesh set up with the Solo distribution of Istio and Gloo Gateway.
Figure: Multicluster ambient mesh set up with the Solo distribution of Istio and Gloo Gateway.

For more information about ambient mesh or the ambient components that are installed in these steps, see About ambient mesh. For a more advanced multicluster ambient mesh installation, see the Helm installation guide.

Before you begin

  1. Create or use at least two existing Kubernetes clusters. The instructions in this guide assume two clusters. The cluster name must be alphanumeric with no special characters except a hyphen (-), lowercase, and begin with a letter (not a number).

  2. Save the names and kubeconfig contexts of each cluster.

    export REMOTE_CLUSTER1=<cluster1>
export REMOTE_CLUSTER2=<cluster2>
export REMOTE_CONTEXT1=<cluster1-context>
export REMOTE_CONTEXT2=<cluster2-context>

  3. Install the following command-line (CLI) tools.

    • helm, the Kubernetes package manager.
    • kubectl, the Kubernetes command line tool. Download the kubectl version that is within one minor version of the Kubernetes clusters you plan to use.

  4. Save your Enterprise-level license key for Gloo Mesh as an environment variable, which is required for multicluster mesh functionality. Contact your account representative to obtain a valid license.

    export GLOO_MESH_LICENSE_KEY=<enterprise_license_key>

Install a multicluster ambient mesh

Use the Gloo Operator to deploy and link service meshes in each workload cluster.

Set up the environment

  1. Save the Solo distribution of Istio patch version. In Gloo Mesh (OSS APIs) version 2.7 and later, multicluster setups require version 1.24.3 or later.

    export ISTIO_VERSION=1.27.0
export ISTIO_IMAGE=${ISTIO_VERSION}-solo

  2. Save the repo key for the minor version of the Solo distribution of Istio that you want to install. This is the 12-character hash at the end of the repo URL us-docker.pkg.dev/gloo-mesh/istio-<repo-key>, which you can find in the Istio images built by Solo.io support article.

    # 12-character hash at the end of the repo URL
export REPO_KEY=<repo_key>

  3. Get the Solo distribution of Istio binary and install istioctl, which you use for multicluster linking and gateway commands.

    1. Get the OS and architecture that you use on your machine.
      OS=$(uname | tr '[:upper:]' '[:lower:]' | sed -E 's/darwin/osx/')
ARCH=$(uname -m | sed -E 's/aarch/arm/; s/x86_64/amd64/; s/armv7l/armv7/')
echo $OS
echo $ARCH
    2. Download the Solo distribution of Istio binary and install istioctl.
      mkdir -p ~/.istioctl/bin
curl -sSL https://storage.googleapis.com/istio-binaries-$REPO_KEY/$ISTIO_IMAGE/istioctl-$ISTIO_IMAGE-$OS-$ARCH.tar.gz | tar xzf - -C ~/.istioctl/bin
chmod +x ~/.istioctl/bin/istioctl

export PATH=${HOME}/.istioctl/bin:${PATH}
    3. Verify that the istioctl client runs the Solo distribution of Istio that you want to install.
      istioctl version --remote=false
      Example output:
      client version: 1.27.0-solo

  4. Create a shared root of trust for the workload clusters. These example commands use the Istio CA to generate a self-signed root certificate and key, and use them to sign the workload certificates. For more information, see the Plug in CA Certificates guide in the community Istio documentation.

    curl -L https://istio.io/downloadIstio | ISTIO_VERSION=${ISTIO_VERSION} sh -
cd istio-${ISTIO_VERSION}
mkdir -p certs
pushd certs
make -f ../tools/certs/Makefile.selfsigned.mk root-ca

function create_cacerts_secret() {
  context=${1:?context}
  cluster=${2:?cluster}
  make -f ../tools/certs/Makefile.selfsigned.mk ${cluster}-cacerts
  kubectl --context=${context} create ns istio-system || true
  kubectl --context=${context} create secret generic cacerts -n istio-system \
    --from-file=${cluster}/ca-cert.pem \
    --from-file=${cluster}/ca-key.pem \
    --from-file=${cluster}/root-cert.pem \
    --from-file=${cluster}/cert-chain.pem
}

create_cacerts_secret ${REMOTE_CONTEXT1} ${REMOTE_CLUSTER1}
create_cacerts_secret ${REMOTE_CONTEXT2} ${REMOTE_CLUSTER2}

  5. Apply the CRDs for the Kubernetes Gateway API to each cluster, which are required to create components such as waypoint proxies for L7 traffic policies, gateways with the Gateway resource, and more.

    for context in ${REMOTE_CONTEXT1} ${REMOTE_CONTEXT2}; do
  kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.2.1/standard-install.yaml --context ${REMOTE_CONTEXT1}
done

Deploy ambient meshes

  1. Install the Gloo Operator to the gloo-mesh namespace of each cluster. This operator deploys and manages your Istio installations.

    for context in ${REMOTE_CONTEXT1} ${REMOTE_CONTEXT2}; do
  helm install gloo-operator oci://us-docker.pkg.dev/solo-public/gloo-operator-helm/gloo-operator \
  --version 0.3.1 \
  -n gloo-mesh \
  --create-namespace \
  --kube-context ${context} \
  --set manager.env.SOLO_ISTIO_LICENSE_KEY=${GLOO_MESH_LICENSE_KEY}
done

  2. Verify that the operator pods are running.

    for context in ${REMOTE_CONTEXT1} ${REMOTE_CONTEXT2}; do
  kubectl get pods -n gloo-mesh --context ${context} -l app.kubernetes.io/name=gloo-operator
done

  3. Apply the following ServiceMeshController resource for the Gloo Operator to create an Istio installation.

    function apply_smc() {
context=${1:?context}
cluster=${2:?cluster}

kubectl apply -n gloo-mesh --context ${context} -f - <<EOF
apiVersion: operator.gloo.solo.io/v1
kind: ServiceMeshController
metadata:
  name: managed-istio
  labels:
    app.kubernetes.io/name: managed-istio
spec:
  cluster: ${cluster}
  network: ${cluster}
  dataplaneMode: Ambient # required for multicluster setups
  installNamespace: istio-system
  version: ${ISTIO_VERSION}
EOF
}

apply_smc ${REMOTE_CONTEXT1} ${REMOTE_CLUSTER1}
apply_smc ${REMOTE_CONTEXT2} ${REMOTE_CLUSTER2}

  4. Verify that the components of the Istio control and data plane are successfully installed. Because the ztunnel and the CNI are deployed as daemon sets, the number of ztunnel pods and CNI pods each equal the number of nodes in your cluster. Note that it might take a few seconds for the pods to become available.

    for context in ${REMOTE_CONTEXT1} ${REMOTE_CONTEXT2}; do
  kubectl get pods -n istio-system --context ${context}
done

    Example output for one cluster:

    NAME                          READY   STATUS    RESTARTS   AGE
istio-cni-node-6s5nk          1/1     Running   0          2m53s
istio-cni-node-blpz4          1/1     Running   0          2m53s
istiod-gloo-bb86b959f-msrg7   1/1     Running   0          2m45s
istiod-gloo-bb86b959f-w29cm   1/1     Running   0          3m
ztunnel-mx8nw                 1/1     Running   0          2m52s
ztunnel-w8r6c                 1/1     Running   0          2m52s

  1. Create an east-west gateway in the istio-eastwest namespace of each cluster to facilitate traffic between services in each cluster in your multicluster mesh.

    for context in ${REMOTE_CONTEXT1} ${REMOTE_CONTEXT2}; do
  kubectl create namespace istio-eastwest --context ${context}
  istioctl multicluster expose --namespace istio-eastwest --context ${context}
done

  2. Link clusters to enable cross-cluster service discovery and allow traffic to be routed through east-west gateways across clusters. In each cluster, Gateway resources are created that use the istio-remote GatewayClass, which allows the gateways to connect to other clusters by using the clusters’ contexts.

    istioctl multicluster link --namespace istio-eastwest --contexts=${REMOTE_CONTEXT1},${REMOTE_CONTEXT2}

  3. Verify that east-west and remote peering gateways are successfully created in each cluster.

    for context in ${REMOTE_CONTEXT1} ${REMOTE_CONTEXT2}; do
  kubectl get gateways -n istio-eastwest --context ${context}
done

    Example output:

    NAME                         CLASS            ADDRESS                                                                  PROGRAMMED   AGE
istio-eastwest               istio-eastwest   ae8503e41fd9340149d6136c8072d453-777752786.us-east-2.elb.amazonaws.com   True         52s
istio-remote-peer-cluster2   istio-remote     aa9d7cff6521f4bcda12c54c71de5df2-662078021.us-east-1.elb.amazonaws.com   True         63s
NAME                         CLASS            ADDRESS                                                                  PROGRAMMED   AGE
istio-eastwest               istio-eastwest   aa9d7cff6521f4bcda12c54c71de5df2-662078021.us-east-1.elb.amazonaws.com   True         54s
istio-remote-peer-cluster1   istio-remote     ae8503e41fd9340149d6136c8072d453-777752786.us-east-2.elb.amazonaws.com   True         64s

Deploy a sample app

For testing purposes, you can deploy the Bookinfo sample app across multiple clusters, add the app services to your ambient mesh, and make the services available across clusters in the mesh.

  1. Create the bookinfo namespace in each cluster, and label the namespaces so that the services become part of the ambient service mesh.

    for context in ${REMOTE_CONTEXT1} ${REMOTE_CONTEXT2}; do
  kubectl --context ${context} create ns bookinfo
  kubectl --context ${context} label namespace bookinfo istio.io/dataplane-mode=ambient
done

  2. Deploy the Bookinfo app to each cluster.

    for context in ${REMOTE_CONTEXT1} ${REMOTE_CONTEXT2}; do
  # deploy bookinfo application components for all versions
  kubectl --context ${context} -n bookinfo apply -f https://raw.githubusercontent.com/istio/istio/1.27.0/samples/bookinfo/platform/kube/bookinfo.yaml -l 'app'
  # deploy an updated product page with extra container utilities such as 'curl' and 'netcat'
  kubectl --context ${context} -n bookinfo apply -f https://raw.githubusercontent.com/solo-io/gloo-mesh-use-cases/main/policy-demo/productpage-with-curl.yaml
  # deploy all bookinfo service accounts
  kubectl --context ${context} -n bookinfo apply -f https://raw.githubusercontent.com/istio/istio/1.27.0/samples/bookinfo/platform/kube/bookinfo.yaml -l 'account'
  # deploy individual services for each microservice version
  kubectl --context ${context} -n bookinfo apply -f https://raw.githubusercontent.com/istio/istio/1.27.0/samples/bookinfo/platform/kube/bookinfo-versions.yaml
done

  3. Verify that the Bookinfo app is deployed successfully.

    kubectl --context ${REMOTE_CONTEXT1} get pods,svc -n bookinfo
kubectl --context ${REMOTE_CONTEXT2} get pods,svc -n bookinfo

  4. Label the productpage service in each cluster to create one productpage global service entry. This global service unifies both service endpoints behind one global service hostname, which increases the availability of the Bookinfo app and makes both endpoints available across the multicluster mesh. For more information, see Namespace sameness.

    kubectl --context ${REMOTE_CONTEXT1} label service productpage -n bookinfo solo.io/service-scope=global
kubectl --context ${REMOTE_CONTEXT2} label service productpage -n bookinfo solo.io/service-scope=global

  5. Apply the networking.istio.io/traffic-distribution=Any annotation to the services. This annotation allows requests to the productpage global service to be routed to each service endpoint equally.

    kubectl --context ${REMOTE_CONTEXT1} annotate service productpage -n bookinfo networking.istio.io/traffic-distribution=Any
kubectl --context ${REMOTE_CONTEXT2} annotate service productpage -n bookinfo networking.istio.io/traffic-distribution=Any

  6. Verify that the global service entry with the productpage.bookinfo.mesh.internal hostname is created.

    kubectl get serviceentry -n istio-system --context ${REMOTE_CONTEXT1}
kubectl get serviceentry -n istio-system --context ${REMOTE_CONTEXT2}

    Example output:

    NAMESPACE      NAME                           HOSTS                                    LOCATION   RESOLUTION   AGE
istio-system   autogen.bookinfo.productpage   ["productpage.bookinfo.mesh.internal"]              STATIC       94s

  7. Use the ratings app to send a request to the productpage.bookinfo.mesh.internal global hostname. Verify that you get back a 200 HTTP response code.

    kubectl -n bookinfo --context $REMOTE_CONTEXT1 debug -i pods/$(kubectl get pod -l app=ratings \
--context $REMOTE_CONTEXT1 -A -o jsonpath='{.items[0].metadata.name}') \
--image=curlimages/curl -- curl -vik http://productpage.bookinfo.mesh.internal:9080/productpage

The productpage services for each Bookinfo instance are now unified behind one hostname, which increases the availability of the Bookinfo app.

Optional: Expose apps with an ingress gateway

You can now use the productpage global service hostname in routing configurations. For example, to expose the productpage global service hostname with an ingress gateway, check out the guide to use Gloo Gateway as an ingress gateway to your ambient mesh. To review all options, see the ingress gateway guide for an ambient mesh.

Optional: Analyze your mesh with Gloo UI observability

You can now install the Gloo UI to evaluate the health and efficiency of your service mesh. The Gloo UI provides analysis and insights for your service mesh, such as recommendations to harden your Istio environment and steps to implement them in your environment. Additionally, you can use observability features such as the Gloo UI Graph to visualize the network traffic flows to and within your multicluster service mesh.

To get started, deploy the Gloo management plane in your cluster.

Next steps

Now that you have an ambient mesh up and running, check out some of the following resources to learn more and expand your service mesh capabilities.

Istio:

Gloo Mesh (OSS APIs):

  • Learn about how you can enhance your Istio setup with Gloo Mesh.
  • Install the Gloo UI to evaluate the health and efficiency of your multicluster service mesh.

Help and support:

Cleanup

If you no longer need this quick-start ambient mesh environment, you can follow the steps in the uninstall guide.