Skip to content
home
2.11.x
Istio sidecar mesh
Install
Single cluster
You are viewing the documentation for Solo Enterprise for Istio, formerly known as Gloo Mesh (OSS APIs).

Migrate to the Gloo Operator

Page as Markdown
Switch from your existing sidecar installations to Gloo-managed service meshes.
Overview

If you have existing Istio installations and want to switch to using the Gloo Operator for service mesh management, you can use one of the following guides:
  • Revisioned Helm: You installed Istio with Helm. To add namespaces to the service mesh, you used revision labels such as istio.io/rev=1-28.
  • Revisionless Helm: You installed Istio with Helm. To add namespaces to the service mesh, you used the sidecar injection label, istio-injection=enabled.
  • Istio lifecycle manager: You installed Istio and gateways by using Solo’s deprecated Istio lifecycle manager, such as by using the default settings in the getting started guides, the istioInstallations Helm settings in your Gloo Helm chart, or by directly creating IstioLifecycleManager and GatewayLifecycleManager custom resources.
Migrate from revisioned Helm installations

If you currently install Istio by using Helm and use revisions to manage your installations, you can migrate from your community Istio revision, such as 1-28, to the gloo revision. The Gloo Operator uses the gloo revision by default to manage Istio installations in your cluster.
  1. Save your Istio installation values in environment variables.
    1. If you do not already have a license, decide the level of licensed features that you want, and contact an account representative to obtain the license.
    2. Choose the version of Istio that you want to install or upgrade to by reviewing the supported versions table.
    3. Save each value in an environment variable. If you prefer to specify license keys in a secret instead, see Licensing. Note that the Gloo Operator installs the Solo distribution of Istio by default for the version you specify, so neither the -solo image tag nor the repo URL are required.
      export SOLO_ISTIO_LICENSE_KEY=<license_key>
   export ISTIO_VERSION=1.28.5
    4. Install or upgrade istioctl with the same version of Istio that you saved.
      curl -L https://istio.io/downloadIstio | ISTIO_VERSION=${ISTIO_VERSION} sh -
   cd istio-${ISTIO_VERSION}
   export PATH=$PWD/bin:$PATH
  2. Install the Gloo Operator and deploy a managed istiod control plane.
    1. Install the Gloo Operator to the gloo-mesh namespace. This operator deploys and manages your Istio installation. For more information, see the Helm reference.Note that if you already installed Solo Enterprise for Istio, you can optionally reference the secret that Solo Enterprise for Istio automatically creates for your license in the –set manager.env.SOLO_ISTIO_LICENSE_KEY_SECRET_REF=gloo-mesh/license-keys flag instead.
      helm install gloo-operator oci://us-docker.pkg.dev/solo-public/gloo-operator-helm/gloo-operator \
  --version 0.5.2 \
  -n gloo-mesh \
  --create-namespace \
  --set manager.env.SOLO_ISTIO_LICENSE_KEY=${SOLO_ISTIO_LICENSE_KEY}
    2. Verify that the operator pod is running.
      kubectl get pods -n gloo-mesh -l app.kubernetes.io/name=gloo-operator
      Example output:
      gloo-operator-78d58d5c7b-lzbr5     1/1     Running   0          48s
    3. Create a ServiceMeshController custom resource to configure an Istio installation. For more information about the configurable fields, see the installation guide.
      kubectl apply -n gloo-mesh -f -<<EOF
apiVersion: operator.gloo.solo.io/v1
kind: ServiceMeshController
metadata:
  name: managed-istio
  labels:
    app.kubernetes.io/name: managed-istio
spec:
  dataplaneMode: Sidecar
  version: ${ISTIO_VERSION}
  # Uncomment if you installed the istio-cni
  # onConflict: Force
EOF
    4. Verify that the ServiceMeshController is ready. In the Status section of the output, make sure that all statuses are True, and that the phase is SUCCEEDED.
      kubectl describe servicemeshcontroller -n gloo-mesh managed-istio
      Example output:
      ...
Status:
  Conditions:
    Last Transition Time:  2024-12-27T20:47:01Z
    Message:               Manifests initialized
    Observed Generation:   1
    Reason:                ManifestsInitialized
    Status:                True
    Type:                  Initialized
    Last Transition Time:  2024-12-27T20:47:02Z
    Message:               CRDs installed
    Observed Generation:   1
    Reason:                CRDInstalled
    Status:                True
    Type:                  CRDInstalled
    Last Transition Time:  2024-12-27T20:47:02Z
    Message:               Deployment succeeded
    Observed Generation:   1
    Reason:                DeploymentSucceeded
    Status:                True
    Type:                  ControlPlaneDeployed
    Last Transition Time:  2024-12-27T20:47:02Z
    Message:               Deployment succeeded
    Observed Generation:   1
    Reason:                DeploymentSucceeded
    Status:                True
    Type:                  CNIDeployed
    Last Transition Time:  2024-12-27T20:47:02Z
    Message:               Deployment succeeded
    Observed Generation:   1
    Reason:                DeploymentSucceeded
    Status:                True
    Type:                  WebhookDeployed
    Last Transition Time:  2024-12-27T20:47:02Z
    Message:               All conditions are met
    Observed Generation:   1
    Reason:                SystemReady
    Status:                True
    Type:                  Ready
  Phase:                   SUCCEEDED
Events:                    <none>
  3. Migrate your Istio-managed workloads to the managed gloo control plane.
    1. Get the workload namespaces that you previously labeled with an Istio revision, such as 1-28 in the following example.
      kubectl get namespaces -l istio.io/rev=1-28
    2. Overwrite the revision label for each of the workload namespaces with the gloo revision label.
      kubectl label namespace <namespace> istio.io/rev=gloo --overwrite
    3. Restart the workloads in each labeled namespace so that they are managed by the Gloo Operator Istio installation.
      • To restart all deployments in the namespace:
        kubectl rollout restart deployment -n <namespace>
      • To restart individual deployments in the namespace, such as to test a small number of deployments or to stagger the restart process:
        kubectl rollout restart deployment <deployment> -n <namespace>
    4. Verify that the workloads are successfully migrated. In the output, the name of istiod includes the gloo revision, indicating that the workload is now part of the Gloo-revisioned service mesh.
      istioctl proxy-status
      Example output:
      NAME                                                              CLUSTER     ...     ISTIOD                         VERSION
details-v1-7b6df9d8c8-s6kg5.bookinfo                              cluster1    ...     istiod-gloo-7c8f6fd4c4-m9k9t   1.28.5-solo
productpage-v1-bb494b7d7-xbtxr.bookinfo                           cluster1    ...     istiod-gloo-7c8f6fd4c4-m9k9t   1.28.5-solo
ratings-v1-55b478cfb6-wv2m5.bookinfo                              cluster1    ...     istiod-gloo-7c8f6fd4c4-m9k9t   1.28.5-solo
reviews-v1-6dfcc9fc7d-7k6qh.bookinfo                              cluster1    ...     istiod-gloo-7c8f6fd4c4-m9k9t   1.28.5-solo
reviews-v2-7dddd799b5-m5n2z.bookinfo                              cluster1    ...     istiod-gloo-7c8f6fd4c4-m9k9t   1.28.5-solo
  4. Update any existing Istio ingress or egress gateways to the gloo revision.
    1. Get the name and namespace of your gateway Helm release.
      helm ls -A
    2. Get the current values for the gateway Helm release in your cluster.
      helm get values <gateway_release> -n <namespace> -o yaml > gateway.yaml
    3. Upgrade your gateway Helm release.
      helm upgrade -i <gateway_release> istio/gateway \
  --version 1.28.5 \
  --namespace <namespace> \
  --set "revision=gloo" \
  -f gateway.yaml
    4. Verify that the gateway is successfully migrated. In the output, the name of istiod includes the gloo revision, indicating that the gateway is now included in the Gloo-revisioned data plane.
      istioctl proxy-status | grep gateway
      Example output:
      NAME                                                  CLUSTER    ...     ISTIOD                           VERSION
istio-ingressgateway-bdc4fd65f-ftmz9.istio-ingress    cluster1   ...     istiod-gloo-6495985689-rkwwd     1.28.5-solo
  5. Verify that Istio still correctly routes traffic requests to apps in your mesh. For example, if you deployed the Bookinfo sample app, you can send a curl request to the product page.
    kubectl port-forward -n istio-ingress svc/istio-ingressgateway 8080:80
curl -v http://localhost:8080/productpage
  6. Get the name and namespace of your previous istiod Helm release.
    helm ls -A
  7. Uninstall the unmanaged control plane.
    helm uninstall <istiod_release> -n istio-system
  8. Optional: If you previously installed the Istio CNI pods with a Helm chart, uninstall the release and delete the secret stored by Helm.
    helm uninstall <cni_release> -n istio-system
kubectl delete secret "sh.helm.release.v1.istio-cni.v1" -n istio-system
  9. Send another request to your apps to verify that traffic is still flowing.
    kubectl port-forward -n istio-ingress svc/istio-ingressgateway 8080:80
curl -v http://localhost:8080/productpage
The migration of your service mesh is now complete!
Migrate from revisionless Helm installations

If you currently install Istio by using Helm and do not use revisions to manage your installations, such as by labeling namespaces with istio-injection: enabled, you can migrate the management of the MutatingWebhookConfiguration to the Gloo Operator. The Gloo Operator uses the gloo revision by default to manage Istio installations in your cluster.
  1. Save your Istio installation values in environment variables.
    1. If you do not already have a license, decide the level of licensed features that you want, and contact an account representative to obtain the license.
    2. Choose the version of Istio that you want to install or upgrade to by reviewing the supported versions table.
    3. Save each value in an environment variable. If you prefer to specify license keys in a secret instead, see Licensing. Note that the Gloo Operator installs the Solo distribution of Istio by default for the version you specify, so neither the -solo image tag nor the repo URL are required.
      export SOLO_ISTIO_LICENSE_KEY=<license_key>
   export ISTIO_VERSION=1.28.5
    4. Install or upgrade istioctl with the same version of Istio that you saved.
      curl -L https://istio.io/downloadIstio | ISTIO_VERSION=${ISTIO_VERSION} sh -
   cd istio-${ISTIO_VERSION}
   export PATH=$PWD/bin:$PATH
  2. Install the Gloo Operator and deploy a managed istiod control plane.
    1. Install the Gloo Operator to the gloo-mesh namespace. This operator deploys and manages your Istio installation. For more information, see the Helm reference.Note that if you already installed Solo Enterprise for Istio, you can optionally reference the secret that Solo Enterprise for Istio automatically creates for your license in the –set manager.env.SOLO_ISTIO_LICENSE_KEY_SECRET_REF=gloo-mesh/license-keys flag instead.
      helm install gloo-operator oci://us-docker.pkg.dev/solo-public/gloo-operator-helm/gloo-operator \
  --version 0.5.2 \
  -n gloo-mesh \
  --create-namespace \
  --set manager.env.SOLO_ISTIO_LICENSE_KEY=${SOLO_ISTIO_LICENSE_KEY}
    2. Verify that the operator pod is running.
      kubectl get pods -n gloo-mesh -l app.kubernetes.io/name=gloo-operator
      Example output:
      gloo-operator-78d58d5c7b-lzbr5     1/1     Running   0          48s
    3. Create a ServiceMeshController custom resource to configure an Istio installation. For more information about the configurable fields, see the installation guide.
      kubectl apply -n gloo-mesh -f -<<EOF
apiVersion: operator.gloo.solo.io/v1
kind: ServiceMeshController
metadata:
  name: managed-istio
  labels:
    app.kubernetes.io/name: managed-istio
spec:
  dataplaneMode: Sidecar
  version: ${ISTIO_VERSION}
  # Uncomment if you installed the istio-cni
  # onConflict: Force
EOF
    4. Describe the ServiceMeshController and note that it cannot take over the istio-injection: enabled label until the webhook is deleted.
      kubectl describe ServiceMeshController -n gloo-mesh managed-istio
      Example output:
        - lastTransitionTime: "2024-12-12T19:41:52Z"
    message: MutatingWebhookConfiguration istio-sidecar-injector references default
      Istio revision istio-system/istiod; must be deleted before migration
    observedGeneration: 1
    reason: ErrorConflictDetected
    status: "False"
    type: WebhookDeployed
    5. Delete the existing webhook.
      kubectl delete mutatingwebhookconfiguration istio-sidecar-injector -n istio-system
    6. Verify that the ServiceMeshController is now healthy. In the Status section of the output, make sure that all statuses are True, and that the phase is SUCCEEDED.
      kubectl describe servicemeshcontroller -n gloo-mesh managed-istio
      Example output:
      ...
Status:
  Conditions:
    Last Transition Time:  2024-12-27T20:47:01Z
    Message:               Manifests initialized
    Observed Generation:   1
    Reason:                ManifestsInitialized
    Status:                True
    Type:                  Initialized
    Last Transition Time:  2024-12-27T20:47:02Z
    Message:               CRDs installed
    Observed Generation:   1
    Reason:                CRDInstalled
    Status:                True
    Type:                  CRDInstalled
    Last Transition Time:  2024-12-27T20:47:02Z
    Message:               Deployment succeeded
    Observed Generation:   1
    Reason:                DeploymentSucceeded
    Status:                True
    Type:                  ControlPlaneDeployed
    Last Transition Time:  2024-12-27T20:47:02Z
    Message:               Deployment succeeded
    Observed Generation:   1
    Reason:                DeploymentSucceeded
    Status:                True
    Type:                  CNIDeployed
    Last Transition Time:  2024-12-27T20:47:02Z
    Message:               Deployment succeeded
    Observed Generation:   1
    Reason:                DeploymentSucceeded
    Status:                True
    Type:                  WebhookDeployed
    Last Transition Time:  2024-12-27T20:47:02Z
    Message:               All conditions are met
    Observed Generation:   1
    Reason:                SystemReady
    Status:                True
    Type:                  Ready
  Phase:                   SUCCEEDED
Events:                    <none>
  3. Migrate your Istio-managed workloads to the managed control plane.
    1. Get the workload namespaces that you previously included in the service mesh by using the istio-injection=enabled label.
      kubectl get namespaces -l istio-injection=enabled
    2. Label each workload namespace with the gloo revision label.
      kubectl label namespace <namespace> istio.io/rev=gloo --overwrite
    3. Restart your workloads so that they are managed by the Gloo Operator Istio installation.
      • To restart all deployments in the namespace:
        kubectl rollout restart deployment -n <namespace>
      • To restart individual deployments in the namespace, such as to test a small number of deployments or to stagger the restart process:
        kubectl rollout restart deployment <deployment> -n <namespace>
    4. Verify that the workloads are successfully migrated. In the output, the name of istiod includes the gloo revision, indicating that the workload is now part of the Gloo-revisioned service mesh.
      istioctl proxy-status
      Example output:
      NAME                                                              CLUSTER     ...     ISTIOD                         VERSION
details-v1-7b6df9d8c8-s6kg5.bookinfo                              cluster1    ...     istiod-gloo-7c8f6fd4c4-m9k9t   1.28.5-solo
productpage-v1-bb494b7d7-xbtxr.bookinfo                           cluster1    ...     istiod-gloo-7c8f6fd4c4-m9k9t   1.28.5-solo
ratings-v1-55b478cfb6-wv2m5.bookinfo                              cluster1    ...     istiod-gloo-7c8f6fd4c4-m9k9t   1.28.5-solo
reviews-v1-6dfcc9fc7d-7k6qh.bookinfo                              cluster1    ...     istiod-gloo-7c8f6fd4c4-m9k9t   1.28.5-solo
reviews-v2-7dddd799b5-m5n2z.bookinfo                              cluster1    ...     istiod-gloo-7c8f6fd4c4-m9k9t   1.28.5-solo
    5. Remove the istio-injection=enabled label from the workload namespaces.
      kubectl label ns <namespace> istio-injection-
  4. Migrate any existing Istio ingress or egress gateways to the managed gloo control plane.
    1. Get the deployment name of your gateway.
      kubectl get deploy -n <gateway_namespace>
    2. Update each Istio gateway by restarting it.
      kubectl rollout restart deploy <gateway_name> -n <namespace>
    3. Verify that the gateway is successfully migrated. In the output, the name of istiod includes the gloo revision, indicating that the gateway is now included in the Gloo-revisioned data plane.
      istioctl proxy-status | grep gateway
      Example output:
      NAME                                                  CLUSTER    ...     ISTIOD                           VERSION
istio-ingressgateway-bdc4fd65f-ftmz9.istio-ingress    cluster1   ...     istiod-gloo-6495985689-rkwwd     1.28.5-solo
  5. Verify that Istio still correctly routes traffic requests to apps in your mesh. For example, if you deployed the Bookinfo sample app, you can send a curl request to the product page.
    kubectl port-forward -n istio-ingress svc/istio-ingressgateway 8080:80
curl -v http://localhost:8080/productpage
  6. Get the name and namespace of your previous istiod Helm release.
    helm ls -A
  7. Uninstall the unmanaged control plane.
    helm uninstall <istiod_release> -n istio-system
  8. Optional: If you previously installed the Istio CNI pods with a Helm chart, uninstall the release and delete the secret stored by Helm.
    helm uninstall <cni_release> -n istio-system
kubectl delete secret "sh.helm.release.v1.istio-cni.v1" -n istio-system
  9. Send another request to your apps to verify that traffic is still flowing.
    kubectl port-forward -n istio-ingress svc/istio-ingressgateway 8080:80
curl -v http://localhost:8080/productpage
The migration of your service mesh is now complete!
Migrate from the Istio lifecycle manager

You might have previously installed Istio and gateways by using Solo’s deprecated Istio lifecycle manager, such as by using the default settings in the getting started guides, the istioInstallations Helm settings in your Gloo Helm chart, or by directly creating IstioLifecycleManager and GatewayLifecycleManager custom resources. You can migrate from the Istio revision that your lifecycle manager currently runs, such as 1-28, to the revision that the Gloo Operator uses by default to manage Istio installations in your cluster, gloo.
Single cluster

  1. Save your Istio installation values in environment variables.
    1. If you do not already have a license, decide the level of licensed features that you want, and contact an account representative to obtain the license.
    2. Choose the version of Istio that you want to install or upgrade to by reviewing the supported versions table.
    3. Save each value in an environment variable. If you prefer to specify license keys in a secret instead, see Licensing. Note that the Gloo Operator installs the Solo distribution of Istio by default for the version you specify, so neither the -solo image tag nor the repo URL are required.
      export SOLO_ISTIO_LICENSE_KEY=<license_key>
   export ISTIO_VERSION=1.28.5
    4. Install or upgrade istioctl with the same version of Istio that you saved.
      curl -L https://istio.io/downloadIstio | ISTIO_VERSION=${ISTIO_VERSION} sh -
   cd istio-${ISTIO_VERSION}
   export PATH=$PWD/bin:$PATH
  2. Install the Gloo Operator and deploy a managed istiod control plane.
    1. Install the Gloo Operator to the gloo-mesh namespace. This operator deploys and manages your Istio installation. For more information, see the Helm reference.Note that if you already installed Solo Enterprise for Istio, you can optionally reference the secret that Solo Enterprise for Istio automatically creates for your license in the –set manager.env.SOLO_ISTIO_LICENSE_KEY_SECRET_REF=gloo-mesh/license-keys flag instead.
      helm install gloo-operator oci://us-docker.pkg.dev/solo-public/gloo-operator-helm/gloo-operator \
  --version 0.5.2 \
  -n gloo-mesh \
  --create-namespace \
  --set manager.env.SOLO_ISTIO_LICENSE_KEY=${SOLO_ISTIO_LICENSE_KEY}
    2. Verify that the operator pod is running.
      kubectl get pods -n gloo-mesh -l app.kubernetes.io/name=gloo-operator
      Example output:
      gloo-operator-78d58d5c7b-lzbr5     1/1     Running   0          48s
    3. Create a ServiceMeshController custom resource to configure an Istio installation. For more information about the configurable fields, see the installation guide.
      kubectl apply -n gloo-mesh -f -<<EOF
apiVersion: operator.gloo.solo.io/v1
kind: ServiceMeshController
metadata:
  name: managed-istio
  labels:
    app.kubernetes.io/name: managed-istio
spec:
  dataplaneMode: Sidecar
  version: ${ISTIO_VERSION}
  # Uncomment if you installed the istio-cni
  # onConflict: Force
EOF
    4. Verify that the ServiceMeshController is ready. In the Status section of the output, make sure that all statuses are True, and that the phase is SUCCEEDED.
      kubectl describe servicemeshcontroller -n gloo-mesh managed-istio
      Example output:
      ...
Status:
  Conditions:
    Last Transition Time:  2024-12-27T20:47:01Z
    Message:               Manifests initialized
    Observed Generation:   1
    Reason:                ManifestsInitialized
    Status:                True
    Type:                  Initialized
    Last Transition Time:  2024-12-27T20:47:02Z
    Message:               CRDs installed
    Observed Generation:   1
    Reason:                CRDInstalled
    Status:                True
    Type:                  CRDInstalled
    Last Transition Time:  2024-12-27T20:47:02Z
    Message:               Deployment succeeded
    Observed Generation:   1
    Reason:                DeploymentSucceeded
    Status:                True
    Type:                  ControlPlaneDeployed
    Last Transition Time:  2024-12-27T20:47:02Z
    Message:               Deployment succeeded
    Observed Generation:   1
    Reason:                DeploymentSucceeded
    Status:                True
    Type:                  CNIDeployed
    Last Transition Time:  2024-12-27T20:47:02Z
    Message:               Deployment succeeded
    Observed Generation:   1
    Reason:                DeploymentSucceeded
    Status:                True
    Type:                  WebhookDeployed
    Last Transition Time:  2024-12-27T20:47:02Z
    Message:               All conditions are met
    Observed Generation:   1
    Reason:                SystemReady
    Status:                True
    Type:                  Ready
  Phase:                   SUCCEEDED
Events:                    <none>
  3. Migrate your Istio-managed workloads to the managed gloo control plane.
    1. Get the workload namespaces that you previously labeled with an Istio revision, such as 1-28 in the following example.
      kubectl get namespaces -l istio.io/rev=1-28
    2. Overwrite the revision label for each of the workload namespaces with the gloo revision label.
      kubectl label namespace <namespace> istio.io/rev=gloo --overwrite
    3. Restart the workloads in each labeled namespace so that they are managed by the Gloo Operator Istio installation.
      • To restart all deployments in the namespace:
        kubectl rollout restart deployment -n <namespace>
      • To restart individual deployments in the namespace, such as to test a small number of deployments or to stagger the restart process:
        kubectl rollout restart deployment <deployment> -n <namespace>
    4. Verify that the workloads are successfully migrated. In the output, the name of istiod includes the gloo revision, indicating that the workload is now part of the Gloo-revisioned service mesh.
      istioctl proxy-status
      Example output:
      NAME                                                              CLUSTER     ...     ISTIOD                         VERSION
details-v1-7b6df9d8c8-s6kg5.bookinfo                              cluster1    ...     istiod-gloo-7c8f6fd4c4-m9k9t   1.28.5-solo
productpage-v1-bb494b7d7-xbtxr.bookinfo                           cluster1    ...     istiod-gloo-7c8f6fd4c4-m9k9t   1.28.5-solo
ratings-v1-55b478cfb6-wv2m5.bookinfo                              cluster1    ...     istiod-gloo-7c8f6fd4c4-m9k9t   1.28.5-solo
reviews-v1-6dfcc9fc7d-7k6qh.bookinfo                              cluster1    ...     istiod-gloo-7c8f6fd4c4-m9k9t   1.28.5-solo
reviews-v2-7dddd799b5-m5n2z.bookinfo                              cluster1    ...     istiod-gloo-7c8f6fd4c4-m9k9t   1.28.5-solo
  4. For each gateway that the gateway lifecycle manager created, create Helm releases to deploy new Istio gateways to the gloo revision.
    1. Create a new ingress gateway Helm release for the gloo control plane revision. Note that if you maintain your own services to expose gateways, you can disable the load balancer services that are defined by default in the gateway Helm release by including the --set service.type=None flag in this command. Then, you can switch from the old to the new gateways by updating the load balancer services to point to the new gateways.
      helm install istio-ingressgateway istio/gateway \
  --version ${ISTIO_VERSION} \
  --namespace istio-ingress \
  --set "revision=gloo"
    2. Verify that the gateway is successfully deployed. In the output, the name of istiod includes the gloo revision, indicating that the gateway is included in the Gloo-revisioned data plane.
      istioctl proxy-status | grep gateway
      Example output:
      NAME                                                  CLUSTER    ...     ISTIOD                           VERSION
istio-ingressgateway-bdc4fd65f-ftmz9.istio-ingress    cluster1    ...    istiod-gloo-6495985689-rkwwd     1.28.5-solo
  5. Verify that Istio now routes traffic requests to apps in your mesh through the new gateway that you deployed. For example, if you deployed the Bookinfo sample app, you can send a curl request to the product page.
    kubectl port-forward -n istio-ingress svc/istio-ingressgateway 8080:80
curl -v http://localhost:8080/productpage
  6. Delete the GatewayLifecycleManager and IstioLifecycleManager managed installations. The steps vary based on whether you created the resources directly, or used the istioInstallations section of the gloo-platform Helm chart.
    1. Delete the GatewayLifecycleManager resources for the gateways that you installed.
      kubectl delete GatewayLifecycleManager istio-eastwestgateway -n gloo-mesh
kubectl delete GatewayLifecycleManager istio-ingressgateway -n gloo-mesh
kubectl delete GatewayLifecycleManager istio-egressgateway -n gloo-mesh
    2. Verify that the gateway resources are removed before you proceed to the next step.
      kubectl get all -n gloo-mesh-gateways
    3. Delete the IstioLifecycleManager for the istiod control plane. Note that your resource might have a different name.
      kubectl delete IstioLifecycleManager istiod-control-plane -n gloo-mesh
    1. Get the Helm values of your current Solo Enterprise for Istio installation.
      helm get values gloo-platform -o yaml -n gloo-mesh > gloo-single.yaml
open gloo-single.yaml
    2. In the values file, replace the istioInstallations section with the following, and save the file.
      
istioInstallations:
  enabled: false
    3. Upgrade your installation with your updated values file.
      helm upgrade gloo-platform gloo-platform/gloo-platform \
--namespace gloo-mesh \
-f gloo-single.yaml \
--version $GLOO_VERSION
  7. Optional: If you previously installed the Istio CNI pods with a Helm chart, uninstall the release and delete the secret stored by Helm.
    helm uninstall <cni_release> -n istio-system
kubectl delete secret "sh.helm.release.v1.istio-cni.v1" -n istio-system
  8. Send another request to your apps to verify that traffic is still flowing.
    kubectl port-forward -n istio-ingress svc/istio-ingressgateway 8080:80
curl -v http://localhost:8080/productpage
The migration of your service mesh is now complete!
Multicluster

Considerations

Before you install a multicluster sidecar mesh, review the following considerations and requirements.
Version and license requirements
  • Multicluster setups require the Solo distribution of Istio version 1.24.3 or later (1.24.3-solo), including the Solo distribution of istioctl.
  • This feature requires your mesh to be installed with the Solo distribution of Istio and an Enterprise-level license for Solo Enterprise for Istio. Contact your account representative to obtain a valid license.
Components
In the following steps, you install the Istio ambient components in each cluster to successfully create east-west gateways and establish multicluster peering, even if you plan to use a sidecar mesh. However, sidecar mesh setups continue to use sidecar injection for your workloads. Your workloads are not added to an ambient mesh. For more information about running both ambient and sidecar components in one mesh setup, see Ambient-sidecar interoperability.
Migrate each service mesh

  1. Save your Istio installation values in environment variables.
    1. Set your Enterprise level license for Solo Enterprise for Istio as an environment variable. If you do not have one, contact an account representative. If you prefer to specify license keys in a secret instead, see Licensing. Note that you might have previously saved this key in another variable, such as ${SOLO_LICENSE_KEY} or ${GLOO_MESH_LICENSE_KEY}.
      export SOLO_ISTIO_LICENSE_KEY=<enterprise_license_key>
    2. Choose the version of Istio that you want to install or upgrade to by reviewing the supported versions.
    3. Save the Solo distribution of Istio version.
      export ISTIO_VERSION=1.28.5
   export ISTIO_IMAGE=${ISTIO_VERSION}-solo
    4. Save the repo key for the minor version of the Solo distribution of Istio that you want to install. This is the 12-character hash at the end of the repo URL us-docker.pkg.dev/gloo-mesh/istio-<repo-key>, which you can find in the Istio images built by Solo.io support article.
      # 12-character hash at the end of the repo URL
   export REPO_KEY=<repo_key>
   export REPO=us-docker.pkg.dev/gloo-mesh/istio-${REPO_KEY}
   export HELM_REPO=us-docker.pkg.dev/gloo-mesh/istio-helm-${REPO_KEY}
    5. Get the Solo distribution of Istio binary and install istioctl, which you use for multicluster linking and gateway commands. This script automatically detects your OS and architecture, downloads the appropriate Solo distribution of Istio binary, and verifies the installation.
      bash <(curl -sSfL https://raw.githubusercontent.com/solo-io/doc-examples/main/istio/install-istioctl.sh)
   export PATH=${HOME}/.istioctl/bin:${PATH}
    6. Save the names and kubeconfig contexts of each cluster. This guide uses two clusters as an example. To add more clusters to the multicluster setup, include them in the arrays.
      export cluster1=<cluster1_name>
   export context1=<cluster1_context>
   export cluster2=<cluster2_name>
   export context2=<cluster2_context>
  2. Each cluster in the multicluster setup must have a shared root of trust. This can be achieved by providing a root certificate signed by a PKI provider, or a custom root certificate created for this purpose. The root certificate signs a unique intermediate CA certificate for each cluster.
    By default, the Istio CA generates a self-signed root certificate and key, and uses them to sign the workload certificates. For more information, see the Plug in CA Certificates guide in the community Istio documentation.
    For demo installations, you can run the following function to quickly generate and plug in the certificates and key for the Istio CA:
    curl -L https://istio.io/downloadIstio | ISTIO_VERSION=${ISTIO_VERSION} sh -
cd istio-${ISTIO_VERSION}

mkdir -p certs
pushd certs
make -f ../tools/certs/Makefile.selfsigned.mk root-ca

function create_cacerts_secret() {
  context=${1:?context}
  cluster=${2:?cluster}
  make -f ../tools/certs/Makefile.selfsigned.mk ${cluster}-cacerts
  kubectl --context=${context} create ns istio-system || true
  kubectl --context=${context} create secret generic cacerts -n istio-system \
    --from-file=${cluster}/ca-cert.pem \
    --from-file=${cluster}/ca-key.pem \
    --from-file=${cluster}/root-cert.pem \
    --from-file=${cluster}/cert-chain.pem
}

create_cacerts_secret ${context1} ${cluster1}
create_cacerts_secret ${context2} ${cluster2}

cd ../..
    To enhance the security of your setup even further and have full control over the Istio CA lifecycle, you can generate and store the root and intermediate CA certificates and keys with your own PKI provider. You can then use tools such as cert-manager to send certificate signing requests on behalf of istiod to your PKI provider. Cert-manager stores the signed intermediate certificates and keys in the cacerts Kubernetes secret so that istiod can use these credentials to issue leaf certificates for the workloads in the service mesh. You can set up cert-manager to also check the certificates and renew them before they expire.
    AWS Private CA issuer and cert-manager: For an architectural overview of this certificate setup, see Bring your own Istio CAs with AWS. For steps on how to deploy this certificate setup, check out this Solo.io blog post. Be sure to repeat the steps so that a cacerts secret exists in each cluster.
  3. Save the name and kubeconfig context of a workload cluster in the following environment variables. Each time you repeat the steps in this guide, you change these variables to the next workload cluster’s name and context.
    export CLUSTER_NAME=<workload-cluster-name>
export CLUSTER_CONTEXT=<workload-cluster-context>
  4. Install the Gloo Operator and deploy a managed istiod control plane.
    1. Install the Gloo Operator to the gloo-mesh namespace. This operator deploys and manages your Istio installation. For more information, see the Helm reference.Note that if you already installed Solo Enterprise for Istio, you can optionally reference the secret that Solo Enterprise for Istio automatically creates for your license in the –set manager.env.SOLO_ISTIO_LICENSE_KEY_SECRET_REF=gloo-mesh/license-keys flag instead.
      helm install gloo-operator oci://us-docker.pkg.dev/solo-public/gloo-operator-helm/gloo-operator \
  --version 0.5.2 \
  -n gloo-mesh \
  --create-namespace \
  --set manager.env.SOLO_ISTIO_LICENSE_KEY=${SOLO_ISTIO_LICENSE_KEY}
    2. Verify that the operator pod is running.
      kubectl get pods -n gloo-mesh --context ${CLUSTER_CONTEXT} -l app.kubernetes.io/name=gloo-operator
      Example output:
      gloo-operator-78d58d5c7b-lzbr5     1/1     Running   0          48s
    3. Create a ServiceMeshController custom resource to configure an Istio installation. For more information about the configurable fields, see the installation guide.
      kubectl --context ${CLUSTER_CONTEXT} apply -n gloo-mesh -f -<<EOF
apiVersion: operator.gloo.solo.io/v1
kind: ServiceMeshController
metadata:
  name: managed-istio
  labels:
    app.kubernetes.io/name: managed-istio
spec:
  cluster: ${CLUSTER_NAME}
  network: ${CLUSTER_NAME}
  dataplaneMode: Ambient # required for multicluster setups
  installNamespace: istio-system
  version: ${ISTIO_VERSION}
  # Uncomment if you installed the istio-cni
  # onConflict: Force
EOF
    4. Verify that the ServiceMeshController is ready. In the Status section of the output, make sure that all statuses are True, and that the phase is SUCCEEDED.
      kubectl --context ${CLUSTER_CONTEXT} describe servicemeshcontroller -n gloo-mesh managed-istio
      Example output:
      ...
Status:
  Conditions:
    Last Transition Time:  2024-12-27T20:47:01Z
    Message:               Manifests initialized
    Observed Generation:   1
    Reason:                ManifestsInitialized
    Status:                True
    Type:                  Initialized
    Last Transition Time:  2024-12-27T20:47:02Z
    Message:               CRDs installed
    Observed Generation:   1
    Reason:                CRDInstalled
    Status:                True
    Type:                  CRDInstalled
    Last Transition Time:  2024-12-27T20:47:02Z
    Message:               Deployment succeeded
    Observed Generation:   1
    Reason:                DeploymentSucceeded
    Status:                True
    Type:                  ControlPlaneDeployed
    Last Transition Time:  2024-12-27T20:47:02Z
    Message:               Deployment succeeded
    Observed Generation:   1
    Reason:                DeploymentSucceeded
    Status:                True
    Type:                  CNIDeployed
    Last Transition Time:  2024-12-27T20:47:02Z
    Message:               Deployment succeeded
    Observed Generation:   1
    Reason:                DeploymentSucceeded
    Status:                True
    Type:                  WebhookDeployed
    Last Transition Time:  2024-12-27T20:47:02Z
    Message:               All conditions are met
    Observed Generation:   1
    Reason:                SystemReady
    Status:                True
    Type:                  Ready
  Phase:                   SUCCEEDED
Events:                    <none>
  5. Migrate your Istio-managed workloads to the managed gloo control plane. The steps vary based on whether you labeled workload namespaces with revision labels, such as istio.io/rev=1-28, or with injection labels, such as istio-injection=enabled.
    1. Get the workload namespaces that you previously labeled with an Istio revision, such as 1-28 in the following example.
      kubectl get namespaces -l istio.io/rev=1-28
    2. Overwrite the revision label for each of the workload namespaces with the gloo revision label.
      kubectl label namespace <namespace> istio.io/rev=gloo --overwrite
    3. Restart the workloads in each labeled namespace so that they are managed by the Gloo Operator Istio installation.
      kubectl --context ${CLUSTER_CONTEXT} rollout restart deployment -n <namespace> <deployment>
    4. Verify that the workloads are successfully migrated. In the output, the name of istiod includes the gloo revision, indicating that the workload is now part of the Gloo-revisioned service mesh.
      istioctl --context ${CLUSTER_CONTEXT} proxy-status
      Example output:
      NAME                                                              CLUSTER     ...     ISTIOD                         VERSION
details-v1-7b6df9d8c8-s6kg5.bookinfo                              cluster1    ...     istiod-gloo-7c8f6fd4c4-m9k9t   1.28.5-solo
productpage-v1-bb494b7d7-xbtxr.bookinfo                           cluster1    ...     istiod-gloo-7c8f6fd4c4-m9k9t   1.28.5-solo
ratings-v1-55b478cfb6-wv2m5.bookinfo                              cluster1    ...     istiod-gloo-7c8f6fd4c4-m9k9t   1.28.5-solo
reviews-v1-6dfcc9fc7d-7k6qh.bookinfo                              cluster1    ...     istiod-gloo-7c8f6fd4c4-m9k9t   1.28.5-solo
reviews-v2-7dddd799b5-m5n2z.bookinfo                              cluster1    ...     istiod-gloo-7c8f6fd4c4-m9k9t   1.28.5-solo
    1. Get the workload namespaces that you previously included in the service mesh by using the istio-injection=enabled label.
      kubectl get namespaces -l istio-injection=enabled
    2. Label each workload namespace with the gloo revision label.
      kubectl label namespace <namespace> istio.io/rev=gloo --overwrite
    3. Restart your workloads so that they are managed by the Gloo Operator Istio installation.
      • To restart all deployments in the namespace:
        kubectl rollout restart deployment -n <namespace>
      • To restart individual deployments in the namespace, such as to test a small number of deployments or to stagger the restart process:
        kubectl rollout restart deployment <deployment> -n <namespace>
    4. Verify that the workloads are successfully migrated. In the output, the name of istiod includes the gloo revision, indicating that the workload is now part of the Gloo-revisioned service mesh.
      istioctl proxy-status
      Example output:
      NAME                                                              CLUSTER     ...     ISTIOD                         VERSION
details-v1-7b6df9d8c8-s6kg5.bookinfo                              cluster1    ...     istiod-gloo-7c8f6fd4c4-m9k9t   1.28.5-solo
productpage-v1-bb494b7d7-xbtxr.bookinfo                           cluster1    ...     istiod-gloo-7c8f6fd4c4-m9k9t   1.28.5-solo
ratings-v1-55b478cfb6-wv2m5.bookinfo                              cluster1    ...     istiod-gloo-7c8f6fd4c4-m9k9t   1.28.5-solo
reviews-v1-6dfcc9fc7d-7k6qh.bookinfo                              cluster1    ...     istiod-gloo-7c8f6fd4c4-m9k9t   1.28.5-solo
reviews-v2-7dddd799b5-m5n2z.bookinfo                              cluster1    ...     istiod-gloo-7c8f6fd4c4-m9k9t   1.28.5-solo
    5. Remove the istio-injection=enabled label from the workload namespaces.
      kubectl label ns <namespace> istio-injection-
  6. For each ingress or egress gateway that the gateway lifecycle manager created, create Helm releases to deploy new Istio gateways to the gloo revision.
    1. For ingress gateways: Create a new ingress gateway Helm release for the gloo control plane revision. Note that if you maintain your own services to expose the gateways, you can disable the load balancer services that are defined by default in the gateway Helm release by including the --set service.type=None flag in this command. Then, you can switch from the old to the new gateways by updating the load balancer services to point to the new gateways.
      helm install istio-ingressgateway istio/gateway \
  --kube-context ${CLUSTER_CONTEXT} \
  --version ${ISTIO_VERSION} \
  --namespace istio-ingress \
  --create-namespace \
  --set "revision=gloo"
    2. Verify that the gateways are successfully deployed. In the output, the name of istiod includes the gloo revision, indicating that the gateways are included in the Gloo-revisioned data plane.
      istioctl --context ${CLUSTER_CONTEXT} proxy-status | grep gateway
      Example output:
      NAME                                                  CLUSTER    ...     ISTIOD                           VERSION
istio-eastwestgateway-bdc4fd65f-ftmz9.istio-eastwest  cluster1    ...     istiod-gloo-6495985689-rkwwd     1.28.5-solo
istio-ingressgateway-bdc4fd65f-ftmz9.istio-ingress    cluster1    ...     istiod-gloo-6495985689-rkwwd     1.28.5-solo
  7. Verify that Istio now routes traffic requests to apps in your mesh through the new gateway that you deployed. For example, if you deployed the Bookinfo sample app, you can send a curl request to the product page.
    kubectl --context ${CLUSTER_CONTEXT} port-forward -n istio-ingress svc/istio-ingressgateway 8080:80
curl -v http://localhost:8080/productpage
  8. Optional: If you previously installed the Istio CNI pods with a Helm chart, uninstall the release and delete the secret stored by Helm.
    helm uninstall <cni_release> -n istio-system
kubectl delete secret "sh.helm.release.v1.istio-cni.v1" -n istio-system
  9. Apply the CRDs for the Kubernetes Gateway API to your cluster, which are required to create components such as waypoint proxies for L7 traffic policies, gateways with the Gateway resource, and more.
    kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.4.0/standard-install.yaml --context ${CLUSTER_CONTEXT}
  10. Create an east-west gateway in the istio-eastwest namespace. In each cluster, the east-west gateway is implemented as a ztunnel that facilitates traffic between services across clusters in your multicluster mesh.
    You can use either LoadBalancer or NodePort addresses for cross-cluster traffic.
    Create the east-west gateway in both clusters. Cross-cluster traffic though this gateway resolves to the LoadBalancer address. For customization options, see the gateway guide in the Istio docs.
    • Solo distribution of istioctl: For more information about this command, see the CLI reference.
      function create_ew_gateway_lb() {
  context=${1:?context}
  cluster=${2:?cluster}
  kubectl create namespace istio-eastwest --context ${context}
  istioctl multicluster expose --namespace istio-eastwest --context ${context} --generate > ew-gateway-${cluster}.yaml
  kubectl apply -f ew-gateway-${cluster}.yaml --context ${context}
}
create_ew_gateway_lb ${context1} ${cluster1}
create_ew_gateway_lb ${context2} ${cluster2}
    Create the east-west gateway in both clusters. Note that the gateway must still be created with a stable internal IP address, which is required for xDS communication with the istiod control plane in each cluster. NodePort peering is used for data plane communication, in that requests to services resolve to the NodePort instead of the gateway’s stable IP address. For more information about NodePort peering considerations and requirements, see Cross-cluster traffic addresses.
    • Solo distribution of istioctl: For more information about this command, see the CLI reference.
      function create_ew_gateway_nodeport() {
  context=${1:?context}
  cluster=${2:?cluster}
  kubectl create namespace istio-eastwest --context ${context}
  istioctl multicluster expose --namespace istio-eastwest --context ${context} --generate > ew-gateway-${cluster}.yaml
  kubectl apply -f ew-gateway-${cluster}.yaml --context ${context}
  kubectl annotate gateway istio-eastwest -n istio-eastwest peering.solo.io/data-plane-service-type=NodePort --context ${context}
}

create_ew_gateway_nodeport ${context1} ${cluster1}
create_ew_gateway_nodeport ${context2} ${cluster2}
  11. Verify that the east-west gateway is successfully deployed.
    kubectl get pods -n istio-eastwest --context $CLUSTER_CONTEXT
  12. If you have Istio installations in multiple clusters that the GatewayLifecycleManager and IstioLifecycleManager managed, be sure to repeat steps 3 - 11 in each cluster before you continue. The next step deletes the GatewayLifecycleManager and IstioLifecycleManager resources from the management cluster, which uninstalls the old Istio installations from every workload cluster in your multicluster setup. Be sure to reset the value of the $CLUSTER_NAME and $CLUSTER_CONTEXT environment variables to the next workload cluster.
Link clusters

Link clusters to enable cross-cluster service discovery and allow traffic to be routed through east-west gateways across clusters.
  1. Optional: Before you link clusters, you can check the individual readiness of each cluster for linking by running the istioctl multicluster check --precheck command. For more information about this command, see the CLI reference. If any checks fail, run the command with --verbose, and see Validate your multicluster setup.
    istioctl multicluster check --precheck --contexts="$context1,$context2"
    Before continuing to the next step, make sure that the following checks pass as expected:
    ✅ Relevant environment variables on istiod are supported.
    ✅ The license in use by istiod supports multicluster.
    ✅ All istiod, ztunnel, and east-west gateway pods are healthy.
    ✅ The east-west gateway is programmed.
  2. Link clusters to enable cross-cluster service discovery and allow traffic to be routed through east-west gateways across clusters. Note that you can either link the clusters bi-directionally or asymmetrically. In a standard bi-directional setup, services in any of the linked clusters can send requests to and receive requests from the services in any of the other linked clusters. In an asymmetrical setup, you allow one cluster to send requests to another cluster, but the other cluster cannot send requests back to the first cluster.
    In the Solo distribution of Istio 1.29 or later, you can use the peering Helm chart to link your clusters.
    1. Get the addresses of the east-west gateway in each cluster. The following commands show examples for two clusters.
      export CLUSTER1_EW_ADDRESS=$(kubectl get svc -n istio-eastwest istio-eastwest --context ${context1} -o jsonpath="{.status.loadBalancer.ingress[0]['hostname','ip']}")
export CLUSTER2_EW_ADDRESS=$(kubectl get svc -n istio-eastwest istio-eastwest --context ${context2} -o jsonpath="{.status.loadBalancer.ingress[0]['hostname','ip']}")
echo "Cluster1 east-west gateway: $CLUSTER1_EW_ADDRESS"
echo "Cluster2 east-west gateway: $CLUSTER2_EW_ADDRESS"
    2. Link the clusters by creating Helm releases in each cluster to represent the other clusters. In each cluster where you create Helm releases, a Gateway resource is created that uses the istio-remote GatewayClass. This class allows the gateway to connect to other clusters by using the addresses of the east-west gateways.
      • Be sure to update the region, and optionally zone, of each cluster.
      • If you want to use NodePorts instead of the gateway LoadBalancer IP address for cross-cluster traffic, change the preferredDataplaneServiceType to nodeport. This automatically configures the peering gateways for NodePort-based cross-cluster traffic. For more information about NodePort peering considerations and requirements, see Cross-cluster traffic addresses.
      • The following example depicts a bi-directional setup. In an asymmetrical setup, you create Helm releases to only represent the directionality you want to allow.
      helm upgrade -i peering-remote oci://${HELM_REPO}/peering \
  --version ${ISTIO_IMAGE} \
  --namespace istio-eastwest \
  --kube-context ${context1} \
  -f - <<EOF
remote:
  create: true
  items:
    # Remote peer configuration for cluster2
    - name: istio-remote-peer-${cluster2}
      cluster: ${cluster2}
      network: ${cluster2}
      # Change to "Hostname" if using hostname
      addressType: IPAddress
      address: ${CLUSTER2_EW_ADDRESS}
      # Change to "nodeport" if using NodePorts
      preferredDataplaneServiceType: loadbalancer
      trustDomain: cluster.local
      region: "<cluster2_region>"
      # Uncomment as needed
      # zone: "<cluster2_zone>"
EOF
helm upgrade -i peering-remote oci://${HELM_REPO}/peering \
  --version ${ISTIO_IMAGE} \
  --namespace istio-eastwest \
  --kube-context ${context2} \
  -f - <<EOF
remote:
  create: true
  items:
    # Remote peer configuration for cluster1
    - name: istio-remote-peer-${cluster1}
      cluster: ${cluster1}
      network: ${cluster1}
      # Change to "Hostname" if using hostname
      addressType: IPAddress
      address: ${CLUSTER1_EW_ADDRESS}
      # Change to "nodeport" if using NodePorts
      preferredDataplaneServiceType: loadbalancer
      trustDomain: cluster.local
      region: "<cluster1_region>"
      # Uncomment as needed
      # zone: "<cluster1_zone>"
EOF
    Use the istioctl multicluster link command to quickly link clusters.
    1. Verify that the contexts for the clusters that you want to include in the multicluster mesh are listed in your kubeconfig file, which is required for the istioctl multicluster link command. If you do not have access to the kubeconfig files, use the declarative resources tab.
      kubectl config get-contexts
      • In the output, note the names of the cluster contexts, which you use in the next step to link the clusters.
      • If you have multiple kubeconfig files, you can generate a merged kubeconfig file by running the following command.
        KUBECONFIG=<kubeconfig_file1>.yaml:<file2>.yaml kubectl config view --flatten
    2. Using the names of the cluster contexts, link the clusters so that they can communicate. To take a look at the Gateway resources that this command creates, you can include the --generate flag in the command. For more information about this command, see the CLI reference.
      • Bi-directional: You can use the following istioctl command to quickly link the clusters bi-directionally. In each cluster, Gateway resources are created that use the istio-remote GatewayClass. This class allows the gateways to connect to other clusters by using the addresses of the east-west gateways.
        istioctl multicluster link --namespace istio-eastwest --contexts="$context1,$context2"
```<pre><code> If you want to use NodePorts instead of the gateway LoadBalancer IP address for cross-cluster traffic, use the `--preferred-data-plane-service-type nodeport` flag when linking clusters. This automatically configures the peering gateways for NodePort-based cross-cluster traffic. For more information about NodePort peering considerations and requirements, see [Cross-cluster traffic addresses](#cross-cluster-traffic-addresses).
    istioctl multicluster link --namespace istio-eastwest --contexts=&quot;$context1,$context2&quot; --preferred-data-plane-service-type nodeport
     Example output for two clusters:
 ```
 Gateway istio-eastwest/istio-remote-peer-cluster1 applied to cluster "cluster2" pointing to cluster "cluster1" (network "cluster1")
 Gateway istio-eastwest/istio-remote-peer-cluster2 applied to cluster "cluster1" pointing to cluster "cluster2" (network "cluster2")
 ```

    • Asymmetrical: You can use the following istioctl command to quickly link the clusters asymmetrically. The services in the cluster in the --from flag can send requests to services in the cluster in the --to flag, but sending requests in the reverse direction is not permitted.
      For example, this command allows services in cluster1’s mesh to send requests to services in cluster2’s mesh through cluster2’s east-west gateway. However, the reverse is not permitted: services in cluster2’s mesh cannot send requests through cluster1’s east-west gateway to services in cluster1.
      istioctl multicluster link --namespace istio-eastwest --from ${context1} --to ${context2}
```<pre><code> If you want to use NodePorts instead of the gateway LoadBalancer IP address for cross-cluster traffic, use the `--preferred-data-plane-service-type nodeport` flag when linking clusters. This automatically configures the peering gateways for NodePort-based cross-cluster traffic. For more information about NodePort peering considerations and requirements, see [Cross-cluster traffic addresses](#cross-cluster-traffic-addresses).
    istioctl multicluster link --namespace istio-eastwest --from ${context1} --to ${context2} --preferred-data-plane-service-type nodeport
     Example output:
 ```
 Gateway istio-eastwest/istio-remote-peer-cluster2 applied to cluster "cluster1" pointing to cluster "cluster2" (network "cluster2")
 ```

    Link the clusters by declaratively creating istio-remote peer gateways.
    Bi-directional: Use the following Gateway resources to create an istio-remote peer gateway in each cluster. The istio-remote GatewayClass allows the gateways to connect to other clusters by using the addresses of the east-west gateways.
    1. Get the addresses of the east-west gateway in each cluster. The following commands show examples for two clusters.
      export CLUSTER1_EW_ADDRESS=$(kubectl get svc -n istio-eastwest istio-eastwest --context ${context1} -o jsonpath="{.status.loadBalancer.ingress[0]['hostname','ip']}")
export CLUSTER2_EW_ADDRESS=$(kubectl get svc -n istio-eastwest istio-eastwest --context ${context2} -o jsonpath="{.status.loadBalancer.ingress[0]['hostname','ip']}")

echo "Cluster1 east-west gateway: $CLUSTER1_EW_ADDRESS"
echo "Cluster2 east-west gateway: $CLUSTER2_EW_ADDRESS"
    2. Using the east-west gateway addresses, create a Gateway resource in each cluster to represent the other cluster.
      • In the labels section, be sure to update the locality labels according to the region, and optionally zone, of each cluster. For more information about locality support, see the release notes.
      • If you want to use NodePorts instead of the gateway LoadBalancer IP address for cross-cluster traffic, uncomment the peering.solo.io/preferred-data-plane-service-type: NodePort annotation from each Gateway resource. Additionally, you can comment out the HBONE listener in each gateway, because traffic is routed through the NodePort directly. For more information about NodePort peering considerations and requirements, see Cross-cluster traffic addresses.
      kubectl apply --context ${context1} -f- <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  annotations:
    gateway.istio.io/service-account: istio-eastwest
    gateway.istio.io/trust-domain: ${cluster2}
    # Uncomment for NodePort-based cross-cluster traffic
    # peering.solo.io/preferred-data-plane-service-type: NodePort
  labels:
    topology.istio.io/network: ${cluster2}
    topology.kubernetes.io/region: "<cluster2_region>"
    # Uncomment as needed
    # topology.kubernetes.io/zone: "<cluster2_zone>"
  name: istio-remote-peer-${cluster2}
  namespace: istio-eastwest
spec:
  addresses:
  # Change to 'type: Hostname' as needed, such as for AWS hostnames
  - type: IPAddress
    value: $CLUSTER2_EW_ADDRESS
  gatewayClassName: istio-remote
  listeners:
  # Comment HBONE out for NodePort-based cross-cluster traffic
  - name: cross-network
    port: 15008
    protocol: HBONE
    tls:
      mode: Passthrough
  - name: xds-tls
    port: 15012
    protocol: TLS
    tls:
      mode: Passthrough
EOF
kubectl apply --context ${context2} -f- <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  annotations:
    gateway.istio.io/service-account: istio-eastwest
    gateway.istio.io/trust-domain: ${cluster1}
    # Uncomment for NodePort-based cross-cluster traffic
    # peering.solo.io/preferred-data-plane-service-type: NodePort
  labels:
    topology.istio.io/network: ${cluster1}
    topology.kubernetes.io/region: "<cluster1_region>"
    # Uncomment as needed
    # topology.kubernetes.io/zone: "<cluster1_zone>"
  name: istio-remote-peer-${cluster1}
  namespace: istio-eastwest
spec:
  addresses:
  # Change to 'type: Hostname' as needed, such as for AWS hostnames
  - type: IPAddress
    value: $CLUSTER1_EW_ADDRESS
  gatewayClassName: istio-remote
  listeners:
  # Comment HBONE out for NodePort-based cross-cluster traffic
  - name: cross-network
    port: 15008
    protocol: HBONE
    tls:
      mode: Passthrough
  - name: xds-tls
    port: 15012
    protocol: TLS
    tls:
      mode: Passthrough
EOF
    Asymmetrical: Use the following Gateway resources to create an istio-remote peer gateway in only some clusters. The istio-remote GatewayClass allows the gateway in one cluster to connect to another cluster by using the address of the east-west gateway, but sending requests in the reverse direction is not permitted.
    1. Get the address of the east-west gateway in the cluster that you want to send traffic to. The following command shows an example for cluster2.
      export CLUSTER2_EW_ADDRESS=$(kubectl get svc -n istio-eastwest istio-eastwest --context ${context2} -o jsonpath="{.status.loadBalancer.ingress[0]['hostname','ip']}")

echo "Cluster2 east-west gateway: $CLUSTER2_EW_ADDRESS"
    2. Using the east-west gateway address, create a Gateway resource in the cluster that you want to send requests from. For example, this Gateway resource allows services in cluster1’s mesh to send requests to services in cluster2’s mesh through cluster2’s east-west gateway. However, the reverse is not permitted: services in cluster2’s mesh cannot send requests through cluster1’s east-west gateway to services in cluster1.
      • In the labels section, be sure to update the locality labels according to the region, and optionally zone, of each cluster. For more information about locality support, see the release notes.
      • If you want to use NodePorts instead of the gateway LoadBalancer IP address for cross-cluster traffic, uncomment the peering.solo.io/preferred-data-plane-service-type: NodePort annotation from each Gateway resource. Additionally, you can comment out the HBONE listener in each gateway, because traffic is routed through the NodePort directly. For more information about NodePort peering considerations and requirements, see Cross-cluster traffic addresses.
      kubectl apply --context ${context1} -f- <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  annotations:
    gateway.istio.io/service-account: istio-eastwest
    gateway.istio.io/trust-domain: ${cluster2}
    # Uncomment for NodePort-based cross-cluster traffic
    # peering.solo.io/preferred-data-plane-service-type: NodePort
  labels:
    topology.istio.io/network: ${cluster2}
    topology.kubernetes.io/region: "<cluster2_region>"
    # Uncomment as needed
    # topology.kubernetes.io/zone: "<cluster2_zone>"
  name: istio-remote-peer-${cluster2}
  namespace: istio-eastwest
spec:
  addresses:
  # Change to 'type: Hostname' as needed, such as for AWS hostnames
  - type: IPAddress
    value: $CLUSTER2_EW_ADDRESS
  gatewayClassName: istio-remote
  listeners:
  # Comment HBONE out for NodePort-based cross-cluster traffic
  - name: cross-network
    port: 15008
    protocol: HBONE
    tls:
      mode: Passthrough
  - name: xds-tls
    port: 15012
    protocol: TLS
    tls:
      mode: Passthrough
EOF
  3. Verify that peer linking was successful by running the istioctl multicluster check command. If any checks fail, run the command with --verbose, and see Validate your multicluster setup.
    istioctl multicluster check --contexts="$context1,$context2"
    In this example output, the remote peer gateways are successfully connected, and all other checks passed successfully. No global services exist because no app services are exposed across clusters in the multicluster mesh yet.
    === Cluster: cluster1 ===
✅ Incompatible Environment Variable Check: all relevant environment variables are valid
✅ License Check: license is valid for multicluster
✅ CNI DNS Capture Check: AMBIENT_DNS_CAPTURE is enabled
✅ Pod Check (istiod): all pods healthy
✅ Pod Check (ztunnel): all pods healthy
✅ Pod Check (eastwest gateway istio-eastwest/istio-eastwest): all pods healthy
✅ Gateway Check: all eastwest gateways programmed
     ✅ istio-eastwest/istio-eastwest available at aab8471c7fcfa4a3c82f2d217b015d97-396238517.us-east-1.elb.amazonaws.com
✅ Peers Check: all clusters connected
     ✅ Connected to gloo-gateway-docs-mgt via ab46aa29a49914da789a6d5422aca279-541415195.us-east-2.elb.amazonaws.com
ℹ️  Shared Services Check: no globally shared services found
====== 

=== Cluster: cluster2 ===
✅ Incompatible Environment Variable Check: all relevant environment variables are valid
✅ License Check: license is valid for multicluster
✅ CNI DNS Capture Check: AMBIENT_DNS_CAPTURE is enabled
✅ Pod Check (istiod): all pods healthy
✅ Pod Check (ztunnel): all pods healthy
✅ Pod Check (eastwest gateway istio-eastwest/istio-eastwest): all pods healthy
✅ Gateway Check: all eastwest gateways programmed
     ✅ istio-eastwest/istio-eastwest available at ab46aa29a49914da789a6d5422aca279-541415195.us-east-2.elb.amazonaws.com
✅ Peers Check: all clusters connected
     ✅ Connected to gloo-mesh-core-docs-mgt via aab8471c7fcfa4a3c82f2d217b015d97-396238517.us-east-1.elb.amazonaws.com
ℹ️  Shared Services Check: no globally shared services found
====== 

✅ Intermediate Certs Compatibility Check: all clusters have compatible intermediate certificates
✅ Network Configuration Check: all network configurations are valid
✅ Stale Workloads Check: skipped (flat network not detected)
  4. Optional: Verify that the istiod control plane for each peered cluster is included in each cluster’s proxy status list.
    istioctl proxy-status --context ${context1}
istioctl proxy-status --context ${context2}
    Example output for cluster1, in which you can verify that the istiod control plane for cluster2 is listed:
    NAME                                               CLUSTER          ISTIOD                      VERSION              SUBSCRIBED TYPES
istio-eastwest-67fd5679dc-fhsxs.istio-eastwest     cluster1         istiod-7b7c9cc4c6-bdm9c     1.28.5-solo-fips     2 (WADS,WDS)
istiod-6bc6765484-5bbhd.istio-system               cluster2         istiod-7b7c9cc4c6-bdm9c     1.28.5-solo-fips     3 (FSDS,SGDS,WDS)
ztunnel-5f8rb.kube-system                          cluster1         istiod-7b7c9cc4c6-bdm9c     1.28.5-solo-fips     2 (WADS,WDS)
ztunnel-f96kh.kube-system                          cluster1         istiod-7b7c9cc4c6-bdm9c     1.28.5-solo-fips     2 (WADS,WDS)
ztunnel-vtj4f.kube-system                          cluster1         istiod-7b7c9cc4c6-bdm9c     1.28.5-solo-fips     2 (WADS,WDS)
Delete previous resources

  1. Now that your multicluster mesh is set up, delete the GatewayLifecycleManager and IstioLifecycleManager managed installations. The steps vary based on whether you created the resources directly, or used the istioInstallations section of the gloo-platform Helm chart.
    1. Delete the GatewayLifecycleManager resources for the gateways that you installed.
      kubectl delete GatewayLifecycleManager istio-eastwestgateway -n gloo-mesh --context ${MGMT_CONTEXT}
kubectl delete GatewayLifecycleManager istio-ingressgateway -n gloo-mesh --context ${MGMT_CONTEXT}
kubectl delete GatewayLifecycleManager istio-egressgateway -n gloo-mesh --context ${MGMT_CONTEXT}
    2. Verify that the gateway resources are removed from the workload clusters before you proceed to the next step.
      kubectl --context ${CLUSTER_CONTEXT} get all -n gloo-mesh-gateways
    3. Delete the IstioLifecycleManager for the istiod control plane. Note that your resource might have a different name.
      kubectl --context ${MGMT_CONTEXT} delete IstioLifecycleManager istiod-control-plane -n gloo-mesh
    1. Get the Helm values of your current Solo Enterprise for Istio installation.
      helm get values gloo-platform -o yaml -n gloo-mesh --kube-context ${MGMT_CONTEXT} > mgmt-plane.yaml
open mgmt-plane.yaml
    2. In the values file, replace the istioInstallations section with the following, and save the file.
      
istioInstallations:
  enabled: false
    3. Upgrade your installation with your updated values file.
      helm upgrade gloo-platform gloo-platform/gloo-platform \
--kube-context ${MGMT_CONTEXT} \
--namespace gloo-mesh \
-f mgmt-plane.yaml \
--version ${GLOO_VERSION}
  2. Send another request to your apps to verify that traffic is still flowing.
    kubectl --context ${CLUSTER_CONTEXT} port-forward -n istio-ingress svc/istio-ingressgateway 8080:80
curl -v http://localhost:8080/productpage
The migration of your service mesh is now complete!
Optional: Validate your multicluster setup

Both before and after you link clusters into a multicluster mesh, you can use the istioctl multicluster check command, along with other observability checks, to verify multiple aspects of multicluster ambient mesh support and status.
You can use the istioctl multicluster check --precheck command to check the individual readiness of each cluster before running istioctl multicluster link to link them in a multicluster mesh, and run it again after linking to confirm that the connections were successful. This command performs checks listed in the following sections, which you can review to understand what each check validates. Additionally, if any of the checks fail, run the command with the --verbose option, and review the following troubleshooting recommendations.
istioctl multicluster check --verbose --contexts="$context1,$context2"
For more information about this command, see the CLI reference.
Incompatible environment variables
Checks whether the ENABLE_PEERING_DISCOVERY=true and optionally K8S_SELECT_WORKLOAD_ENTRIES=true environment variables are set incorrectly or are not supported for multicluster ambient mesh.
Example verbose output:
--- Incompatible Environment Variable Check ---

✅ Incompatible Environment Variable Check: K8S_SELECT_WORKLOAD_ENTRIES is valid ("")
✅ Incompatible Environment Variable Check: ENABLE_PEERING_DISCOVERY is valid ("true")
✅ Incompatible Environment Variable Check: all relevant environment variables are valid
If this check fails, check your environment variables in your istiod configuration, such as by running helm get values --kube-context ${CLUSTER_CONTEXT} istiod -n istio-system -o yaml, and update your configuration.
License validity
Checks whether the license in use by istiod is valid for multicluster ambient mesh. Multicluster capabilities require an Enterprise level license for Solo Enterprise for Istio.
Example verbose output:
--- License Check ---

✅ License Check: license is valid for multicluster
If your license does not support multicluster ambient mesh, contact your Solo account representative.
Pod health
Checks the health of the pods in the cluster. All istiod, ztunnel, and east-west gateway pods across the checked clusters must be healthy and running for the multicluster mesh to function correctly.
Example verbose output:
--- Pod Check (istiod) ---

NAME                        READY     STATUS      RESTARTS     AGE
istiod-6d9cdf88cf-l47tf     1/1       Running     0            10m18s

✅ Pod Check (istiod): all pods healthy


--- Pod Check (ztunnel) ---

NAME              READY     STATUS      RESTARTS     AGE
ztunnel-dvlwk     1/1       Running     0            10m6s

✅ Pod Check (ztunnel): all pods healthy


--- Pod Check (eastwest gateway) ---

NAME                                READY     STATUS      RESTARTS     AGE
istio-eastwest-857b77fc5d-qgnrl     1/1       Running     0            9m33s

✅ Pod Check (eastwest gateway): all pods healthy
To check any unhealthy pods, run the following commands. Consider checking the pod logs, and review Debug Istio.
kubectl get po -n istio-system
kubectl get po -n istio-eastwest
East-west gateway status
Checks the status of the east-west gateways in the cluster. When an east-west gateway is created, the gateway controller creates a Kubernetes service to expose the gateway. Once this service is correctly attached to the gateway and has an address assigned, the east-west gateway has a Programmed status of true.
Example verbose output:
--- Gateway Check ---

Gateway: istio-eastwest
Addresses:
- 172.18.7.110
Status: programmed ✅

✅ Gateway Check: all eastwest gateways programmed
If the Programmed status is not true, an issue might exist with the address allocation for the service. Check the east-west gateway with a command such as kubectl get svc -n istio-eastwest, and verify that your cloud provider can correctly allocate addresses to the service.
Remote peer gateway status
Checks the status of the remote peer gateways in the cluster, which represent the other peered clusters in the multicluster setup. These remote gateways configure the connection between the local cluster’s istiod control plane, and the peered clusters’ remote networks to enable xDS communication between peers. When the initial network connection between istiod and a remote peer is made, the gateway’s gloo.solo.io/PeerConnected status updates to true. Then, when the full xDS sync occurs between peers, the gateway’s gloo.solo.io/PeeringSucceeded status also updates to true. This check ensures that both statuses are true.
Example verbose output:
--- Peers Check ---

Cluster: cluster2
Addresses:
- 172.18.7.130
Conditions:
- Accepted: True
- Programmed: True
- gloo.solo.io/PeerConnected: True
- gloo.solo.io/PeeringSucceeded: True
- gloo.solo.io/PeerDataPlaneProgrammed: True
Status: connected ✅

✅ Peers Check: all clusters connected
If the connection is severed between the peers, the gloo.solo.io/PeerConnected status becomes false. A failed connection between peers can be due to either a misconfiguration in the peering setup, or a network issue blocking port 15008 on the remote cluster, which is the cross-network HBONE port that the east-west gateway listens on. Review the steps you took to link clusters together, such as the steps outlined in the Helm default network guide. Additionally, review any firewall rules or network policies that might block access through port 15008 on the remote cluster.
Intermediate certificate compatibility
Confirms the certificate compatibility between peered clusters. This check reads the root-cert.pem from the istio-ca-root-cert configmap in the istio-system namespace, and uses x509 certificate validation to confirm the root cert is compatible with all of the clusters’ ca-cert.pem intermediate certificate chains from the cacerts secret.
Example verbose output:
--- Intermediate Certs Compatibility Check ---

ℹ  Intermediate Certs Compatibility Check: cluster cluster1 root certificate SHA256 sum: 6d18f32e134824c158d97f32618657c45d5a83839f838ada751757139481537e
ℹ  Intermediate Certs Compatibility Check: cluster cluster2 root certificate SHA256 sum: 6d18f32e134824c158d97f32618657c45d5a83839f838ada751757139481537e
✅ Intermediate Certs Compatibility Check: cluster cluster1 has compatible intermediate certificates with cluster cluster2 
✅ Intermediate Certs Compatibility Check: cluster cluster2 has compatible intermediate certificates with cluster cluster1 
✅ Intermediate Certs Compatibility Check: all clusters have compatible intermediate certificates
If this check fails because the root certs are not valid for each peered clusters’ intermediate certificate chain, you can check the istiod logs for TLS errors when attempting to communicate with a peered cluster, such as the following:
2025-12-04T22:09:22.474517Z     warn    deltaadsc       disconnected, retrying in 24.735483751s: delta stream: rpc error: code = Unavailable desc = connection error: desc = "error reading server preface: remote error: tls: unknown certificate authority"       target=peering-cluster2
Ensure each cluster has a cacerts secret in the istio-system namespace. To regenerate invalid certificates for each cluster, follow the example steps in Create a shared root of trust.
Network configuration
Confirms the network configuration of the multicluster mesh. For multicluster peering setups that do not use a flat network topology, each cluster must occupy a unique network. The network name must be defined with the label topology.istio.io/network and set on both the istio-system namespace and the istio-eastwest gateway resource. The same network name must also be set as the NETWORK environment variable on the ztunnel daemonset. Each remote gateway that represents that cluster must have the topology.istio.io/network label equal to the network of the remote cluster.
Example verbose output:
--- Network Configuration Check ---

✅ Cluster cluster1 has network: cluster1
✅ Eastwest gateway istio-eastwest/istio-eastwest has correct network label: cluster1
✅ Cluster cluster2 has network: cluster2
✅ Eastwest gateway istio-eastwest/istio-eastwest has correct network label: cluster2
✅ Remote gateway istio-eastwest/istio-remote-peer-cluster2 references network cluster2 (clusters: [cluster2])
✅ Remote gateway istio-eastwest/istio-remote-peer-cluster1 references network cluster1 (clusters: [cluster1])
✅ Network Configuration Check: all network configurations are valid
Mismatched network identities cause errors in cross-cluster communication, which leads to error logs in ztunnel pods that indicate a network timeout on the outbound communication. Notably, the destination address on these errors is a 240.X.X.X address, instead of the correct remote peer gateway address. You can run kubectl logs -l app=ztunnel -n istio-system --tail=10 --context ${CLUSTER_CONTEXT} | grep -iE "error|warn" to review logs such as the following:
2025-11-18T16:14:53.490573Z     error   access  connection complete     src.addr=240.0.2.27:46802 src.workload="ratings-v1-5dc79b6bcd-zm8v6" src.namespace="bookinfo" src.identity="spiffe://cluster.local/ns/bookinfo/sa/bookinfo-ratings" dst.addr=240.0.9.43:15008 dst.hbone_addr=240.0.9.43:9080 dst.service="productpage.bookinfo.mesh.internal" dst.workload="autogenflat.portfolio1-soloiopoc-cluster1.bookinfo.productpage-v1-54bb874995-hblwp.ee508601917c" dst.namespace="bookinfo" dst.identity="spiffe://cluster.local/ns/bookinfo/sa/bookinfo-productpage" direction="outbound" bytes_sent=0 bytes_recv=0 duration="10001ms" error="connection timed out, maybe a NetworkPolicy is blocking HBONE port 15008: deadline has elapsed"
To troubleshoot these issues, be sure that you use unique network names to represent each cluster, and that you correctly labeled the cluster’s istio-system namespace with that network name, such as by running kubectl label namespace istio-system --context ${CLUSTER_CONTEXT} topology.istio.io/network=${CLUSTER_NAME}. You can also relabel the east-west gateway in the cluster, and the remote peer gateways in other clusters that represent this cluster.
Stale workload entries
In flat network setups, checks for any outdated workload entries that must be removed from the multicluster mesh. Stale workload entries might exist from pods that were deleted, but the autogenerated entries for those workloads were not correctly cleaned up. If you do not use a flat network topology, no autogenerated workload entries exist to be validated, and this check can be ignored.
Example verbose output for a non-flat network setup:
--- Stale Workloads Check ---

⚠  Stale Workloads Check: no autogenflat workload entries found
If you use a flat network topology, and this check fails with stale workload entries, run kubectl get workloadentries -n istio-system | grep autogenflat to list the autogenerated workload entries in the remote cluster, and compare the list to the output of kubectl get pods in the source cluster for those workloads. You can safely manually delete the stale workload entries in the remote cluster for pods that no longer exist in the source cluster, such as by running kubectl get workloadentries -n istio-system <entry_name>.
Next

  • Launch the Gloo UI to review the Istio insights that were captured for your service mesh setup. Solo Enterprise for Istio comes with an insights engine that automatically analyzes your Istio setups for health issues. These issues are displayed in the UI along with recommendations to harden your Istio setups. The insights give you a checklist to address issues that might otherwise be hard to detect across your environment. For more information, see Insights.
  • When it’s time to upgrade your service mesh, you can perform a safe in-place upgrade by using the Gloo Operator.
Gloo OperatorHelm