For the complete documentation index, see llms.txt. Markdown versions of all docs pages are available by appending .md to any docs URL.
In-place upgrades with Helm
Upgrade your ambient control and data plane components in your Solo Enterprise for Istio cluster.
Considerations
Before you upgrade your ambient components, review the following limitations and recommendations.
Revision and canary upgrade limitations
The upgrade guides in this documentation show you how to perform in-place upgrades for your Istio components, which is the recommended upgrade strategy.
If you use Kubernetes Gateway API resources to manage ingress traffic, a tag flip during a canary upgrade can push empty xDS configuration to Gateway pods and cause a brief traffic blackhole. For prerequisites and a zero-downtime procedure, see Canary upgrades with the Gateway API.
CNI upgrades
In an ambient installation, the istio-cni pods are enhanced for safe in-place upgrades. When an istio-cni pod shuts down, it checks whether the istio-cni-node DaemonSet still exists and still targets the node. If so, the shutdown is treated as an upgrade, and istio-cni leaves its CNI plug-in in place on the node. The plug-in stalls any newly scheduled ambient pods on that node, and does not allow the pods to spin up until the replacement istio-cni is up and running.
The istio-cni pods read their settings from the istio-cni-config ConfigMap only at startup. In the following cases, the running pods keep the old settings until you restart the DaemonSet:
- You edit the ConfigMap directly instead of through Helm.
- You run
helm upgradewith changed values only, and theistio-cni-nodeDaemonSet pod template does not have achecksum/cni-configannotation. To check, runkubectl get ds istio-cni-node -n istio-system -o jsonpath='{.spec.template.metadata.annotations}'.
To apply the new settings, restart the DaemonSet.
kubectl rollout restart ds/istio-cni-node -n istio-systemztunnel upgrades
When Kubernetes restarts ztunnel pods during an upgrade, long-lived connections on the node are dropped. You can reduce this impact by tuning the termination grace period and by cordoning nodes or using blue/green node pools to limit the number of active pods being upgraded simultaneously. For configuration steps, see Tune ztunnel performance.
Migration required for platforms with pinned Gateway API
Solo Enterprise for Istio 1.30 no longer watches XListenerSet (gateway.networking.x-k8s.io/v1alpha1). The Gateway API promoted this experimental resource to ListenerSet (gateway.networking.k8s.io/v1) in version 1.5. If you run Istio on a platform that pins the Gateway API CRD below 1.5, such as OpenShift, and you use XListenerSet, existing listeners stop serving traffic silently after the upgrade.
Solo Enterprise for Istio 1.30.3-patch0 and later include EnterpriseListenerSet (enterprise.solo.io/v1alpha1) as a replacement. EnterpriseListenerSet provides the same listener-attachment behavior as ListenerSet and does not require the listenersets.gateway.networking.k8s.io CRD, so it works on platforms that pin Gateway API below 1.5.
You can stage the migration before upgrading to reduce the outage window. Install the EnterpriseListenerSet CRD and create your resources while still on 1.29, because Istio 1.29 ignores them and they sit inert until the upgraded istiod starts. When you run helm upgrade, include SOLO_ENABLE_ENTERPRISE_LISTENER_SET=true in your Helm values.
For the full upgrade steps, follow EnterpriseListenerSet.
Before you begin
Verify that the minor version of the Solo distribution of Istio that you want to upgrade to is tested and supported for your Solo Enterprise for Istio version. For Istio 1.29 and later, you can check available patch builds in the
us-docker.pkg.dev/soloio-img/istiorepo. For Istio version 1.28 and earlier, you can get the minor version repo URL from the Istio images built by Solo.io support article, and check the patch version builds in that repo.Check the Solo Enterprise for Istio release notes for changes from the previous version.
Caution
Before you proceed with the upgrade, make sure that you review the breaking changes that were introduced in this release and the impact that they have on your current environment. For more information, see the Breaking changes.Check the Istio release notes for the upgrade version to prepare for any breaking changes.
Be sure to review the following known Istio version restrictions.
Warning
- To use the Solo distribution of Istio 1.30 for multicluster setups in which waypoint proxies are deployed, use patch version 1.30.2 or later. Patch versions 1.30.0 and 1.30.1 contain a bug in which the peering controller does not create a remote `ServiceEntry` for services whose pod selector matches waypoint proxy pods.
- If you run Istio on a platform that pins the Kubernetes Gateway API below version 1.5, such as OpenShift, and you use
XListenerSet(gateway.networking.x-k8s.io/v1alpha1), Solo Enterprise for Istio 1.30 and later no longer watches this resource. ExistingXListenerSetobjects are silently ignored and their listeners stop serving traffic. UseEnterpriseListenerSet(enterprise.solo.io/v1alpha1) as a replacement, which requires Solo Enterprise for Istio 1.30.3-patch0 or later and theSOLO_ENABLE_ENTERPRISE_LISTENER_SET=trueenvironment variable on istiod. For setup and migration instructions, see EnterpriseListenerSet. - In the Solo distribution of Istio 1.30 and 1.31,
WorkloadEntryresources created by autoregistration might not carry thenetworking.istio.io/tunnel: httplabel. When that label is absent, ambient-enrolled pods send connections to the VM as plaintext instead of over HBONE. Because plaintext connections carry no mTLS client identity, anyAuthorizationPolicythat usessource.principalsto control inbound access to the VM always returns 403. As a workaround, manually label the gatewayWorkloadEntrywithnetworking.istio.io/tunnel: httpto force HBONE, or replacesource.principalsconditions withsource.ipBlocksuntil a patch release is available.
Single cluster
Upgrade istioctl
Save the details for the version of the Solo distribution of Istio that you want to upgrade to.
- Save the Solo distribution of Istio patch version and tag.
export ISTIO_VERSION=1.30.8 # Change the tags as needed export ISTIO_IMAGE=${ISTIO_VERSION}-solo - Save the image and Helm repository information for the Solo distribution of Istio.
- Istio 1.29 and later:
export REPO=us-docker.pkg.dev/soloio-img/istio export HELM_REPO=us-docker.pkg.dev/soloio-img/istio-helm - Istio 1.28 and earlier:
Save the repo key for the minor version of the Solo distribution of Istio that you want to install. This is the 12-character hash at the end of the repo URL
us-docker.pkg.dev/gloo-mesh/istio-<repo-key>, which you can find in the Istio images built by Solo.io support article.# 12-character hash at the end of the minor version repo URL export REPO_KEY=<repo_key> export REPO=us-docker.pkg.dev/gloo-mesh/istio-${REPO_KEY} export HELM_REPO=us-docker.pkg.dev/gloo-mesh/istio-helm-${REPO_KEY}
- Istio 1.29 and later:
- Save the Solo distribution of Istio patch version and tag.
Upgrade your
istioctlCLI client to the new version.curl -L https://istio.io/downloadIstio | ISTIO_VERSION=${ISTIO_VERSION} sh - cd istio-${ISTIO_VERSION} export PATH=$PWD/bin:$PATH
Upgrade CRDs and istiod
Upgrade the custom resources of the Kubernetes Gateway API to the latest supported version, 1.5.0.
kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.5.0/standard-install.yamlUpgrade the Istio CRDs to the new version.
helm get values istio-base -n istio-system -o yaml > istio-base.yaml helm upgrade istio-base oci://${HELM_REPO}/base \ --namespace istio-system \ --version ${ISTIO_IMAGE} \ -f istio-base.yamlNote
If you see an error such as
Error: UPGRADE FAILED: Rendered manifests contain a resource that already exists, see the community Istio docs.Get the current values for the istiod Helm release in your cluster.
helm get values istiod -n istio-system -o yaml > istiod.yaml open istiod.yamlMake edits to the istiod Helm values, and save the file. If you update the Istio minor version, such as in the
global.tagfield, be sure to also update the value of thehubfield to the repo for the correct version of the Solo distribution of Istio.Upgrade your Helm release with the updated values.
helm upgrade istiod oci://${HELM_REPO}/istiod \ -n istio-system \ --version ${ISTIO_IMAGE} \ -f istiod.yamlVerify that the istiod pods are successfully restarted. Note that it might take a few seconds for the pods to become available.
kubectl get pods -n istio-system | grep istiodExample output:
istiod-b84c55cff-tllfr 1/1 Running 0 58s
Upgrade the CNI and ztunnel
Note
If your changes include upgrading the Istio version of the components, be sure to upgrade the istiod control plane before you upgrade the CNI and ztunnel components. Otherwise, these components might have an outdated image.
For the component that you want to update, get the current values for its Helm release in your cluster.
helm get values istio-cni -n istio-system -o yaml > cni.yaml open cni.yamlhelm get values ztunnel -n istio-system -o yaml > ztunnel.yaml open ztunnel.yamlMake edits to the Helm values of the components that you want to upgrade, and save the files. If you update the Istio minor version, such as in
tagfields, be sure to also update the value of thehubfield to the repo for the correct version of the Solo distribution of Istio.Upgrade your Helm releases with the updated values.
helm upgrade istio-cni oci://${HELM_REPO}/cni -n istio-system --version ${ISTIO_IMAGE} -f cni.yamlhelm upgrade ztunnel oci://${HELM_REPO}/ztunnel -n istio-system --version ${ISTIO_IMAGE} -f ztunnel.yamlVerify that the ztunnel and Istio CNI pods are successfully restarted. Note that it might take a few seconds for the pods to become available.
kubectl get pods -n istio-systemExample output:
istiod-85c4dfd97f-mncj5 1/1 Running 0 42s istio-cni-node-pr5rl 1/1 Running 0 42s istio-cni-node-pvmx2 1/1 Running 0 42s istio-cni-node-lcrcd 1/1 Running 0 42s ztunnel-tvtzn 1/1 Running 0 40s ztunnel-vtpjm 1/1 Running 0 40s ztunnel-hllxg 1/1 Running 0 40s
Multicluster
Upgrade the multicluster ambient mesh in your multicluster setup.
Upgrade istioctl
Set environment variables for the Solo distribution of Istio that you want to upgrade to.
- Istio 1.29 and later:
# Solo distribution of Istio patch version # in the format 1.x.x, with no tags export ISTIO_VERSION=1.30.8 export ISTIO_IMAGE=${ISTIO_VERSION}-solo export REPO=us-docker.pkg.dev/soloio-img/istio export HELM_REPO=us-docker.pkg.dev/soloio-img/istio-helm - Istio 1.28 and earlier: Find these values in the Istio images built by Solo.io support article.
# Solo distribution of Istio patch version # in the format 1.x.x, with no tags export ISTIO_VERSION=<istio_version> # Repo key for the minor version of the Solo distribution of Istio # This is the 12-character hash at the end of the repo URL: 'us-docker.pkg.dev/gloo-mesh/istio-<repo-key>' export REPO_KEY=<repo_key> export HELM_REPO=us-docker.pkg.dev/gloo-mesh/istio-helm-${REPO_KEY} export ISTIO_IMAGE=${ISTIO_VERSION}-solo export REPO=us-docker.pkg.dev/gloo-mesh/istio-${REPO_KEY}
- Istio 1.29 and later:
Upgrade your
istioctlCLI client to the new version. This script automatically detects your OS and architecture, downloads the appropriate Solo distribution of Istio binary, and verifies the installation.bash <(curl -sSfL https://raw.githubusercontent.com/solo-io/doc-examples/main/istio/install-istioctl.sh) export PATH=${HOME}/.istioctl/bin:${PATH}
Upgrade CRDs and istiod
Save the kubeconfig context of a cluster where you want to upgrade Istio in an environment variable. Each time you repeat the steps in this guide, you change this variables to the next cluster’s context.
export ctx=<cluster-context>Upgrade the custom resources of the Kubernetes Gateway API to the latest supported version, 1.5.0.
kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.5.0/standard-install.yaml --context ${ctx}Upgrade the Istio CRDs to the new version.
helm get values --kube-context ${ctx} istio-base -n istio-system -o yaml > istio-base.yaml helm upgrade istio-base oci://${HELM_REPO}/base \ --namespace istio-system \ --kube-context ${ctx} \ --version ${ISTIO_IMAGE} \ -f istio-base.yamlNote
If you see an error such as
Error: UPGRADE FAILED: Rendered manifests contain a resource that already exists, see the community Istio docs.Get the current values for the istiod Helm release in your cluster.
helm get values --kube-context ${ctx} istiod -n istio-system -o yaml > istiod.yaml open istiod.yamlMake edits to the istiod Helm values, and save the file. If you update the Istio minor version, such as in the
global.tagfield, be sure to also update the value of thehubfield to the repo for the correct version of the Solo distribution of Istio.Upgrade your Helm release with the updated values.
helm upgrade istiod oci://${HELM_REPO}/istiod \ -n istio-system \ --version ${ISTIO_IMAGE} \ --kube-context ${ctx} \ -f istiod.yamlVerify that the istiod pods are successfully restarted. Note that it might take a few seconds for the pods to become available.
kubectl get pods --context ${ctx} -n istio-system | grep istiodExample output:
istiod-b84c55cff-tllfr 1/1 Running 0 58s
Upgrade the CNI and ztunnel
Note
If your changes include upgrading the Istio version of the components, be sure to upgrade the istiod control plane before you upgrade the CNI and ztunnel components. Otherwise, these components might have an outdated image.
For the component that you want to update, get the current values for its Helm release in your cluster.
helm get values istio-cni --kube-context ${ctx} -n istio-system -o yaml > cni.yaml open cni.yaml helm get values ztunnel --kube-context ${ctx} -n istio-system -o yaml > ztunnel.yaml open ztunnel.yamlMake edits to the Helm values of the components that you want to upgrade, and save the files. If you update the Istio minor version, such as in
tagfields, be sure to also update the value of thehubfield to the repo for the correct version of the Solo distribution of Istio.Upgrade your Helm releases with the updated values.
helm upgrade istio-cni oci://${HELM_REPO}/cni --kube-context ${ctx} -n istio-system --version ${ISTIO_IMAGE} -f cni.yaml helm upgrade ztunnel oci://${HELM_REPO}/ztunnel --kube-context ${ctx} -n istio-system --version ${ISTIO_IMAGE} -f ztunnel.yamlVerify that the ztunnel and Istio CNI pods are successfully restarted. Note that it might take a few seconds for the pods to become available.
kubectl get pods --context ${ctx} -n istio-systemExample output:
istiod-85c4dfd97f-mncj5 1/1 Running 0 42s istio-cni-node-pr5rl 1/1 Running 0 42s istio-cni-node-pvmx2 1/1 Running 0 42s istio-cni-node-lcrcd 1/1 Running 0 42s ztunnel-tvtzn 1/1 Running 0 40s ztunnel-vtpjm 1/1 Running 0 40s ztunnel-hllxg 1/1 Running 0 40s
Repeat for each cluster
Repeat Upgrade CRDs and istiod and Upgrade the CNI and ztunnel for each cluster in your setup. Note that no multicluster components, such as the east-west gateway or global service entries, require upgrades when you update the version of your other control and data plane components.