Navigation :

Upgrade Gloo Mesh-managed Istio

Use Gloo Mesh to upgrade your Gloo Mesh-managed Istio instalations in your workload clusters.

About

After you use the lifecycle manager to deploy Istio installations, changes that you make to the IstioLifecycleManager and GatewayLifecycleManager resources are automatically propagated to your Istio installations. In some cases, these changes might require updates to your control plane or gateway deployments. Depending on the type of change, you apply updates to the installations either in-place or via a canary upgrade. Note that you can force Gloo to always use canary upgrades by setting the spec.upgradeStrategy.alwaysUseCanaryUpgrade boolean value to true in the IstioLifecycleManager and GatewayLifecycleManager resources.

In-place upgrades

In an in-place upgrade, Gloo upgrades your existing control plane or gateway installations. In-place upgrades are triggered when you change one of the following fields:

Patch version in the tag field of the istioOperatorSpec
meshConfig values in the istioOperatorSpec

Canary upgrades (recommended)

In a canary upgrade, you install another Istio installation (canary) alongside your active installation. Note that during a canary upgrade, the validating admissions webhook is enabled only for the canary installation to prevent issues that occur when multiple webhooks are enabled.

Perform a canary upgrade when you change one of the following fields:

Minor version in the tag field of the istioOperatorSpec
hub value in the istioOperatorSpec, such as switching from the default Istio build to a Solo build
components, profile, values, or namespace in the istioOperatorSpec

Performing an in-place upgrade

Update the patch version of Istio or update meshConfig values of the Istio installations.

Get the names of your managed Istio installation resources.

kubectl get IstioLifecycleManager -n gloo-mesh --context $MGMT_CONTEXT
kubectl get GatewayLifecycleManager -n gloo-mesh --context $MGMT_CONTEXT

Upgrade your istiod control plane by editing the IstioLifecycleManager resource. For example, you might update the patch version of Istio by changing the value of istioOperatorSpec.tag. After you save and close the editor, Gloo starts an in-place upgrade of the istiod control planes.
```
kubectl edit IstioLifecycleManager -n gloo-mesh --context $MGMT_CONTEXT <istiod-installation-name>
```
If you created GatewayLifecycleManager resources for ingress or east-west gateways, you can upgrade your gateways by editing each resource. For example, if you updated the patch version of the control plane, you can also update your gateway patch versions by changing the value of istioOperatorSpec.tag. After you save and close the editor, Gloo starts an in-place upgrade of the gateways.
```
kubectl edit GatewayLifecycleManager -n gloo-mesh --context $MGMT_CONTEXT <ingress-gateway-installation-name>
```
```
kubectl edit GatewayLifecycleManager -n gloo-mesh --context $MGMT_CONTEXT <eastwest-gateway-installation-name>
```

Performing a canary upgrade

Update the minor version of Istio, the repository of the Istio image, or the components, profile, values, or namespace of the Istio installations.

Get the names of your managed Istio installation resources.

kubectl get IstioLifecycleManager -n gloo-mesh --context $MGMT_CONTEXT
kubectl get GatewayLifecycleManager -n gloo-mesh --context $MGMT_CONTEXT

Edit the IstioLifecycleManager resource for the istiod control plane to add another installation entry for the canary revision. For the canary revision, be sure to set defaultRevision to false so that only the existing revision continues to run.

kubectl edit IstioLifecycleManager -n gloo-mesh --context $MGMT_CONTEXT <installation-name>

For example, if you upgrade to Istio 1.16, add the following 1-16 canary revision as a new entry in the installations section:

apiVersion: admin.gloo.solo.io/v2
kind: IstioLifecycleManager
metadata:
  name: istiod-control-plane
  namespace: gloo-mesh
spec:
  installations:
    # Existing revision
    - revision: 1-15
      clusters:
      - name: cluster-1
        # Keep this field set to TRUE so that only the existing revision continues to run
        defaultRevision: true
      - name: cluster-2
        defaultRevision: true
      istioOperatorSpec:
        profile: minimal
        ...
    # Canary revision
    - revision: 1-16
      clusters:
      - name: cluster-1
        # Set this field set to FALSE so that only the existing revision continues to run
        defaultRevision: false
      - name: cluster-2
        defaultRevision: false
      istioOperatorSpec:
        profile: minimal
        ...

Updating the minor version of Istio? In your canary revision section, be sure to update both the repo key in the hub field, and the Istio version in the tag field. You can get the repo key for the Istio version that you want to install from the Istio images built by Solo.io support article.

If you created GatewayLifecycleManager resources for east-west or ingress gateways, edit those resources to add another installation entry for the canary revision. Be sure to set activeGateway to false so that only the existing revision continues to run.

kubectl edit GatewayLifecycleManager -n gloo-mesh --context $MGMT_CONTEXT <installation-name>

For example, if you upgrade to Istio 1.16, add the following 1-16 canary revision as a new entry in the installations section:

apiVersion: admin.gloo.solo.io/v2
kind: GatewayLifecycleManager
metadata:
  name: istio-ingressgateway
  namespace: gloo-mesh
spec:
  installations:
    # Existing revision
    - gatewayRevision: 1-15
      clusters:
      - name: cluster-1
        # Keep this field set to TRUE so that only the existing revision continues to run
        activeGateway: true
      - name: cluster-2
        activeGateway: true
      istioOperatorSpec:
        profile: empty
        ...
    # Canary revision
    - gatewayRevision: 1-16
      clusters:
      - name: cluster-1
        # Set this field set to FALSE so that only the existing revision continues to run
        activeGateway: false
      - name: cluster-2
        activeGateway: false
      istioOperatorSpec:
        profile: empty
        ...

For most use cases, you can set the revision for the IstioLifecycleManager and the gatewayRevision for the GatewayLifecycleManager to the same version. However, gateway installations can point to any istiod control plane revision by using the controlPlaneRevision field. For simplicity, if you do not specify controlPlaneRevision, the gateway installation uses a control plane with the same revision as itself.

Verify that Istio resources for the canary installation are created. For example, if you updated the Istio minor version to 1-16, verify that resources are created in the gm-iop-1-16 namespace, and that resources for 1-16 are created alongside the existing resources for the previous version in the istio-system and gloo-mesh-gateways namespaces. Note that the gateway load balancers for the canary revision contain the revision in the name, such as istio-ingressgateway-1-16.
- Single cluster setup
- Multicluster setup
```
kubectl get all -n gm-iop-1-16
kubectl get all -n istio-system
kubectl get all -n gloo-mesh-gateways
```
```
kubectl get all -n gm-iop-1-16 --context $REMOTE_CONTEXT1
kubectl get all -n istio-system --context $REMOTE_CONTEXT1
kubectl get all -n gloo-mesh-gateways --context $REMOTE_CONTEXT1
```
```
kubectl get all -n gm-iop-1-16 --context $REMOTE_CONTEXT2
kubectl get all -n istio-system --context $REMOTE_CONTEXT2
kubectl get all -n gloo-mesh-gateways --context $REMOTE_CONTEXT2
```

After performing any necessary testing, switch to the new istiod control plane revision by changing defaultRevision to false for the old revision and to true for the new revision.

kubectl edit IstioLifecycleManager -n gloo-mesh --context $MGMT_CONTEXT istiod-control-plane

Example:

apiVersion: admin.gloo.solo.io/v2
kind: IstioLifecycleManager
metadata:
  name: istiod-control-plane
  namespace: gloo-mesh
spec:
  installations:
    # Old revision
    - revision: 1-15
      clusters:
      - name: cluster-1
        # Set this field set to FALSE
        defaultRevision: false
      - name: cluster-2
        defaultRevision: false
      istioOperatorSpec:
        profile: minimal
        ...
    # New revision
    - revision: 1-16
      clusters:
      - name: cluster-1
        # Set this field set to TRUE
        defaultRevision: true
      - name: cluster-2
        defaultRevision: true
      istioOperatorSpec:
        profile: minimal
        ...

Upgrade your apps’ Istio sidecars.

For any namespace where you run Istio-managed apps, change the label to use the new revision. For example, you might update the bookinfo namespace.
If you did not previously use revision labels for your apps, you can upgrade your application's sidecars by running ‘kubectl label ns bookinfo istio-injection-’ and ‘kubectl label ns bookinfo istio.io/rev=$REVISION’.
- Single cluster setup
- Multicluster setup
```
kubectl label ns bookinfo istio.io/rev=$REVISION --overwrite
```
```
kubectl label ns bookinfo istio.io/rev=$REVISION --overwrite --context $REMOTE_CONTEXT1
kubectl label ns bookinfo istio.io/rev=$REVISION --overwrite --context $REMOTE_CONTEXT2
```

Update any Istio-managed apps by rolling out restarts. The Istio sidecars for each microservice are updated to use the new Istio version. Make sure that you only restart one microservice at a time. For example, in the following commands to update the Bookinfo microservices, 20 seconds elapse between each restart to ensure that the pods have time to start running.

Single cluster setup
Multicluster setup

kubectl rollout restart deployment -n bookinfo details-v1
sleep 20s
kubectl rollout restart deployment -n bookinfo productpage-v1
sleep 20s
kubectl rollout restart deployment -n bookinfo reviews-v1
sleep 20s
kubectl rollout restart deployment -n bookinfo reviews-v2
sleep 20s
kubectl rollout restart deployment -n bookinfo reviews-v3
sleep 20s
kubectl rollout restart deployment -n bookinfo ratings-v1
sleep 20s

kubectl rollout restart deployment -n bookinfo details-v1 --context $REMOTE_CONTEXT1
sleep 20s
kubectl rollout restart deployment -n bookinfo ratings-v1 --context $REMOTE_CONTEXT1
sleep 20s
kubectl rollout restart deployment -n bookinfo productpage-v1 --context $REMOTE_CONTEXT1
sleep 20s
kubectl rollout restart deployment -n bookinfo reviews-v1 --context $REMOTE_CONTEXT1
sleep 20s
kubectl rollout restart deployment -n bookinfo reviews-v2 --context $REMOTE_CONTEXT1
sleep 20s
kubectl rollout restart deployment -n bookinfo reviews-v3 --context $REMOTE_CONTEXT2
sleep 20s
kubectl rollout restart deployment -n bookinfo ratings-v1 --context $REMOTE_CONTEXT2
sleep 20s

Verify that your workloads and new gateways point to the new revision.

Single cluster setup
Multicluster setup

istioctl proxy-status

istioctl proxy-status --context $REMOTE_CONTEXT1

Example output:

NAME                                                              CLUSTER     ...     ISTIOD                           VERSION
details-v1-7b6df9d8c8-s6kg5.bookinfo                              cluster-1    ...     istiod-1-16-7c8f6fd4c4-m9k9t     1.16.2-solo
istio-eastwestgateway-1-16-bdc4fd65f-ftmz9.gloo-mesh-gateways     cluster-1    ...     istiod-1-16-6495985689-rkwwd     1.16.2-solo
productpage-v1-bb494b7d7-xbtxr.bookinfo                           cluster-1    ...     istiod-1-16-7c8f6fd4c4-m9k9t     1.16.2-solo
ratings-v1-55b478cfb6-wv2m5.bookinfo                              cluster-1    ...     istiod-1-16-7c8f6fd4c4-m9k9t     1.16.2-solo
reviews-v1-6dfcc9fc7d-7k6qh.bookinfo                              cluster-1    ...     istiod-1-16-7c8f6fd4c4-m9k9t     1.16.2-solo
reviews-v2-7dddd799b5-m5n2z.bookinfo                              cluster-1    ...     istiod-1-16-7c8f6fd4c4-m9k9t     1.16.2-solo

Switch to any new gateways by changing activeGateway to false for the old revision and to true for the new revision.

kubectl edit GatewayLifecycleManager -n gloo-mesh --context $MGMT_CONTEXT <installation-name>

Example:

apiVersion: admin.gloo.solo.io/v2
kind: GatewayLifecycleManager
metadata:
  name: istio-ingressgateway
  namespace: gloo-mesh
spec:
  installations:
    # Old revision
    - gatewayRevision: 1-15
      clusters:
      - name: cluster-1
        # Keep this field set to FALSE
        activeGateway: false
      - name: cluster-2
        activeGateway: false
      istioOperatorSpec:
        profile: empty
        ...
    # New revision
    - gatewayRevision: 1-16
      clusters:
      - name: cluster-1
        # Set this field set to TRUE
        activeGateway: true
      - name: cluster-2
        activeGateway: ture
      istioOperatorSpec:
        profile: empty
        ...

New load balancers are created for the canary gateways. To instead change the control plane revision in use by the existing gateway load balancers, you can set the istio.io/rev label on the gateway deployment, which triggers a rolling restart.

Verify that the active gateways for the new revision are created, which do not have the revision appended to the name. Note that gateways for the inactive revision that you previously ran also exist in the namespace, in the case that a rollback is required.

kubectl get all -n gloo-mesh-gateways --context $REMOTE_CONTEXT1
kubectl get all -n gloo-mesh-gateways --context $REMOTE_CONTEXT2

Example output, in which the active gateway (istio-ingressgateway) for the new revision and inactive gateway (such as istio-ingressgateway-1-15) for the old revision are created:

NAME                            TYPE           CLUSTER-IP     EXTERNAL-IP     PORT(S)     
istio-ingressgateway            LoadBalancer   10.44.4.140   34.150.235.221   15021:31321/TCP,80:32525/TCP,443:31826/TCP   48s                                                 AGE
istio-ingressgateway-1-15       LoadBalancer   10.56.15.36    34.145.163.61   15021:31936/TCP,80:30196/TCP,443:32286/TCP,15443:31851/TCP   45s

To uninstall the previous installation, or if you need to uninstall the canary installation, you can edit the IstioLifecycleManager and GatewayLifecycleManager resources to remove the revision's entry from the installations list.