In Gloo Mesh Gateway 2.3 and later, the gloo-mesh-enterpise, gloo-mesh-agent, and other included Helm charts are considered legacy. If you installed Gloo Mesh Gateway by using these legacy Helm charts, or if you used meshctl version 2.2 or earlier to install Gloo Mesh Gateway, you can migrate your existing installation to the new gloo-platform Helm chart by using the meshctl migrate helm command.

gloo-platform chart overview

In Gloo version 2.3 and later, all Gloo components are available in a single Helm chart, gloo-platform. Additionally, the custom resource definitions (CRDs) that are required by Gloo controllers are maintained by the gloo-platform-crds Helm chart.

When you migrate to the gloo-platform chart, you unlock new key benefits such as the following:

  • ✅ Because all Gloo components are consolidated into one chart, the Gloo Mesh Gateway installation and upgrade processes are now quicker and simpler than ever.
  • ✅ The setup process now includes basic install profiles, which provide pre-defined values files that help you quickly get your Gloo Gateway installation up and running.
  • ✅ Because the Gloo CRDs are bundled into the new gloo-platform-crds Helm chart, you can apply any new and updated CRDs by upgrading this Helm chart, instead of manually applying CRDs.

Within the gloo-platform chart, you can find the configuration options for all components in the following sections. To review these sections before you migrate, you can run helm show values gloo-platform/gloo-platform --version v2.3.23.

Component sectionDescription
commonCommon values shared across components. When applicable, these can be overridden in specific components.
demoDemo-specific features that improve quick setups. Do not use in production.
experimentalExperimental features for Gloo Mesh Gateway. Disabled by default. Do not use in production.
extAuthServiceConfiguration for the Gloo external authentication service.
glooAgentConfiguration for the Gloo agent.
glooAnalyzerConfiguration for the Gloo analyzer, which gathers data on Gloo and Istio components.
glooInsightsEngineConfiguration for the Gloo insights engine, which creates Solo insights.
glooMgmtServerConfiguration for the Gloo management server.
glooNetworkGloo Network agent configuration options.
glooPortalServerConfiguration for the Gloo Portal server deployment.
glooUiConfiguration for the Gloo UI.
istioInstallationsConfiguration for deploying managed Istio control plane and gateway installations by using the Istio lifecycle manager.
licensingGloo product licenses.
prometheusHelm values for configuring Prometheus. See the Prometheus Helm chart for the complete set of values.
rateLimiterConfiguration for the Gloo rate limiting service.
redisConfiguration for the default Redis instance.
telemetryCollectorConfiguration for the Gloo telemetry collector agents. See the OpenTelemetry Helm chart for the complete set of values.
telemetryCollectorCustomizationOptional customization for the Gloo telemetry collector agents.
telemetryGatewayConfiguration for the Gloo telemetry gateway. See the OpenTelemetry Helm chart for the complete set of values.
telemetryGatewayCustomizationOptional customization for the Gloo telemetry gateway.

Migration process

To migrate your existing Gloo Mesh Gateway installation to the new Helm chart, you can use the meshctl migrate helm command. This command generates a Helm install command that includes a values file for the new gloo-platform-crds chart, and one or more Helm upgrade commands that include values files for the new gloo-platform chart, based on the values of your existing Helm releases.

For example, you might have a multi-cluster Gloo Mesh Gateway setup. The management server components are maintained by the legacy gloo-mesh-enterprise Helm chart, with the release named gloo-mgmt. The agent components are maintained by the legacy gloo-mesh-agent Helm chart, with the release named gloo-agent.

When you run meshctl migrate helm for the management cluster, the command generates:

  • A Helm install command for a gloo-platform-crds Helm chart release. This command includes a values file that contains the Gloo CRDs for the Gloo version you want to use.
  • A Helm upgrade command for your existing gloo-mgmt release. This command includes a values file that maps your values for the management server components from the legacy chart fields to the new fields in the gloo-platform chart.

When you run meshctl migrate helm for a workload cluster, the command generates:

  • A Helm install command for a gloo-platform-crds Helm chart release. This command includes a values file that contains the Gloo CRDs for the Gloo version you want to use.
  • A Helm upgrade command for your existing gloo-agent release. This command includes a values file that maps your values for the agent components from the legacy chart fields to the new fields in the gloo-platform chart.

After you run the generated commands, you can verify that your Gloo Mesh Gateway installation is running as expected. If you need to make changes to the environment, you can roll back the Helm upgrade and make adjustments.

Before you begin

  1. Review the changelog for version you want to upgrade to. Focus especially on any Breaking Changes that might require a different upgrade procedure.

  2. Check that your underlying Kubernetes platform runs a supported version for the Gloo version.

    1. Review the supported versions.
    2. Compare the supported version against the version of Kubernetes that you run in your clusters.
    3. If necessary, upgrade Kubernetes. Consult your cluster infrastructure provider.

Migrate the management cluster to the gloo-platform chart

  1. Upgrade the meshctl CLI to the version that you want to upgrade to during the migration. For example, if you currently use version 2.2, upgrade your CLI to the latest version of 2.3, 2.3.23. During the migration, your Gloo Mesh Gateway installation is upgraded to this version.

  2. Switch to your context to the management cluster.

      kubectl use-context $MGMT_CONTEXT

  3. Run the migration command.

      meshctl migrate helm

  4. From the command output, copy and paste the following generated commands:

    • helm repo add gloo-platform ...: Run this command to add the Helm repository for Gloo Platform.
    • helm upgrade --install gloo-platform-crds ...: Run this command to create the gloo-platform-crds Helm release. This release applies the Gloo CRDs for the Gloo version you are upgrading to. In future upgrades, you apply CRDs for target versions by upgrading this Helm release.

  5. Remove CRDs that are deprecated in 2.3.

      kubectl delete crd apischemas.apimanagement.gloo.solo.io
kubectl delete crd cloudproviders.networking.gloo.solo.io
kubectl delete crd cloudresources.networking.gloo.solo.io

  6. Optional for Istio and gateway lifecycle management: If you manually created IstioLifecycleManager and GatewayLifecycleManager CRs to deploy and manage your gateway proxies, you can either continue to manage your gateways in the CRs separately from your Helm release, or move your gateway management into your Helm release.

    To manage your gateways by using the CRs, no changes to the migration process are required. In the future, you continue to upgrade your gateways through CRs separately from your main Helm installation.

    To include gateway management into your Helm installation release, you perform a canary deployment by using your main installation Helm chart. Including gateway management in your Helm chart simplifies future upgrades.

    1. Edit the generated values file for the main installation, such as by running open gloo-mgmt-gloo-mesh-2.3.23-values.yaml.

    2. Add the following istioInstallations section to the values file. You’ll add the configuration for each installation in subsequent steps.

        
istioInstallations:
  enabled: true
  controlPlane:
    enabled: true
    installations:
      ...
  eastWestGateways: 
    - enabled: true
      name: istio-eastwestgateway
      installations:
        ...
  northSouthGateways:
    - enabled: true
      name: istio-ingressgateway
      installations:
        ...

    3. Get the current configuration in your IstioLifecycleManager and GatewayLifecycleManager CRs.

      • Control plane:
          kubectl get IstioLifecycleManager -n gloo-mesh istiod-control-plane -o yaml
      • East-west gateway (multicluster only):
          kubectl get GatewayLifecycleManager -n gloo-mesh istio-eastwestgateway -o yaml
      • Ingress gateway:
          kubectl get GatewayLifecycleManager -n gloo-mesh istio-ingressgateway -o yaml

    4. For each CRD, copy all configuration after the installations field, and paste it in the corresponding istioInstallations.controlPlane.installations, istioInstallations.northSouthGateways.installations, and istioInstallations.eastWestGateways.installations sections of your values file.

    5. Make the following changes to the values file.

      • Change the value of revision and gatewayRevision to a different value than your existing revision. For example, you might use ${REVISION}-MIGRATION.
      • Set defaultRevision and activeGateway to false. When you use this values file in the Helm upgrade for your Gloo Mesh Gateway installation, this setting ensures that only the Istio control plane and gateway revision that is managed by your existing CRs continues to run.

        Example configuration, which uses placeholder values that you replace with your own details:
        
istioInstallations:
  enabled: true
  controlPlane:
    enabled: true
    installations:
        # Different revision than your current installations
      - revision: ${REVISION}-MIGRATION
        clusters:
          - name: $REMOTE_CLUSTER1
            # Set to false for this canary installation
            defaultRevision: false
          - name: $REMOTE_CLUSTER2
            # Set to false for this canary installation
            defaultRevision: false
        istioOperatorSpec:
          profile: minimal
          hub: $REPO
          tag: $ISTIO_IMAGE
          namespace: istio-system
          ...
  eastWestGateways: 
    - enabled: true
      name: istio-eastwestgateway
      installations:
        - clusters:
            - name: $REMOTE_CLUSTER1
              # Set to false for this canary installation
              defaultRevision: false
            - name: $REMOTE_CLUSTER2
              # Set to false for this canary installation
              defaultRevision: false
          gatewayRevision: ${REVISION}-MIGRATION
          istioOperatorSpec:
            profile: empty
            hub: $REPO
            tag: $ISTIO_IMAGE
            namespace: gloo-mesh-gateways
            ...
  northSouthGateways:
    - enabled: true
      name: istio-ingressgateway
      installations:
          # Different revision than your current installations
        - gatewayRevision: ${REVISION}-MIGRATION
          clusters:
            - name: $REMOTE_CLUSTER1
              # Set to false for this canary installation
              defaultRevision: false
            - name: $REMOTE_CLUSTER2
              # Set to false for this canary installation
              defaultRevision: false
          istioOperatorSpec:
            profile: empty
            hub: $REPO
            tag: $ISTIO_IMAGE
            namespace: gloo-mesh-gateways
            ...

    6. Save and close the file.

    7. Proceed to the next step to migrate your main installation, in which you use your updated values file.

    8. After you complete all of the steps in this migration guide, continue with step 2 in the gateway proxy upgrade guide to verify your canary Helm-managed revision, set the canary revision to the default version, and uninstall your old revision by deleting the CRs.

  7. Migrate your Gloo Mesh Gateway values to the gloo-platform chart by upgrading your existing release.

    1. Check out the contents of the generated values file for the main installation, such as by running cat gloo-mgmt-gloo-mesh-2.3.23-values.yaml.

    2. Copy and paste the generated helm upgrade --install gloo-mgmt ... command to map the values from the legacy chart fields to the new fields in the gloo-platform chart. You release might have a different name.

    3. Verify that the Helm release now lists gloo-platform-2.3.23 as the base Helm chart.

        helm list -A

      Example output:

        NAME          NAMESPACE      REVISION	    UPDATED                               	STATUS      CHART                     	APP VERSION
gloo-mgmt     gloo-mesh      1       	    2023-02-08 13:25:57.937179 -0500 -0500	deployed    gloo-platform-2.3.23

    4. Verify your Gloo Mesh Gateway pods are running.

        kubectl get pods -n gloo-mesh --context $MGMT_CONTEXT

      Example output: 

        NAME                                    READY   STATUS    RESTARTS   AGE
gloo-mesh-mgmt-server-7cdcbcbd4-4s8wp   1/1     Running   0          30d
gloo-mesh-redis-794d79b7df-r2rtp        1/1     Running   0          30d
gloo-mesh-ui-748fd66f5c-lftcx           3/3     Running   0          30d
prometheus-server-647b488bb-vg7t5       2/2     Running   0          30d

  8. Important metrics updates: Depending on whether you run the Gloo OpenTelemetry (OTel) pipeline or legacy metrics pipeline, you might need to make changes to your installation.

    If you set up the Gloo OpenTelemetry (OTel) pipeline prior to migration, your installation continues to use the OTel pipeline after the migration process.

    However, due to a service name change for telemetry components, you must update the telemetry gateway IP address that the collector agents send metrics to.

    1. Save the new telemetry gateway IP address in an environment variable. In multicluster setups, run these commands in your management cluster.
        export TELEMETRY_GATEWAY_IP=$(kubectl get svc -n gloo-mesh gloo-telemetry-gateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
export TELEMETRY_GATEWAY_PORT=$(kubectl -n gloo-mesh get service gloo-telemetry-gateway -o jsonpath='{.spec.ports[?(@.name=="otlp")].port}')
export TELEMETRY_GATEWAY_ADDRESS=${TELEMETRY_GATEWAY_IP}:${TELEMETRY_GATEWAY_PORT}
echo $TELEMETRY_GATEWAY_ADDRESS
    2. When you upgrade your workload clusters in step 5 of the next section, include the --set telemetryCollector.config.exporters.otlp.endpoint=$TELEMETRY_GATEWAY_ADDRESS flag in the command.

    If you did not previously set up the Gloo OpenTelemetry (OTel) pipeline, and you use the legacy metrics pipeline, your installation continues to use the legacy metrics pipeline after the migration process.

    However, note that Istio version 1.17 does not support the legacy metrics pipeline. Before you upgrade or deploy gateway proxies with Istio 1.17, be sure that you set up the Gloo OpenTelemetry (OTel) pipeline instead. The Gloo telemetry pipeline is automatically set up if you follow the Get started or Install with Helm guides.

Migrate workload clusters to the gloo-platform chart

  1. Switch to the context for a workload cluster. Remember to change the context for each workload cluster that you upgrade.

      kubectl use-context $REMOTE_CONTEXT

  2. Run the migration command.

      meshctl migrate helm

  3. From the command output, copy and paste the following generated commands:

    • helm upgrade --install gloo-platform-crds ...: Run this command to create the gloo-platform-crds Helm release. This release applies the Gloo CRDs for the Gloo version you are upgrading to. In future upgrades, you apply CRDs for target versions by upgrading this Helm release.
    • helm upgrade --install gloo-agent-addons ...: If the migration generated this command, run it to upgrade your add-ons release. In the future, you continue to upgrade this Helm release separately from your main installation release.

  4. Migrate your Gloo agent values to the gloo-platform chart by upgrading your existing release.

    1. Check out the contents of the generated values file for the main installation, such as by running cat gloo-agent-gloo-mesh-2.3.23-values.yaml.

    2. Copy and paste the generated command helm upgrade --install gloo-agent ... command to map the values from the legacy chart fields to the new fields in the gloo-platform chart. You release might have a different name. Note: If you use the OTel telemetry pipeline, include the --set telemetryCollector.config.exporters.otlp.endpoint=$TELEMETRY_GATEWAY_ADDRESS flag in the upgrade command.

    3. Verify that the Helm release now lists gloo-platform-2.3.23 as the base Helm chart.

        helm list -A

      Example output:

        NAME          NAMESPACE      REVISION	    UPDATED                               	STATUS      CHART                     	APP VERSION
gloo-agent    gloo-mesh      1       	    2023-02-08 13:25:57.937179 -0500 -0500	deployed    gloo-platform-2.3.23

    4. Verify your Gloo agent components, and optionally agent add-on components, are running.

        meshctl check

      Example output:

        ...
🟢 Gloo deployment status

Namespace        | Name                  | Ready | Status 
gloo-mesh        | gloo-mesh-agent       | 1/1   | Healthy
gloo-mesh-addons | ext-auth-service      | 1/1   | Healthy
gloo-mesh-addons | rate-limiter          | 1/1   | Healthy
gloo-mesh-addons | redis                 | 1/1   | Healthy

  5. Repeat steps 1 - 4 for each workload cluster.

  6. Verify your management and agent components are connected.

      meshctl check --kubecontext $MGMT_CONTEXT

    Example output:

      🟢 License status

 INFO  gloo-mesh enterprise license expiration is 25 Aug 24 10:38 CDT
 INFO  No GraphQL license module found for any product

🟢 CRD version check

🟢 Gloo deployment status

Namespace        | Name                  | Ready | Status
gloo-mesh        | gloo-mesh-mgmt-server | 1/1   | Healthy
gloo-mesh        | gloo-mesh-redis       | 1/1   | Healthy
gloo-mesh        | gloo-mesh-ui          | 1/1   | Healthy
gloo-mesh        | prometheus-server     | 1/1   | Healthy

🟢 Mgmt server connectivity to workload agents

Cluster  | Registered | Connected Pod                                   
cluster1 | true       | gloo-mesh/gloo-mesh-mgmt-server-65bd557b95-v8qq6
cluster2 | true       | gloo-mesh/gloo-mesh-mgmt-server-65bd557b95-v8qq6