Modify Helm chart values

Gloo Platform uses Helm chart values for settings that control features such as relay certificates, rate limiting, or external authentication.

As you use Gloo, you might need to modify the Helm chart values to enable or disable these settings for the features that you want to use. Upgrading the Helm chart settings typically does not restart the gloo-mesh-mgmt-server or gloo-mesh-agent pods in your clusters. However, to verify any prerequisite steps or impact, consult the guides for the feature that you want to enable before changing the Helm chart values.

About Gloo Helm chart components

The following table provides an overview of the Gloo Helm chart components. You might want to modify some of these charts’ values for your Gloo setup.

Helm chart Component Prefix Description
gloo-mesh-enterpise Management server glooMeshMgmtServer A component of the Gloo Mesh Enterprise chart to deploy the management server on the management cluster.
gloo-mesh-enterpise UI glooMeshUi A component of the Gloo Mesh Enterprise chart to render the UI that you can launch with the meshctl dashboard command.
gloo-mesh-enterpise Redis glooMeshRedis An optional component of the Gloo Mesh Enterprise chart to store the ID tokens you use to log in to the UI.
gloo-mesh-agent Agent glooMeshAgent The Gloo Mesh agent, deployed on each workload cluster.
ext-auth-service* External auth ext-auth-service An optional subchart of the enterprise server or agent Helm charts to configure the settings of the Gloo external auth service.
rate-limiter* Rate limiting rate-limiter An optional subchart of the enterprise server or agent Helm charts to configure the settings of the Gloo rate limiting service.

* Note that you modify external auth and rate limiting subcharts through the main Helm chart that is deployed in the cluster. In multicluster scenarios, the external auth and rate limiting services are deployed to the workload clusters, so you modify these subcharts through the agent Helm chart. If you have a single cluster setup, however, you might deploy everything through the management server Helm chart. For more information, see Modify external auth or rate limiting subcharts.

Before you begin

  1. Review the Helm value reference documentation for a description of the Helm chart settings that you can modify. Drill down to the reference section for the version of Gloo Mesh Enterprise that you run. To check your version, run meshctl version --kubecontext $MGMT_CONTEXT.

  2. Save the environment variables that you need.

    # Save the kubeconfig contexts for your clusters. Run kubectl config get-contexts, look for your cluster in the CLUSTER column, and get the context name in the NAME column. Note: Do not use context names with underscores. The context name is used as a SAN specification in the generated certificate that connects workload clusters to the management cluster, and underscores in SAN are not FQDN compliant. You can rename a context by running kubectl config rename-context "<oldcontext>" <newcontext>.
    export MGMT_CONTEXT=<management_cluster_config>
    export REMOTE_CONTEXT=<remote_cluster_config>
    
    # Set the Gloo Mesh Enterprise version. The latest version is used as an example. You can find other versions in the Changelog documentation.
    export GLOO_VERSION=2.2.0-beta1
    
    # Add your Gloo Mesh Enterprise license that you got from your Solo account representative.
    export GLOO_MESH_LICENSE_KEY=<license_key>
    

Installing or upgrading modified Helm values

Install or upgrade the Gloo Mesh Enterprise Helm chart with helm install or helm upgrade CLI commands by using a configuration file such as values.yaml or the --set option. You can download the bundled Gloo Mesh Enterprise charts from https://storage.googleapis.com/gloo-mesh-enterprise/.

Make sure to include your Helm values when you upgrade either as a configuration file in the –values flag or with –set flags. Otherwise, any previous custom values that you set might be overwritten. In single cluster setups, this might mean that your Gloo agent and ingress gateways are removed. For more information, see Get your Helm chart values in the upgrade guide.

Want to modify the default deployment values for the external auth or rate limiting services, such as to set resource requests and limits? See Modify external auth or rate limiting subcharts.

  1. Target your cluster.

    
        kubectl config use-context ${MGMT_CONTEXT}
        
    
        kubectl config use-context ${REMOTE_CONTEXT}
        

  2. Add and update the Helm chart repos. Add or update the gloo-mesh-enterprise repo for the management cluster and gloo-mesh-agent for the workload clusters.

    
        helm repo add gloo-mesh-enterprise https://storage.googleapis.com/gloo-mesh-enterprise/gloo-mesh-enterprise
        
    
        helm repo add gloo-mesh-agent https://storage.googleapis.com/gloo-mesh-enterprise/gloo-mesh-agent
        
    
        helm repo update
        

  3. Optional: View the latest chart versions.

    helm search repo
    
  4. Optional: If you already installed the Helm chart, view the current Helm values of the installed chart.

    
       helm show values gloo-mesh-enterprise/gloo-mesh-enterprise --version $GLOO_VERSION
       
    
       helm show values gloo-mesh-agent/gloo-mesh-agent --version $GLOO_VERSION
       
  5. If you did not already install the Helm chart, create the gloo-mesh namespace.

    kubectl create ns gloo-mesh
    
  6. Continue to the next sections to install or upgrade Helm chart values, depending on which method you want to use.

Modifying values with –set

Specify the Helm value that you want to modify in the --set command. Remember to include the prefix for the component that you want to modify. For a description of the Helm chart settings that you can modify, review the Helm value reference documentation.


helm install gloo-mgmt gloo-mesh-enterprise/gloo-mesh-enterprise \
--namespace gloo-mesh \
--kube-context ${MGMT_CONTEXT} \
--version ${GLOO_VERSION} \
--set global.cluster=$MGMT_CLUSTER \
--set mgmtClusterName=$MGMT_CLUSTER \
--set glooMeshLicenseKey=${GLOO_MESH_LICENSE_KEY} \
--set glooMeshMgmtServer.floatingUserId=true \
--set glooMeshUi.floatingUserId=true \
--set glooMeshRedis.floatingUserId=true

helm upgrade gloo-mgmt gloo-mesh-enterprise/gloo-mesh-enterprise \
--namespace gloo-mesh \
--kube-context ${MGMT_CONTEXT} \
--version ${GLOO_VERSION} \
--set global.cluster=$MGMT_CLUSTER \
--set mgmtClusterName=$MGMT_CLUSTER \
--set glooMeshLicenseKey=${GLOO_MESH_LICENSE_KEY} \
--set glooMeshMgmtServer.floatingUserId=true \
--set glooMeshUi.floatingUserId=true \
--set glooMeshRedis.floatingUserId=true

helm install gloo-agent gloo-mesh-agent/gloo-mesh-agent \
--namespace gloo-mesh \
--kube-context=${REMOTE_CONTEXT} \
--version ${GLOO_VERSION} \
--set insecure=true

helm upgrade gloo-agent gloo-mesh-agent/gloo-mesh-agent \
--namespace gloo-mesh \
--kube-context=${REMOTE_CONTEXT} \
--version ${GLOO_VERSION} \
--set insecure=true

Modifying values with a configuration file

  1. Create a configuration file with a name such as values.yaml, and include the Helm values that you want to modify, such as the following example. The example modifies the settings of the glooMeshMgmtServer and glooMeshUi components so that you can provide your own relay certificates. The example also shows how you might use the deploymentOverrides and serviceOverrides to modify the default deployment of the glooMeshMgmtServer with your own Kubernetes resources, like a config map or service account.

    For a description of each Helm chart setting that you can modify, review the Helm value reference documentation.

    insecure: false
    global:
       cluster: $MGMT_CLUSTER
    glooMeshMgmtServer:
      deploymentOverrides:
        spec:
          template:
            spec:
              volumeMounts:
                - name: envoy-config
                  configMap:
                    name: my-custom-envoy-config
      floatingUserId: false
      relay:
        disableCa: true
        disableCaCertGeneration: true
        tlsSecret:
          name: relay-server-tls-secret
          namespace: gloo-mesh
      serviceOverrides:
        spec:
          serviceAccountName: other-service-account
    glooMeshUi:
      floatingUserId: false
    
  2. Optional: Review your changes in the context of the entire template for the Helm chart that you plan to install, such as gloo-mesh-enterprise/gloo-mesh-enterprise for the management cluster. The output includes the YAML manifests for all the resources that Gloo Mesh Enterprise installs in your cluster with the Helm chart, including the changes that you just made.

    helm template gloo-mgmt https://storage.googleapis.com/gloo-mesh-enterprise/gloo-mesh-enterprise/gloo-mesh-enterprise-$GLOO_VERSION.tgz --namespace gloo-mesh --values values.yaml
    
  3. Install or upgrade the Helm chart with your updated Helm values.


helm install gloo-mgmt gloo-mesh-enterprise/gloo-mesh-enterprise \
--namespace gloo-mesh \
--kube-context ${MGMT_CONTEXT} \
--version ${GLOO_VERSION} \
--set glooMeshLicenseKey=${GLOO_MESH_LICENSE_KEY} \
--values values.yaml

helm upgrade gloo-mgmt gloo-mesh-enterprise/gloo-mesh-enterprise \
--namespace gloo-mesh \
--kube-context ${MGMT_CONTEXT} \
--version ${GLOO_VERSION} \
--values values.yaml

helm install gloo-agent gloo-mesh-agent/gloo-mesh-agent \
--namespace gloo-mesh \
--kube-context=${REMOTE_CONTEXT} \
--version ${GLOO_VERSION} \
--values values.yaml

helm upgrade gloo-agent gloo-mesh-agent/gloo-mesh-agent \
--namespace gloo-mesh \
--kube-context=${REMOTE_CONTEXT} \
--version ${GLOO_VERSION} \
--values values.yaml

Modify external auth or rate limiting subcharts

Gloo Platform includes two optional Helm subcharts to deploy external auth and rate limiting services. After deployment, these services are used to enforce external auth and rate limiting policies. You can modify the default Kubernetes deployment settings through the subcharts. For example, you might want to modify the resource requests based on your usage.

You modify the external auth and rate limiting subcharts through the main Helm chart that is deployed in the cluster. In multicluster scenarios, the external auth and rate limiting services are deployed to the workload clusters, so you modify these subcharts through the agent Helm chart. If you have a single cluster setup, however, you might deploy everything through the management server Helm chart.

Make sure to include your Helm values when you upgrade either as a configuration file in the –values flag or with –set flags. Otherwise, any previous custom values that you set might be overwritten. In single cluster setups, this might mean that your Gloo agent and ingress gateways are removed. For more information, see Get your Helm chart values in the upgrade guide.

Before you begin: Deploy the rate limiting or external auth services.

  1. Target the cluster with the rate limiting or external auth service that you want to modify.

    
    kubectl config use-context ${MGMT_CONTEXT}
    
    
    kubectl config use-context ${REMOTE_CONTEXT}
    

  2. Add and update the Helm chart repos. Add or update the gloo-mesh-enterprise repo for the management cluster and gloo-mesh-agent for the workload clusters.

    
    helm repo add gloo-mesh-enterprise https://storage.googleapis.com/gloo-mesh-enterprise/gloo-mesh-enterprise
    
    
    helm repo add gloo-mesh-agent https://storage.googleapis.com/gloo-mesh-enterprise/gloo-mesh-agent
    
    
    helm repo update
    

  3. View the latest versions for the rate-limiter and ext-auth-service subcharts.

    helm search repo
    

    Example output:

    NAME                                CHART VERSION	APP VERSION  DESCRIPTION
    ext-auth-service/ext-auth-service   0.23.0       	             Solo.io external auth server Helm chart for Kub..
    rate-limiter/rate-limiter           0.7.3        	             Solo.io rate limit server Helm chart for Kubern...
    
  4. Review the current Helm values. Replace the <version> variables with the chart version that you previously retrieved.

    
    helm show values ext-auth-service/ext-auth-service --version <ext-auth-service-version>
    
    
    helm show values rate-limiter/rate-limiter --version <rate-limiter-version>
    
  5. From the output of the previous step, review the Helm values to identify the values that you want to change, such as the following examples. Note the full path of the setting, such as extAuth.resources.requests.cpu.

    
    extAuth:
      # Watch all namespaces by default
      watchNamespace: ""
      image:
        pullPolicy: IfNotPresent
        registry: gcr.io/gloo-mesh
        repository: ext-auth-service
        tag: 0.23.0
      resources:
        requests:
          cpu: 125m
          memory: 256Mi
      logLevel: INFO
      userIdHeader: ""
      # Provide the server's secret signing key.
      # If empty, a random key will be generated.
      signingKey: ""
      # Set signingKeyFile.enabled to true to mount secret as file rather than pass
      # the signing key as a environment variable. To ensure maximum security by
      # default, the file will be limited to 0440 permissions and have the fsGroup
      # set to match the runAsGroup.
      signingKeyFile:
        enabled: false
        fileMode: 288
        groupSettingEnabled: true
        fsGroup: 10101
        runAsUser: 10101
        runAsGroup: 10101
      # Directory in which the server expects Go plugin .so files.
      pluginDirectory: "/auth-plugins/"
      # Headers that will be redacted in the server logs.
      headersToRedact:
      - authorization
      # When receiving a termination signal, the pod will wait this amount of seconds
      # for a request that it can use to notify Envoy that it should fail the health check
      # for this endpoint. If no request is received within this interval, the server will
      # shutdown gracefully. The interval should be greater than the active health check
      # interval configured in Envoy for this service.
      healthCheckFailTimeout: 15
      healthCheckHttpPath: /healthcheck
      service:
        type: ClusterIP
        grpcPort: 8083
        debugPort: 9091
        healthPort: 8082
        # Only relevant if the service is of NodePort type
        grpcNodePort: 32000
        debugNodePort: 32001
        healthNodePort: 32002
    
    
    rateLimiter:
      logLevel: INFO
      # Watch all namespaces by default
      watchNamespace: ""
      image:
        pullPolicy: IfNotPresent
        registry: gcr.io/gloo-mesh
        repository: rate-limiter
        tag: 0.7.3
      resources:
        requests:
          cpu: 125m
          memory: 256Mi
      ports:
        grpc: 8083
        ready: 8084
        debug: 9091
      readyPath: /ready
      alivePath: /alive
      installClusterRoles: true  # If true, use ClusterRoles.  If false, use Roles.
    redis:
      image:
        pullPolicy: IfNotPresent
        registry: docker.io
        repository: redis
        tag: 6.2.6
      service:
        port: 6379
        name: redis
        socket: tcp # Values may be 'unix', 'tcp', or 'tls'
      hostname: redis  # The hostname clients should use for dialing Redis
      auth:
        enabled: false  # If true, will use Redis AUTH
        secretName: redis-secrets  # Name of the secre that contains the username and password 
        passwordKey: redis-password # Key that contains the password
        usernameKey: redis-username  # Key that contains the username.  Use 'default' if Redis doesn't have an explicit username
      enabled: true # When true, Redis will be installed
      certs:
        enabled: false # When true, rate-limiter (and Redis, if enabled) will use an explicit cacert  
        mountPoint: "/etc/tls" # mount point for the certs
        caCert: "redis.crt" # File name that contains the ca cert
        signingKey: "redis.key"  # File name that contains the signing key (relevant to Redis only)
        secretName: "redis-certs-keys"
    

  6. Modify the external auth or rate limiting subchart, by using the --set command or passing in a values.yaml file.

    If you did not deploy the rate limiting or external auth services to the gloo-mesh-addons namespace, you might have different Helm chart names than the following examples. For example, in a single cluster scenario, you might deploy the services as part of the management server Helm chart. If so, update the correct name, such as helm upgrade gloo-mgmt gloo-mesh-enterprise/gloo-mesh-enterprise.

    • Example to modify with the --set command:
      
      helm upgrade gloo-agent-addons gloo-mesh-agent/gloo-mesh-agent \
      --namespace gloo-mesh-addons \
      --set ext-auth-service.enabled=true \
      --set ext-auth-service.extAuth.resources.requests.cpu=250m \
      --set ext-auth-service.extAuth.resources.requests.memory=512M
      
      
      helm upgrade gloo-agent-addons gloo-mesh-agent/gloo-mesh-agent \
      --namespace gloo-mesh-addons \
      --set rate-limiter.enabled=true \
      --set rate-limiter.rateLimiter.resources.requests.cpu=250m \
      --set rate-limiter.rateLimiter.resources.requests.memory=512M
      
      
      helm upgrade gloo-agent-addons gloo-mesh-enterprise/gloo-mesh-enterprise \
      --namespace gloo-mesh-addons \
      --set registerMgmtPlane.ext-auth-service.enabled=true \
      --set registerMgmtPlane.ext-auth-service.extAuth.resources.requests.cpu=250m \
      --set registerMgmtPlane.ext-auth-service.extAuth.resources.requests.memory=512M
      
      
      helm upgrade gloo-agent-addons gloo-mesh-enterprise/gloo-mesh-enterprise \
      --namespace gloo-mesh-addons \
      --set registerMgmtPlane.rate-limiter.enabled=true \
      --set registerMgmtPlane.rate-limiter.rateLimiter.resources.requests.cpu=250m \
      --set registerMgmtPlane.rate-limiter.rateLimiter.resources.requests.memory=512M
      
    • Examples to modify with a values.yaml file:
      
      cat << EOF > values.yaml
      glooMeshAgent:
        enabled: false
      ext-auth-service:
        enabled: true
        extAuth:
          resources:
            requests:
              cpu: 250m
              memory: 512M
      EOF
      helm upgrade gloo-agent gloo-mesh-agent/gloo-mesh-agent \
      --namespace gloo-mesh-addons \
      --values values.yaml
      
      
      cat << EOF > values.yaml
      glooMeshAgent:
        enabled: false
      rate-limiter:
        enabled: true
        rateLimiter:
          resources:
            requests:
              cpu: 250m
              memory: 512M
      EOF
      helm upgrade gloo-agent gloo-mesh-agent/gloo-mesh-agent \
      --namespace gloo-mesh-addons \
      --values values.yaml