1.12.0+ Upgrade Notice
Review and apply version-specific changes in Gloo Edge 1.12.x and Gloo Edge Enterprise 1.12.x before upgrading your Gloo Edge installation.
New CRDs are automatically applied to your cluster when performing a helm install
operation.
However, they are not applied when performing an helm upgrade
operation. This is a deliberate design choice on the part of the
Helm maintainers, given the risk associated with changing CRDs. Given this limitation, you must apply new CRDs to the cluster before upgrading.
Architectural changes
To reduce cross-pod communication, the gateway functionality is removed from a standalone pod, and is now included in the Gloo Edge control plane pod. Specifically, the validation webhook is moved to the gloo
pod, which reduces the number of gRPC calls required for validation.
The following changes apply to your Gloo Edge installation in 1.12.x:
- Installed pods: The
gateway
pod is removed from the set of installed components. - Installation settings:
- The sections of the Helm chart for the gateway deployment and service accounts are removed.
- Because the
gateway
pod is no longer installed, specify any environment variables for the gateway on thegloo
deployment instead. For example, in your Helm values file, you might set thePROXY_STATUS_MAX_SIZE_BYTES
environment variable to truncate the length of the proxy status field:gloo: deployment: customEnv: - name: "PROXY_STATUS_MAX_SIZE_BYTES" value: "100"
- To improve performance, the proxy CRD is not written to etcd by default and is instead kept in memory. If you must allow external tools and other pods to read the proxy CR contents, you can persist them to etcd by setting the
gateway.persistProxySpec
setting totrue
in your Helm values file. Important: If you use the Kubernetes ExternalDNS functionality, you must set thegateway.persistProxySpec
setting totrue
.
- Proxy CR contents: To view the contents of a proxy CR, you can continue to use the
glooctl get proxy <proxy> -o yaml
command. However, because the proxy CRD is no longer written to etcd by default, you cannot usekubectl
commands to check proxy CR contents. - Snapshot metrics: The
gateway
snapshot is removed. If you use metrics from the gateway snapshot, those metrics are moved to thegloo
snapshot; for example,api_gloosnapshot_gloo_solo_io_emitter_*
.
Helm changes
In Gloo Edge v1.12.0 / Gloo Edge Enterprise v1.12.0 and later, the default Gateways, ExtAuth Upstreams, and RateLimit Upstreams are installed via a Job during the Helm installation and upgrade process, instead of being included directly in the Helm chart. After you upgrade to these versions, those resources no longer have Helm annotations (e.g. meta.helm.sh/release-name
, meta.helm.sh/release-namespace
). If you need to roll back to an earlier Gloo Edge version, you must manually add the Helm release annotations back to these resources before performing the rollback.
Do not manually edit (e.g. kubectl apply
or kubectl edit
) the Gloo Edge custom resources (e.g. Gateways, Upstreams) that are installed with the Helm chart, as this might cause errors in future upgrades.
If you already manually edited a resource, verify before you upgrade that the resource’s metadata.annotations.kubectl.kubernetes.io/last-applied-configuration
annotation does not contain a resourceVersion
field. This field can cause conflicts during the upgrade process.
If the field exists, you can remove it in either of the following ways:
kubectl edit
the resource and manually delete theresourceVersion
field from themetadata.annotations.kubectl.kubernetes.io/last-applied-configuration
annotation, or- remove the
metadata.annotations.kubectl.kubernetes.io/last-applied-configuration
annotation altogether if not needed, e.g.kubectl annotate gateway -n gloo-system gateway-proxy kubectl.kubernetes.io/last-applied-configuration-
CRD changes
New and updated CRDs
- In the ExtAuth CR, the
spec.configs.oauth2.oidcAuthorizationCode.headers
section is updated to include theuseBearerSchemaForAuthorization
setting to add the “Bearer” prefix to the upstream access token header value ("Authorization: Bearer <access_token>"
). In Gloo Edge 1.12, this setting defaults to false. In Gloo Edge 1.13, this setting will default to true. For more information, see the ExtAuth reference documentation.
Deprecated CRDs None
Removed CRDs None
Upgrade steps
To upgrade Gloo Edge:
-
Follow steps 1 - 2 in Upgrade Steps to prepare for upgrading, and to upgrade
glooctl
. -
Apply the new and updated CRDs. Replace the version with the specific patch version that you are upgrading to.
helm repo update helm pull gloo/gloo --version 1.12.0 --untar kubectl apply -f gloo/crds
helm repo update helm pull glooe/gloo-ee --version 1.12.0 --untar kubectl apply -f gloo-ee/charts/gloo/crds # If Gloo Federation is enabled kubectl apply -f gloo-ee/charts/gloo-fed/crds
-
Prepare for any updates you want to make related to the gateway architectural changes. For example, you might want to create a Helm values file to pass gateway environment variables to the
gloo
pod during the Helm upgrade, or update any systems that watch for snapshot metrics. -
Continue to upgrade the Gloo Edge server components via Helm.
-
If you use the
spec.configs.oauth2.oidcAuthorizationCode.headers.accessTokenHeader: "Authorization"
setting in anyAuthConfig
resources, and you do not want to add the “Bearer” prefix to the header, setuseBearerSchemaForAuthorization
tofalse
. If you do not set this setting, it will default to true in Gloo Edge 1.13.