☰
Navigation :

1.3.0+ Upgrade Notice

Upgrade using helm to the latest Gloo Edge release, in-place with no downtime! As there were some important helm chart fixes related to helm hooks between Gloo Edge 1.2.x and Gloo Edge 1.3.x, there are some one-time steps you may need to take in addition helm upgrade to get your helm release into the proper state. This document outlines all the required and recommended steps.

Upgrade Prereqs

Install:

Without glooctl 1.2.5+, glooctl proxy url will output the version mismatch warning on stdout during upgrade, and our curl and hey commands will fail. Without glooctl 1.3.0+, the --version flag will not exist.

This upgrade guide also requires that Gloo Edge was installed via helm or with glooctl version 1.2.0+ (i.e., gloo is a helm release named “gloo”, you can confirm this exists with helm ls --all-namespaces). If upgrading from an older version of Gloo Edge, follow the steps outlined here for a blue-green deployment and cutover.

If upgrading to Gloo Edge 1.3.15 or lower, or to one of the Gloo Edge Enterprise 1.3.0-beta releases, set the EDS warming timeout to a nonzero value (recommended 5 minutes):

kubectl patch settings -n gloo-system default \
    --patch '{"spec": {"gloo": {"endpointsWarmingTimeout": "5m"}}}' --type=merge

Please refer to this page for general configuration recommendations to avoid downtime during upgrades.

Example: Test a Zero-Downtime helm upgrade

You can skip straight to the upgrade steps here.

Setup

Install gloo:

glooctl install gateway --version 1.2.0

Add an upstream:

apiVersion: gloo.solo.io/v1
kind: Upstream
metadata:
  name: json-upstream
  namespace: gloo-system
spec:
  static:
    hosts:
      - addr: jsonplaceholder.typicode.com
        port: 80

And make it routable:

apiVersion: gateway.solo.io/v1
kind: VirtualService
metadata:
  name: test-prefix
  namespace: gloo-system
spec:
  virtualHost:
    routes:
      - matchers:
         - prefix: /posts
        routeAction:
          single:
            upstream:
              name: json-upstream
              namespace: gloo-system
        options:
          autoHostRewrite: true

Wait until the following returns:

curl $(glooctl proxy url)/posts

  ... # omitted for brevity
  {
    "userId": 10,
    "id": 100,
    "title": "at nam consequatur ea labore ea harum",
    "body": "cupiditate quo est a modi nesciunt soluta\nipsa voluptas error itaque dicta in\nautem qui minus magnam et distinctio eum\naccusamus ratione error aut"
  }
]

Upgrading

Now Gloo Edge is installed and ready for upgrade.

If running Gloo Edge 1.2.x or lower (open-source or enterprise), label the default gateways and settings to become part of your helm release:

This requires helm 3.2 or newer. If you must use helm 2, drop into our slack for help upgrading your setup.

# adopt the old gateway into your existing helm release
kubectl annotate gateway gateway-proxy meta.helm.sh/release-name=gloo -n gloo-system
kubectl annotate gateway gateway-proxy meta.helm.sh/release-namespace=gloo-system -n gloo-system
kubectl label gateway gateway-proxy app.kubernetes.io/managed-by=Helm -n gloo-system

# adopt the old ssl gateway into your existing helm release
kubectl annotate gateway gateway-proxy-ssl meta.helm.sh/release-name=gloo -n gloo-system
kubectl annotate gateway gateway-proxy-ssl meta.helm.sh/release-namespace=gloo-system -n gloo-system
kubectl label gateway gateway-proxy-ssl app.kubernetes.io/managed-by=Helm -n gloo-system

# adopt the old settings into your existing helm release
kubectl annotate settings default meta.helm.sh/release-name=gloo -n gloo-system
kubectl annotate settings default meta.helm.sh/release-namespace=gloo-system -n gloo-system
kubectl label settings default app.kubernetes.io/managed-by=Helm -n gloo-system

Now you’re ready to upgrade!

Upgrade to Open-Source
Click for Open-Source upgrade instruction

Upgrade to open-source Gloo Edge 1.3.x (helm 2 or helm 3): 


helm upgrade gloo gloo/gloo --namespace gloo-system --version 1.3.17 \
    --set gatewayProxies.gatewayProxy.podTemplate.probes=true


hey -n 6000 -c 10 -q 10 $(glooctl proxy url)/posts & helm upgrade gloo gloo/gloo --namespace gloo-system --version 1.3.17 \
    --set gatewayProxies.gatewayProxy.podTemplate.probes=true

Upgrade to Gloo Edge Enterprise
Click for Gloo Edge Enterprise upgrade instructions

If upgrading from Gloo Edge Enterprise 1.3.0-beta6 or lower (including 1.2.x) to Gloo Edge Enterprise 1.3.0 (or the beta7), you will also need to delete the grafana deployment and service to work around breaking changes in the grafana subchart:

kubectl delete deployment -n gloo-system glooe-grafana
kubectl delete service -n gloo-system glooe-grafana

Upgrade to Gloo Edge Enterprise 1.3.0 (helm 2 or helm 3):

The --set grafana.persistence.storageClassName=<currently installed pvc storage class> is only required if upgrading from Gloo Edge Enterprise 1.3.0-beta6 or lower and grafana is enabled (the default). In most installations the storage class will be standard, but gp2 is common for EKS and it’s worth confirming the deployed storage class before attempting installation with kubectl get pvc -n gloo-system.


helm upgrade gloo glooe/gloo-ee --namespace gloo-system --version=1.3.0 \
    --set license_key=$LICENSE_KEY \
    --set gloo.gatewayProxies.gatewayProxy.podTemplate.probes=true \
    --set grafana.persistence.storageClassName=standard # if required, storage class must match the PVC that's already deployed


hey -n 6000 -c 10 -q 10 $(glooctl proxy url)/posts & helm upgrade gloo glooe/gloo-ee --namespace gloo-system --version=1.3.0 \
    --set license_key=$LICENSE_KEY \
    --set gloo.gatewayProxies.gatewayProxy.podTemplate.probes=true \
    --set grafana.persistence.storageClassName=standard # if required, storage class must match the PVC that's already deployed

If your initial version was Gloo Edge Enterprise 1.3.0-beta6 or lower, you will always need to provide the grafana.persistence.storageClassName value going forward for upgrades, so add that to your helm values.

Wait for the request distribution at the end to confirm that requests weren’t dropped. You should see a response similar to the following (note that all requests get http 200):

[1] 20563
Release "gloo" has been upgraded. Happy Helming!
NAME: gloo
LAST DEPLOYED: Tue Mar 31 09:02:16 2020
NAMESPACE: gloo-system
STATUS: deployed
REVISION: 6
TEST SUITE: None
➜  gloo git:(doc_upgrade) (⎈ minikube:default)
Summary:
  Total:	17.8115 secs
  Slowest:	1.0715 secs
  Fastest:	0.0179 secs
  Average:	0.1624 secs
  Requests/sec:	280.7173


Response time histogram:
  0.018 [1]	|
  0.123 [1892]	|■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
  0.229 [2411]	|■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■
  0.334 [493]	|■■■■■■■■
  0.439 [121]	|■■
  0.545 [31]	|■
  0.650 [2]	|
  0.755 [12]	|
  0.861 [2]	|
  0.966 [8]	|
  1.072 [27]	|


Latency distribution:
  10% in 0.0730 secs
  25% in 0.1033 secs
  50% in 0.1438 secs
  75% in 0.1958 secs
  90% in 0.2452 secs
  95% in 0.3176 secs
  99% in 0.6126 secs

Details (average, fastest, slowest):
  DNS+dialup:	0.0000 secs, 0.0179 secs, 1.0715 secs
  DNS-lookup:	0.0000 secs, 0.0000 secs, 0.0000 secs
  req write:	0.0000 secs, 0.0000 secs, 0.0005 secs
  resp wait:	0.1587 secs, 0.0167 secs, 1.0713 secs
  resp read:	0.0036 secs, 0.0002 secs, 0.2522 secs

Status code distribution:
  [200]	5000 responses




[1]  + 20563 done       hey -n 5000 $(glooctl proxy url)/posts

Run glooctl check to ensure everything is healthy:

glooctl check