Proxy CRD Spec Compression

Gloo Edge’s 2-layer API aggregates all the routes in to a single Proxy CRD. This is helpful for usability and debugging, and allows rapid support in various user level APIs.

When using many routes, the Proxy CRDs can get quite big, and may surpass the storage limits set by etcd (you will see a message in the logs: etcdserver: request is too large).

To workaround that (as of Gloo Edge 1.5.0+), you can enable compression for the Proxy CRD spec, so that the data stored in etcd is reduced significantly (by a factor of 10 or so).

Note that this is a workaround and should only be used when necessary.

To enable proxy spec compression, edit the gloo the settings, like so:

kubectl patch settings -n gloo-system default --type='merge' -p '{"spec":{"gateway":{"compressedProxySpec":true}}}'

This will result in settings looking like this:

kind: Settings
    app: gloo
  name: default
  namespace: gloo-system
    compressedProxySpec: true
    readGatewaysFromAllNamespaces: false
      allowWarnings: true
      alwaysAccept: true
      proxyValidationServerAddr: gloo:9988

Once set, the Proxies generated by the gateway proxy pod will contain a compressed spec. For example:

kind: Proxy
    # do NOT remove this annotation! "true"
    created_by: gateway
  name: gateway-proxy
  namespace: gloo-system
  compressedSpec: eJzEUj1rwzA…SfOKY=
  reportedBy: gloo
  state: 1
The proxy object will have an extra annotation: "true". Do not remove it!

To view the contents of the spec in an uncompressed form, you can use one of the following commands:

kubectl get proxy gateway-proxy -n gloo-system -o 'jsonpath={@.spec.compressedSpec}'|base64 -d | openssl zlib -d|jq

kubectl get proxy gateway-proxy -n gloo-system -o 'jsonpath={@.spec.compressedSpec}'|base64 -d | perl -e 'use Compress::Raw::Zlib;my $d=new Compress::Raw::Zlib::Inflate();my $o;undef $/;$d->inflate(<>,$o);print $o;'|jq

kubectl get proxy gateway-proxy -n gloo-system -o 'jsonpath={@.spec.compressedSpec}' | python3 -c 'import base64,zlib,sys;sys.stdout.write(zlib.decompress(base64.decodebytes("utf8"))).decode("utf8"))'|jq

glooctl get proxy -o yaml gateway-proxy