Storing Gloo Edge Config in Consul

While Kubernetes provides APIs for config storage (CRDs), credential storage (Secrets), and service discovery (Services), users may wish to run Gloo Edge without using Kubernetes.

Gloo Edge provides alternate mechanisms for configuration, credential storage, and service discovery that do not require Kubernetes, including the use of local .yaml files, Consul Key-Value storage and Vault Key-Value storage.

This document describes how to write configuration YAML to Consul’s Key-Value store to configure Gloo Edge.

Configuring Gloo Edge using custom Settings

When Gloo Edge boots, it attempts to read a resource from a preconfigured location. By default, Gloo Edge will attempt to connect to a Kubernetes cluster and look up the Custom Resource in namespace gloo-system, named default.

When desiring to run without Kubernetes, it is possible to instead provide this file to Gloo Edge inside of a configuration directory.

When running the gloo, discovery, and gateway processes, it is necessary to provide a --dir flag pointing to the config directory containing the Settings YAML.

If we were to create a directory called data, the structure of the directory should look like so:

├── artifacts
├── gloo-system
│   └── default.yaml
└── secrets

3 directories, 1 file

When we pass the flag --dir=./data to Gloo Edge, Gloo Edge will look for the settings file in data/<namespace>/*.yaml. The default namespace for Gloo Edge is gloo-system. This can be overridden with the --namespace flag.

Customizing the Gloo Edge Settings file

The full list of options for Gloo Edge Settings, including the ability to set auth/TLS parameters for Consul can be found in the v1.Settings API reference .

Here is provided an example Settings so Gloo Edge will read config from Consul Key-Value store:

# metadata of the Settings resource contained in this file
# name should always be set to default
# namespace should be "gloo-system" or the value of the --namespace used to start Gloo Edge
  name: default
  namespace: gloo-system

# bind address for gloo's configuration server

# connection options for consul
  # address of the consul agent/server
  # enable service discovery using consul
  serviceDiscovery: {}

# enable configuration using consul key-value storage
consulKvSource: {}

# enable secrets to be read from the local filesystem
  directory: /data/secret

# currently unused, but required
# /data/artifacts will be used
# for large file storage
  directory: /data

# the namespace/parent directory
# to which to write discovered resources, such as upstreams
discoveryNamespace: gloo-system

# refresh rate for polling config backends for changes
# this is used for watching vault secrets and the local filesystem
refreshRate: 15s

# status will be reported by Gloo Edge as "Accepted"
# if booted successfully
status: {}

Writing Config Objects to Consul

Consul Values should be written using Gloo Edge-style YAML, whose structure is described in the API Reference.

glooctl provides a convenience to get started writing Gloo Edge resources for use with Consul.

Using glooctl add route ... -o yaml and glooctl create ... -o yaml will output YAML-formatted objects which can be stored as values in Consul.

For example:

glooctl add route \
    --path-exact /sample-route-1 \
    --dest-name petstore \
    --prefix-rewrite /api/pets -oyaml

Will produce the following:

  name: default
  namespace: gloo-system
status: {}
  - '*'
  - matchers:
     - exact: /sample-route-1
          name: petstore
          namespace: gloo-system
      prefixRewrite: /api/pets

Gloo Edge YAML must be stored in Consul with the correct Key names.

Consul keys adhere to the following format:

<root key>/<resource group>/<group version>/<resource kind>/<resource namespace>/<resource name>


The paths for Gloo Edge’s API objects are as follows:

Resource Key gloo/<namespace>/<name> gloo/<namespace>/<name> gloo/<namespace>/<name> gloo/<namespace>/<name>

To store a Gloo Edge resource in Consul, one can use curl or the consul CLI:

# store using curl:
curl -v \
    -XPUT \
    --data-binary "@<resource yaml file>.yaml" \
    "<resource group>/<group version>/<resource kind>/<namespace>/<name>"

# store using consul:
consul kv put gloo/<resource group>/<group version>/<resource kind>/<namespace>/<name> @<resource yaml file>.yaml

For example, to store a Virtual Service:

# store using curl:
curl -v \
    -XPUT \
    --data-binary "@virtual-service.yaml" \

# store using consul:
consul kv put gloo/ @virtual-service.yaml

Stored resources can be viewed via the consul UI:

Consul UI

This can be useful for modifying configuration, or viewing the status reported by Gloo Edge.