Delegating with Route Tables

The Gloo Gateway Virtual Service makes it possible to define all routes for a domain on a single configuration resource.

However, condensing all routing config onto a single object can be cumbersome when dealing with a large number of routes.

Gloo Gateway provides a feature referred to as delegation. Delegation allows a complete routing configuration to be assembled from separate config objects. The root config object delegates responsibility to other objects, forming a tree of config objects. The tree always has a Virtual Service as its root, which delegates to any number of Route Tables. Route Tables can further delegate to other Route Tables.

Motivation

Use cases for delegation include:

Using delegation, organizations can delegate ownership of routing config to individuals or teams. Those individuals or teams can then further delegate routes to others.

Config Model

A configuration using Delegation can be understood as a tree.

The root node in the tree is a Virtual Service while every child node is a RouteTable:

graph LR; vs[Virtual Service

*.petclinic.com] -->|delegate /api prefix | rt1(Route Table

/api/pets
/api/vets) vs -->|delegate /site prefix | rt2(Route Table

/site/login
/site/logout) style vs fill:#0DFF00,stroke:#233,stroke-width:4px

Route Tables can be nested for any level of granularity:

graph LR; vs[Virtual Service

*.petclinic.com] -->|delegate /api prefix | rt1(Route Table

/api/pets
/api/vets) rt1 -->|delegate /api/pets/.* prefix | rt3(Route Table

/api/pets/.*/records
/api/pets/.*/billing) vs -->|delegate /site prefix | rt2(Route Table

/site/login
/site/logout) rt1 -->|delegate /api/vets prefix | rt4(Route Table

GET /api/vets
POST /api/vets) style vs fill:#0DFF00,stroke:#233,stroke-width:4px

Non-delegating routes can be defined at every level of the config tree:

graph LR; vs[Virtual Service

*.petclinic.com] -->|delegate /api prefix | rt1(Route Table

/api/pets
/api/vets) vs -->|delegate /site prefix | rt2(Route Table

/site/login
/site/logout) vs -->|route /pharmacy | us1(Upstream

pharmacy-svc.petstore.cluster.svc.local:80) rt1 -->|route /api/pets | us2(Upstream

pet-svc.petstore.cluster.svc.local:80) style vs fill:#0DFF00,stroke:#233,stroke-width:4px style us1 fill:#f9f,stroke:#333,stroke-width:4px style us2 fill:#f9f,stroke:#333,stroke-width:4px

Routes defined at any level must inherit the prefix delegated to them, else Gloo Gateway will not consider the config tree valid:

graph LR; subgraph invalid vs[Virtual Service

*.petclinic.com] -->|delegate /api prefix | rt1(Route Table

/api/v1
/v2) style vs fill:#f54,stroke:#233,stroke-width:4px end subgraph valid vsValid[Virtual Service

*.petclinic.com] -->|delegate /api prefix | rt1Valid(Route Table

/api/v1
/api/v2) style vsValid fill:#0DFF00,stroke:#233,stroke-width:4px end

Gloo Gateway will flatten the non-delegated routes defined in config tree down to a single Proxy object, such that:

graph LR; vs[Virtual Service

*.petclinic.com] -->|delegate /api prefix | rt1(Route Table

/api/pets
/api/vets) vs -->|route /pharmacy | us1(Upstream

pharmacy-svc.petstore.cluster.svc.local:80) rt1 -->|route /api/pets | us2(Upstream

pet-svc.petstore.cluster.svc.local:80) rt1 -->|route /api/vets | us3(Upstream

vet-svc.petstore.cluster.svc.local:80) style vs fill:#0DFF00,stroke:#233,stroke-width:4px style us1 fill:#f9f,stroke:#333,stroke-width:4px style us2 fill:#f9f,stroke:#333,stroke-width:4px style us3 fill:#f9f,stroke:#333,stroke-width:4px

Would become:

graph LR; px{Proxy

*.petclinic.com} --> |route /api/pets | us2(Upstream

pet-svc.petstore.cluster.svc.local:80) px -->|route /api/vets | us3(Upstream

vet-svc.petstore.cluster.svc.local:80) px --> |route /pharmacy | us1(Upstream

pharmacy-svc.petstore.cluster.svc.local:80) style px fill:#0DFFDD,stroke:#333,stroke-width:4px style us1 fill:#f9f,stroke:#333,stroke-width:4px style us2 fill:#f9f,stroke:#333,stroke-width:4px style us3 fill:#f9f,stroke:#333,stroke-width:4px

Example Configuration

The delegateAction object (which can be defined on routes, both on VirtualServices and RouteTables) can assume one of two forms:

  1. ref: delegates to a specific route table;
  2. selector: delegates to all the route tables that match the selection criteria.

In the next two sections we will see examples of both these delegation actions.

Delegation via direct reference

A complete configuration that uses a delegateAction which references specific route tables might look as follows:

A root-level VirtualService which delegates routing decisions to the a-routes and b-routes RouteTables. Please note that routes with delegateActions can only use a prefix matcher.

apiVersion: gateway.solo.io/v1
kind: VirtualService
metadata:
  name: 'example'
  namespace: 'gloo-system'
spec:
  virtualHost:
    domains:
    - 'example.com'
    routes:
    - matchers:
       - prefix: '/a' # delegate ownership of routes for `example.com/a`
      delegateAction:
        ref:
          name: 'a-routes'
          namespace: 'a'
    - matchers:
       - prefix: '/b' # delegate ownership of routes for `example.com/b`
      delegateAction:
        ref:
          name: 'b-routes'
          namespace: 'b'

A RouteTable which defines two routes.

apiVersion: gateway.solo.io/v1
kind: RouteTable
metadata:
  name: 'a-routes'
  namespace: 'a'
spec:
  routes:
    - matchers:
        # the path matchers in this RouteTable must begin with the prefix `/a/`
       - prefix: '/a/1'
      routeAction:
        single:
          upstream:
            name: 'foo-upstream'
    - matchers:
       - prefix: '/a/2'
      routeAction:
        single:
          upstream:
            name: 'bar-upstream'

A RouteTable which both defines a route and delegates to another RouteTable.

apiVersion: gateway.solo.io/v1
kind: RouteTable
metadata:
  name: 'b-routes'
  namespace: 'b'
spec:
  routes:
    - matchers:
        # the path matchers in this RouteTable must begin with the prefix `/b/`
       - regex: '/b/3'
      routeAction:
        single:
          upstream:
            name: 'baz-upstream'
    - matchers:
       - prefix: '/b/c/'
      # routes in the RouteTable can perform any action, including a delegateAction
      delegateAction:
        ref:
          name: 'c-routes'
          namespace: 'c'

A RouteTable which is a child of another route table.

apiVersion: gateway.solo.io/v1
kind: RouteTable
metadata:
  name: 'c-routes'
  namespace: 'c'
spec:
  routes:
    - matchers:
       - exact: '/b/c/4'
      routeAction:
        single:
          upstream:
            name: 'qux-upstream'

The above configuration can be visualized as:

graph LR; vs[Virtual Service

example.com] vs -->|delegate /a prefix | rt1(Route Table

/a/1
/a/2) vs -->|delegate /b prefix | rt2(Route Table

/b/1
/b/2) rt1 -->|route /a/1 | us1(Upstream

foo-upstream) rt1 -->|route /a/2 | us2(Upstream

bar-upstream) rt2 -->|route /b/3 | us3(Upstream

baz-upstream) rt2 -->|route /b/c | rt3(Route Table

/b/c/4) rt3 -->|route /b/c/4 | us4(Upstream

qux-upstream) style vs fill:#0DFF00,stroke:#233,stroke-width:4px style us1 fill:#f9f,stroke:#333,stroke-width:4px style us2 fill:#f9f,stroke:#333,stroke-width:4px style us3 fill:#f9f,stroke:#333,stroke-width:4px style us4 fill:#f9f,stroke:#333,stroke-width:4px

And would result in the following Proxy:

graph LR; px{Proxy

example.com} style px fill:#0DFFDD,stroke:#333,stroke-width:4px px -->|route /a/1 | us1(Upstream

foo-upstream) px -->|route /a/2 | us2(Upstream

bar-upstream) px -->|route /b/3 | us3(Upstream

baz-upstream) px -->|route /b/c/4 | us4(Upstream

qux-upstream) style us1 fill:#f9f,stroke:#333,stroke-width:4px style us2 fill:#f9f,stroke:#333,stroke-width:4px style us3 fill:#f9f,stroke:#333,stroke-width:4px style us4 fill:#f9f,stroke:#333,stroke-width:4px

Delegation via route table selector

By using a RouteTableSelector , a route can delegate to multiple route tables. You can specify three types of selection criteria (labels and expressions cannot be used together):

  1. labels: if present, Gloo Gateway will select route tables whose labels match the specified ones;
  2. expressions: if present, Gloo Gateway will select according to the expression (adhering to the same semantics as kubernetes label selectors). An example config for this selection model follows here;
  3. namespaces: if present, Gloo Gateway will select route tables in these namespaces. If omitted, Gloo Gateway will only select route tables in the same namespace as the resource (Virtual Service or Route Table) that owns this selector. The reserved value * can be used to select Route Tables in all namespaces watched by Gloo Gateway.

If a RouteTableSelector matches multiple route tables and the route tables do not specify different weights, Gloo Gateway will sort the routes which belong to those tables to avoid short-circuiting (e.g. having a route with a /foo prefix matcher coming before a route with a /foo/bar one). The sorting occurs by descending specificity: routes with longer paths will come first, and in case of equal paths, precedence will be given to the route that defines the more restrictive matchers. The algorithm used for sorting the routes can be found here. In this scenario, Gloo Gateway will also alert the user by adding a warning to the status of the parent resource (the one that specifies the RouteTableSelector).

Please see the Route Table weight section below for more information about how to control the ordering of your delegated routes.

A complete configuration might look as follows:

A root-level VirtualService which delegates routing decisions to any RouteTables in the gloo-system namespace that contain the domain: example.com label.

apiVersion: gateway.solo.io/v1
kind: VirtualService
metadata:
  name: 'example'
  namespace: 'gloo-system'
spec:
  virtualHost:
    domains:
    - 'example.com'
    routes:
    - matchers:
       - prefix: '/' # delegate ownership of all routes for `example.com`
      delegateAction:
        selector:
          labels:
            domain: example.com
          namespaces:
          - gloo-system

Two RouteTables which match the selection criteria:

apiVersion: gateway.solo.io/v1
kind: RouteTable
metadata:
  name: 'a-routes'
  namespace: 'gloo-system'
  labels:
    domain: example.com
spec:
  weight: 20
  routes:
    - matchers:
        # the path matchers in this RouteTable can begin with any prefix
       - prefix: '/a'
      routeAction:
        single:
          upstream:
            name: 'foo-upstream'
apiVersion: gateway.solo.io/v1
kind: RouteTable
metadata:
  name: 'a-b-routes'
  namespace: 'gloo-system'
  labels:
    domain: example.com
spec:
  weight: 10
  routes:
    - matchers:
        # the path matchers in this RouteTable can begin with any prefix
       - regex: '/a/b'
      routeAction:
        single:
          upstream:
            name: 'bar-upstream'

The above configuration can be visualized as:

graph LR; vs[Virtual Service

example.com] vs -->|delegate / prefix | rt1(Route Table

/a) vs -->|delegate / prefix | rt2(Route Table

/a/b/) rt1 -->|route /a | us1(Upstream

foo-upstream) rt2 -->|route /a/b | us3(Upstream

bar-upstream) style vs fill:#0DFF00,stroke:#233,stroke-width:4px style us1 fill:#f9f,stroke:#333,stroke-width:4px style us3 fill:#f9f,stroke:#333,stroke-width:4px

And would result in the following Proxy:

graph LR; px{Proxy

example.com} style px fill:#0DFFDD,stroke:#333,stroke-width:4px px -->|route /a/b | us1(Upstream

bar-upstream) px -->|route /a | us2(Upstream

foo-upstream) style us1 fill:#f9f,stroke:#333,stroke-width:4px style us2 fill:#f9f,stroke:#333,stroke-width:4px

Route Table Selector Expression

To allow for flexible route table label matching in more complex architecture scenarios, the expressions attribute of the delegateAction.selector field can be used as such:

apiVersion: gateway.solo.io/v1
kind: VirtualService
metadata:
  name: 'example'
  namespace: 'gloo-system'
spec:
  virtualHost:
    domains:
    - 'example.com'
    routes:
    - matchers:
       - prefix: '/'
      delegateAction:
        selector:
          # 'expressions' and 'labels' cannot coexist within a single selector
          expressions:
            - key: domain
              operator: In
              values:
                - example.com
            # Do not include route tables exposing the 'shouldnotexist' label
            - key: shouldnotexist
              operator: !
          namespaces:
          - gloo-system

Note that candidate route tables must match all selector expressions (logical AND) to be selected.

Route Table weight

As you might have noticed, we specified a weight attribute on the above route tables. This attribute can be used to determine the order in which the routes will appear on the final Proxy resource when multiple route tables match a RouteTableSelector. The field is optional; if no value is specified, the weight defaults to 0 (zero). Gloo Gateway will process the route tables matched by a selector in ascending order by weight and collect the routes of each route table in the order they are defined.

In the above example, we want the /a/b route to come before the /a route, to avoid the latter one short-circuiting the former; hence, we set the weight of the a-b-routes table to 10 and the weight of the a-routes table to 20. As you can see in the diagram above, the resulting Proxy object defines the routes in the desired order.

Matcher restrictions

The Gloo Gateway route delegation model imposes some restrictions on the virtual service and parent route table’s matchers (i.e., any resource delegating routing config to another route table). Most notably, parent matchers must have only a single prefix matcher. Further, any parent matcher must have its own prefix start with the same prefix as its parent (if any).

Further, in versions prior to Gloo Gateway 1.5.0-beta21, parent matchers cannot use header, query parameter, or method matchers. In more recent Gloo Gateway versions, parent matchers can now use those matchers so long as their child route tables have headers, query parameters, and methods that are a superset of those defined on the parent. In Gloo Gateway versions Gloo Gateway 1.5.0-beta25 and higher, the inheritableMatchers boolean field was added to virtual services and route tables, which allows users to enable matcher inheritance for non-path matchers from parents (rather than requiring the whole subset enumerated on any chlidren).

In all versions of Gloo Gateway, the leaf route table can use any kind of path matcher, so long as it begins with the same prefix as its parent.

Learn more

Explore Gloo Gateway’s Routing API in the API documentation:

Please submit questions and feedback to the solo.io slack channel, or open an issue on GitHub.