☰
Navigation :

Workspace configuration

You configure two Gloo custom resources to achieve multitenancy: Workspace and WorkspaceSettings.

The WorkspaceSettings custom resource is separate from the Workspace so that you can more easily delegate its configuration to each team in your org. Settings include import and export, service isolation, and east-west gateway selection, if applicable.

Feature to set up Workspace WorkspaceSettings
Workspace name ✅ -
Cluster the workspace includes ✅ -
Namespace the workspace includes ✅ -
Gloo CR restrictions by namespace ✅ -
Service isolation within workspace - ✅
East-west gateway selection - ✅
Resource sharing across workspaces with import and export - ✅

The following diagram shows how different personas might configure workspaces and workspace settings.

Figure of workspace and workspace settings.

Workspace configuration

Review the configuration guidance for creating the workspace, selecting namespaces and clusters, and commonly shared workspaces. For more information, see the API docs.

Workspace metadata

Name your workspace for the tenant that it represents, such as the Ops team, Finance department, or Web apps. You create all of your workspaces in the gloo-mesh namespace of the management cluster.

Cluster and namespace boundaries for workspaces

A Kubernetes namespace can belong to only one workspace. A single workspace, however, can have Kubernetes namespaces from many different clusters. In the workspace configuration, you group together the Kubernetes namespaces and clusters for the workspace to manage. Review the following selection options.

The simplest way to select the cluster and namespace is by name. In the following example:

  • Clusters: Only the cluster1 cluster is selected.
  • Namespaces: Only the web-ui namespace in cluster1 is selected.

The downfall of this approach is that the workspace is static and does not update dynamically as new namespaces and clusters are added. Also, if cluster1 does not have the web-ui namespace, then the workspace and Gloo resources do not work until the web-ui namespace is created.

apiVersion: admin.gloo.solo.io/v2
kind: Workspace
metadata:
  name: web-team
  namespace: gloo-mesh
spec:
  workloadClusters:
  - name: 'cluster1'    
    namespaces:
    - name: 'web-ui'

Wildcards help your workspaces automatically scale as you add clusters. In the following example:

  • Clusters: A * wildcard selects any cluster.
  • Namespaces: Only the web-ui namespace in cluster1 is selected.

Say that you have three clusters, but only cluster1 and cluster2 have a web-ui namespace. In this case, cluster1 and cluster2 become part of the workspace, but cluster3 does not. Later, if you create a web-ui namespace in cluster3, that cluster automatically becomes part of the workspace, too.

apiVersion: admin.gloo.solo.io/v2
kind: Workspace
metadata:
  name: web-team
  namespace: gloo-mesh
spec:
  workloadClusters:
  - name: '*'            # any cluster that has the namespace 'web-ui'
    namespaces:
    - name: web-ui

Labels give you more fine-grained control than wildcards, while still letting your workspaces automatically scale up. Labels are also a good way to organize resources that are dedicated to a particular purpose. You might want to separate workspaces by region for a backup failover environment.

In the following example:

  • Clusters: The workspace includes only workload clusters with the region: us-east label.
  • Namespaces: Only namespaces that are named web in those region: us-east clusters are included in the workspace.
  • Label: The workspace includes an optional label of gloo.solo.io/regional: 'true'. Later, other Gloo resources might use this label to identify the workspace as regional.
kubectl apply -n gloo-mesh -f- <<EOF
apiVersion: admin.gloo.solo.io/v2
kind: Workspace
metadata:
  name: web-team
  namespace: gloo-mesh
  labels:
    gloo.solo.io/regional: 'true'
spec:
  workloadClusters:
    - selector:
        region: us-east
      namespaces:
        - name: web
EOF

Be careful if you use a wildcard or label to select namespaces! Sometimes, you might be tempted to select namespaces by wildcard or by label. For example, you might want everything in cluster1 to belong to an Ops workspace, or everything that is within an Istio mesh to belong to a Mesh workspace.

However, this approach can yield unexpected results. Consider the scenario in which the istiod namespace runs in each of your registered clusters. The Ops workspace selects everything in cluster1, including istiod. However, the Mesh workspace also tries to select every namespace across all clusters with the istio-injection: enabled label, including istiod. The same Kubernetes namespace can belong to only one workspace, so both the Ops and Mesh workspace encounter errors.

apiVersion: admin.gloo.solo.io/v2
kind: Workspace
metadata:
  name: ops
  namespace: gloo-mesh
spec:
  workloadClusters:
  - name: 'cluster1'     # all namespaces in cluster1
    namespaces:
    - name: '*'
---
apiVersion: admin.gloo.solo.io/v2
kind: Workspace
metadata:
  name: mesh
  namespace: gloo-mesh
spec:
  workloadClusters:
  - name: '*'            # all namespaces in all clusters with label istio-injection=enabled
    namespaces:
    - name: '*'
      labels:
        istio-injection: enabled

Workspace configuration in the management plane

Consider keeping all your Gloo configuration in a single management cluster. This way, only users with access to the management cluster can create Gloo configuration, which simplifies access control. Also, if an issue occurs, the Gloo resources are easier to find.

apiVersion: admin.gloo.solo.io/v2
kind: Workspace
metadata:
  name: web-team
  namespace: gloo-mesh
spec:
  workloadClusters:
  - name: 'mgmt'               # Gloo configuration only namespace in the management plane
    namespaces:
    - name: 'web-team-config'
  - name: '*'                  # Any cluster that has the namespace 'web-ui'
      namespaces:
      - name: 'web-ui'
    configEnabled: false       # Prevent Gloo from reading configuration in data plane
---
apiVersion: admin.gloo.solo.io/v2
kind: Workspace
metadata:
  name: apis-team
  namespace: gloo-mesh
spec:
  workloadClusters:
  - name: 'mgmt'               # Gloo configuration only namespace in the management plane
    namespaces:
    - name: 'apis-team-config'
  - name: 'cluster1'           # APIs namespace in cluster1
      namespaces:
      - name: 'apis'
    configEnabled: false       # Prevent Gloo from reading configuration in data plane

Examples of common workspaces

Review the following examples of common workspaces. Also, refer to the Example workspace YAML configuration.

A simple workspace scenario is to use a workspace for each team. But what if a team owns several different apps in different production environments? You can create a Gloo workspace for each app. This approach might mean that multiple microservices and Kubernetes namespaces are grouped together into a single workspace.

If you want to apply policies and control access for each microservice, you might even make a separate Gloo workspace for each Kubernetes namespace.

In the following example:

  • The web-team workspace includes only one data plane namespace, web-ui. This namespace has all the resources to run the team's frontend app.
  • The backend-apis-team workspace includes several data plane namespaces: ads, cart, catalog, currency, and recs. Each of these namespaces has a separate microservice. Together, these microservices provide all of the backend functionality that the frontend app serves to the end users. By grouping all of these microservices into one Gloo workspace, the backend team can consistently configure sharing and security features with the frontend team.
apiVersion: admin.gloo.solo.io/v2
kind: Workspace
metadata:
  name: web-team
  namespace: gloo-mesh
spec:
  workloadClusters:
  - name: 'mgmt'
    namespaces:
    - name: web-team
  - name: '*'
    namespaces:
    - name: web-ui
---
apiVersion: admin.gloo.solo.io/v2
kind: Workspace
metadata:
  name: backend-apis-team
  namespace: gloo-mesh
spec:
  workloadClusters:
  - name: 'mgmt'
    namespaces:
    - name: backend-apis-team
  - name: '*'
    namespaces:
    - name: ads
    - name: cart
    - name: catalog
    - name: currency
    - name: recs

To use Gloo to configure ingress traffic, the ingress gateway proxy must be part of a workspace. Usually, you want the gateway proxy in a separate workspace from your apps so that you can manage the gateway proxy separately.

apiVersion: admin.gloo.solo.io/v2
kind: Workspace
metadata:
  name: ops-team
  namespace: gloo-mesh
spec:
  workloadClusters:
  - name: 'cluster1'            # istio-ingress and istio-eastwest in cluster1
    namespaces:
    - name: 'istio-ingress'

Istio east-west gateways are used for multicluster routing within Gloo Mesh. Because these gateways are for internal-only traffic, you might not need the east-west gateways in the same workspace as your ingress gateway. You might create a separate east-west gateway workspace, or create an internal or Ops team workspace for the east-west gateway and other internal-only components like Gloo Platform add-ons.

apiVersion: admin.gloo.solo.io/v2
kind: Workspace
metadata:
  name: gloo-internal
  namespace: gloo-mesh
spec:
  workloadClusters:
  - name: '*'            # selects istio-eastwest namespaces in all clusters
    namespaces:
    - name: 'istio-eastwest'

Gloo add-ons are a set of services that add functionality to your Gloo Platform environment. These services include rate limiting and external auth such as OpenID Connect (OIDC). Typically, these services are deployed on each workload cluster in the gloo-mesh-addons namespace. However, you might also deploy your own.

No matter your setup, certain Gloo policies depend on rate limiting and external auth services. To use these policies, the add-ons must be in a workspace. If you have several teams and workspaces, create a separate workspace for the add-ons, or include the add-ons in an internal or Ops team workspace.

apiVersion: admin.gloo.solo.io/v2
kind: Workspace
metadata:
  name: gloo-addons
  namespace: gloo-mesh
spec:
  workloadClusters:
  - name: '*'               ## any cluster that has the namespace 'gloo-mesh-addons'
    namespaces:
    - name: 'gloo-mesh-addons'

The following example creates the global workspace. You might use a global workspace when you are first trying out or migrating to Gloo Platform. For production use cases, however, create separate workspaces.

Note that you must name the global workspace global and create it in the gloo-mesh namespace of the management cluster.

  • Clusters: The workspace includes all workload clusters by using a regex to match all cluster names.
  • Namespaces: The workspace includes all namespaces in all workload clusters. To avoid performance impacts, do not use wildcard selectors for production scenarios.
kubectl apply -n gloo-mesh -f- <<EOF
apiVersion: admin.gloo.solo.io/v2
kind: Workspace
metadata:
  name: global
  namespace: gloo-mesh
spec:
  workloadClusters:
  - name: '*'
    namespaces:
    - name: '*'
EOF

Note: In the next section, you configure workspace settings. To create global workspace settings that apply to all namespaces in all clusters, you can create a WorkspaceSettings resource that is also named global in the gloo-mesh namespace of the management cluster.

Workspace settings configuration

As the platform admin, you can delegate workspace settings to each team owner. This way, each team can make its own decisions about service isolation, east-west gateways, and resource sharing. The team owners can work together to determine import and export settings so that they can share resources with one another.

Service isolation

To reduce network traffic and “noisy neighbor” concerns, you might want to isolate your services. In Kubernetes, your services are not isolated by default, but instead can communicate with other services on the same node. Similarly, namespaces are also not isolated from other namespaces in a cluster. To isolate services, you must set up additional resources like network policies from the container networking interface (CNI) plug-in.

With Gloo workspaces, you can quickly enforce service isolation at the workspace level. The workspace becomes the networking boundary. For even more fine-grained support, you can disable service isolation and use Gloo access policies to set up a zero-trust architecture instead.

Enabling service isolation at the workspace level

When you enable service isolation, Gloo creates Istio authorization policies and peer authentication resources in each namespace of the workspace. These resources allow communication across services within the workspace, but deny all other communication from outside the workspace. Even more convenient, if you set up import and export with other workspaces, Gloo allows communication from those workspaces automatically.

In a service mesh environment, service isolation also lets you quickly set up traffic encryption for all of your services within the workspace. Connections across the services are secured via mutual TLS (mTLS).

Click the following Service isolation config example to expand an example configuration file. For more information, see the API docs.

Service isolation config example

Disabling service isolation as part of a zero-trust architecture

Instead of creating a “blanket” isolation policy for your entire workspace, you might want more fine-grained access control. For example, if you also have a Gloo Mesh Enterprise license, you can build a zero-trust architecture from the ground up by starting with a “deny-all” policy and then allowing access for one resource at a time.

  1. Disable service isolation at the workspace level. 
    apiVersion: admin.gloo.solo.io/v2
kind: WorkspaceSettings
metadata:
  name: apis-team
  namespace: apis-team-config
spec:
  exportTo:
  - workspaces:
    - name: web-team
  options:
    serviceIsolation:
      enabled: false
  2. Use Gloo access policies to scope access to your resources.

Proxy trimming

Service isolation discovers and creates a lot of Istio resources pretty quickly. As such, you might notice performance impacts. For large environments, you can trim all the Istio sidecar proxy configs within the workspace to eliminate all outbound destinations.

What happens when I trim all proxy configuration?

Without the outbound destination config, the Istio sidecar proxies cannot talk to other destinations in the workspace by default. Then, you can add back in the proxy config for only the destinations that you want the sidecar proxies to talk to. This way, you reduce the size of the proxy config to improve performance.

Looking for fine-grained access control? If you also have a Gloo Mesh Enterprise license, you can use an access policy to enforce zero-trust networking.

How do I add back in proxy configuration for the destinations I want?

You can choose between enabling the serviceIsolation.trimProxyConfig setting at the workspace level to add in all destinations or by selecting specific destinations in a TrimProxyConfigPolicy.

By using the policy instead of the service isolation workspace-level setting, you get several additional benefits:

  1. The workspace settings method requires you to enable service isolation. With the policy, you can still trim proxy configs when you don't need service isolation. For example, you might not want to limit or explicitly allowlist all of the services within your workspace.
  2. The workspace settings method trims the proxy configs only at the workspace level. With the policy, you can fine-tune the proxy config even within the workspace. For example, you might have one or two workspaces with thousands of services, and want to trim the proxies to select services within the workspace.
  3. You can start with a zero-trust, “allow none” approach. Instead of having configuration for all destinations in the sidecar, you can use the policy to specify only allowed destinations one at a time.

If you use both the workspace setting and the policy, the proxy configs for all of the selected destinations are allowed.

Example steps to trim proxy config

  1. Trim all of the Istio sidecar proxies in your workspace. 
       apiVersion: admin.gloo.solo.io/v2
   kind: WorkspaceSettings
   metadata:
     name: apis-team
     namespace: apis-team-config
   spec:
     exportTo:
     - workspaces:
       - name: web-team
     options:
       trimAllProxyConfig: true
  2. Add destinations back into the proxy config, either by enabling the serviceIsolation.trimProxyConfig setting at the workspace level to add in all destinations or by selecting specific destinations in a TrimProxyConfigPolicy.
    • Workspace-wide trim proxy setting: For more information, see Service isolation or the API docs. 
           apiVersion: admin.gloo.solo.io/v2
     kind: WorkspaceSettings
     metadata:
       name: apis-team
       namespace: apis-team-config
     spec:
       exportTo:
       - workspaces:
         - name: web-team
       options:
         trimAllProxyConfig: true
         serviceIsolation:
           enabled: true
           trimProxyConfig: true      # scope discovery to services within or imported to the workspace
    • Specific destinations by using a policy service: For more information, see the trim proxy config policy guide.
  3. Verify that the Istio sidecar proxy config has the destinations that you allowed. 
    kubectl get sidecar -n backend-apis -o yaml

    Example of the Istio sidecar output:

    apiVersion: networking.istio.io/v1beta1
kind: Sidecar
metadata:
  name: sidecar-backend-v1-backend-apis-cluster1-backend-apis-team
  namespace: backend-apis
spec:
  egress:
  - hosts:
    - '*/backend.backend-apis.svc.cluster.local'
    - '*/backend.solo-io.mesh'
    - '*/frontend.backend-apis.svc.cluster.local'
    - '*/istio-eastwestgateway.istio-gateways.svc.cluster.local'
    - '*/istio-in

East-west gateway selection

In a standard Gloo deployment, you create an Istio east-west gateway per workload cluster to enable multicluster routing. In more advanced scenarios, you might have multiple east-west gateways per cluster, such as to further isolate traffic to certain apps. In such cases, you can choose the east-west gateways that Gloo uses for multicluster routing for a workspace.

By default, Gloo looks for a port named tls on the east-west gateway. However, you can override this behavior in the workspace settings, such as in the following example.

apiVersion: admin.gloo.solo.io/v2
kind: WorkspaceSettings
metadata:
  name: web-team
  namespace: web-team-config
spec:
  options:
    eastWestGateways:
    - selector:
        labels:
          istio: eastwestgateway       # select any gateway with this label
        # port:
        #   name: cross-cluster-mtls   # optional override of named port for mutual tls

Resource sharing with import and export settings

By default, Gloo resources apply throughout the workspace. For example, you might have a failover policy that you want all of the apps in the Web team's workspace to use. Each app in the Web team's workspace can use the same failover policy, even if the app runs in a different namespace and cluster than where the failover policy is created.

But what if you need to use services in a different team's workspace?

With workspace settings, you can easily set up which services in your workload clusters can talk to each other and how you want to share Gloo resources. Both workspaces must agree to share resources by configuring the import and export fields in each workspace settings. For more information, see Import and export resources across workspaces.

Example workspace YAML configuration

The following diagram and GitHub Gist example configures the settings for three workspaces. For more information, see Import and export resources across workspaces.

Figure of workspaces as configuration boundaries.