About workspace configuration

The WorkspaceSettings custom resource is separate from the Workspace so that you can more easily delegate its configuration to each team in your org. Settings include import and export, service isolation, and east-west gateway selection, if applicable.

  • Workspace: A workspace groups together the Kubernetes namespaces and clusters that a team needs access to. The Gloo, Istio, and Kubernetes resources that your team creates in a workspace can be used by all the namespaces in the workspace, across clusters. The platform admin typically creates the workspace.
  • WorkspaceSettings: In the workspace settings, your team configures how its services within the workspace can communicate with each other. The settings help control how the team’s resources interact with resources from other workspaces.

Multitenancy feature by custom resource

The following table summarizes the features you configure by custom resource.

Feature to set upWorkspaceWorkspaceSettings
Workspace name✅-
Cluster the workspace includes✅-
Namespace the workspace includes✅-
Gloo CR restrictions by namespace✅-
Service isolation within workspace-✅
East-west gateway selection-✅
Resource sharing across workspaces with import and export-✅

Keep reading this page for best practices about workspace, workspace settings, and example YAML configuration files.

Example persona-driven diagram

The following diagram shows how different personas might configure workspaces and workspace settings.

  • The platform admin creates the Ops and App team workspaces in the management cluster. She also creates an ops-mgmt and app-mgmt namespace in the management cluster for the teams’ Gloo configuration.
  • The Ops team owner creates the Ops team’s workspace settings and other Gloo custom resources in the ops-mgmt namespace in the management cluster. The Ops workspace is set up to manage resources like the Istio ingress and east-west gateways and Gloo add-ons like the external auth and rate limiting services for each cluster.
  • The App team owner creates the App team’s workspace settings and other Gloo custom resources in the app-mgmt namespace in the management cluster. The App workspace is set up to manage the frontend and backend apps.
Figure: Workspace and workspace settings.
Figure: Workspace and workspace settings.

Workspace configuration

Review the configuration guidance for creating the workspace, selecting namespaces and clusters, and commonly shared workspaces. For more information, see the API docs.

Workspace metadata

Name your workspace for the tenant that it represents, such as the Ops team, Finance department, or Web apps. You create all of your workspaces in the gloo-mesh namespace of the management cluster.

Cluster and namespace boundaries for workspaces

A Kubernetes namespace can belong to only one workspace. A single workspace, however, can have Kubernetes namespaces from many different clusters. In the workspace configuration, you group together the Kubernetes namespaces and clusters for the workspace to manage. Review the following selection options.

The simplest way to select the cluster and namespace is by name. In the following example:

  • Clusters: Only the cluster1 cluster is selected.
  • Namespaces: Only the web-ui namespace in cluster1 is selected.

The downfall of this approach is that the workspace is static and does not update dynamically as new namespaces and clusters are added. Also, if cluster1 does not have the web-ui namespace, then the workspace and Gloo resources do not work until the web-ui namespace is created.

  apiVersion: admin.gloo.solo.io/v2
kind: Workspace
metadata:
  name: web-team
  namespace: gloo-mesh
spec:
  workloadClusters:
  - name: 'cluster1'    
    namespaces:
    - name: 'web-ui'

Wildcards help your workspaces automatically scale as you add clusters. In the following example:

  • Clusters: A * wildcard selects any cluster.
  • Namespaces: Only the web-ui namespace in cluster1 is selected.

Say that you have three clusters, but only cluster1 and cluster2 have a web-ui namespace. In this case, cluster1 and cluster2 become part of the workspace, but cluster3 does not. Later, if you create a web-ui namespace in cluster3, that cluster automatically becomes part of the workspace, too.

  apiVersion: admin.gloo.solo.io/v2
kind: Workspace
metadata:
  name: web-team
  namespace: gloo-mesh
spec:
  workloadClusters:
  - name: '*'            # any cluster that has the namespace 'web-ui'
    namespaces:
    - name: web-ui

Labels give you more fine-grained control than wildcards, while still letting your workspaces automatically scale up. Labels are also a good way to organize resources that are dedicated to a particular purpose. You might want to separate workspaces by region for a backup failover environment.

In the following example:

  • Clusters: The workspace includes only workload clusters with the region: us-east label.
  • Namespaces: Only namespaces that are named web in those region: us-east clusters are included in the workspace.
  • Label: The workspace includes an optional label of gloo.solo.io/regional: 'true'. Later, other Gloo resources might use this label to identify the workspace as regional.
  kubectl apply --context $MGMT_CONTEXT -n gloo-mesh -f- <<EOF
apiVersion: admin.gloo.solo.io/v2
kind: Workspace
metadata:
  name: web-team
  namespace: gloo-mesh
  labels:
    gloo.solo.io/regional: 'true'
spec:
  workloadClusters:
    - selector:
        region: us-east
      namespaces:
        - name: web
EOF

Be careful if you use a wildcard or label to select namespaces! Sometimes, you might be tempted to select namespaces by wildcard or by label. For example, you might want everything in cluster1 to belong to an Ops workspace, or everything that is within an Istio mesh to belong to a Mesh workspace.

However, this approach can yield unexpected results. Consider the scenario in which the istiod namespace runs in each of your registered clusters. The Ops workspace selects everything in cluster1, including istiod. However, the Mesh workspace also tries to select every namespace across all clusters with the istio-injection: enabled label, including istiod. The same Kubernetes namespace can belong to only one workspace, so both the Ops and Mesh workspace encounter errors.

  apiVersion: admin.gloo.solo.io/v2
kind: Workspace
metadata:
  name: ops
  namespace: gloo-mesh
spec:
  workloadClusters:
  - name: 'cluster1'     # all namespaces in cluster1
    namespaces:
    - name: '*'
---
apiVersion: admin.gloo.solo.io/v2
kind: Workspace
metadata:
  name: mesh
  namespace: gloo-mesh
spec:
  workloadClusters:
  - name: '*'            # all namespaces in all clusters with label istio-injection=enabled
    namespaces:
    - name: '*'
      labels:
        istio-injection: enabled

Conflicting workspaces

Workspaces can select multiple Kubernetes namespaces. But what happens when different workspaces select the same Kubernetes namespace? In general, the oldest workspace by creation timestamp takes precedence and gets the namespace. The newer conflicting workspaces do not get a namespace, which means that they might not translate any Gloo custom resources or exhibit other such unexpected behavior until the conflict is resolved. Review the following example scenarios and ways to avoid conflicting workspaces.

Consider the following scenario:

  • Existing workspace alpha has namespace-1 and namespace-2.
  • Workspace beta gets created with namespace-2 and namespace-3.

In this scenario, the following happens:

  • The alpha workspace continues to function as normal. In its status, the workspace has a warning to let you know that another workspace is trying to select the same namespace-2.
  • The beta workspace works only for the non-conflicting namespace-3. Workspace-bound resources such as policies do not apply to resources in namespace-2. In its status, the workspace has a warning to let you know that another workspace takes precedence for namespace-2.

Consider the following scenario:

  • Existing workspace alpha has namespace-1.
  • Workspace beta has namespace-2.
  • Workspace beta is updated to include namespace-1.

In this scenario, the following happens:

  • The alpha workspace continues to function as normal. In its status, the workspace has a warning to let you know that another workspace is trying to select the same namespace-1.
  • The beta workspace continues to function as normal for namespace-2. However, the beta workspace does not include namespace-1. In its status, the workspace has a warning to let you know that another workspace takes precedence for namespace-1.

Consider the following scenario:

  • Existing workspace alpha has namespace-1.
  • Workspace beta has namespace-2.
  • Workspace alpha is updated to include namespace-2.

In this scenario, the following happens:

  • The alpha workspace continues to function as normal for namespace-1. Because alpha existed earlier than beta, the alpha workspace also takes over any resources in namespace-2, which means that workspace-bound resources such as policies in the alpha workspace now apply to resources in namespace-2. In its status, the workspace has a warning to let you know that another workspace is trying to select the same namespace-2.
  • The beta workspace no longer includes namespace-2. Any workspace-bound resources like policies might stop applying or get removed. Because it has no namespaces, the beta workspace enters an error state until you resolve the issue.

Consider the following scenario:

  • Existing workspace alpha has namespace-1 with workspace settings wss-alpha.
  • Workspace beta has namespace-2 with workspace settings wss-beta.
  • Workspace alpha is updated to include namespace-2.

In this scenario, the following happens:

  • The alpha workspace continues to function as normal for namespace-1. Because alpha existed earlier than beta, the alpha workspace also takes over any resources in namespace-2, which means that workspace-bound resources such as policies in the alpha workspace now apply to resources in namespace-2. Additionally, workspace alpha ignores the newly detected wss-beta in namespace-2. In its status, the workspace has a warning to let you know that another workspace is trying to select the same namespace-2.
  • The beta workspace no longer includes namespace-2. Additionally, the workspace enters an error state because it no longer has a workspace settings. Any Gloo resources and their translated resources are removed, which can cause errors in your network. In its status, the workspace has a warning to let you know that it no longer has a workspace settings due to a conflicting namespace.

Avoiding conflicts

To avoid unexpected results from conflicting workspaces, consider the following best practices.

  • Create your most critical workspaces in order of importance. This way, the critical workspaces take precedence in case of conflict.
  • When practical, avoid selecting namespaces by wildcards or labels. Instead, specify each namespace by name. At a minimum, consider scoping any wildcards and enforcing naming conventions across your workspace teams. For example, you might have all namespaces that belong to a Web Team begin with frontend. Then, the workspace uses a frontend* wildcard.
  • Coordinate across workspace teams so that others are aware of any reserved names, reserved labels, or other naming conventions.

Workspace configuration in the management plane

Consider keeping all your Gloo configuration in a single management cluster. This way, only users with access to the management cluster can create Gloo configuration, which simplifies access control. Also, if an issue occurs, the Gloo resources are easier to find.

  • Management plane: The management cluster has a config namespace for each workspace. The team creates all their Gloo custom resources in this config namespace.
    • For the web-team, the web-team-config namespace on the mgmt cluster has all the Gloo config.
    • For the apis-team, the apis-team-config namespace on the mgmt cluster has all the Gloo config.
  • Data plane: For their application workloads, the team deploys their Kubernetes resources in the workload clusters, not the management cluster.
    • For the web-team, the web-ui namespace in any cluster runs their workloads.
    • For the apis-team, the apis namespace only in cluster1 runs their workloads.
    • The configEnabled: false setting is used to stop Gloo from reading Gloo configuration in the data plane namespaces. This way, the teams must create their Gloo configurations in the management plane.
  apiVersion: admin.gloo.solo.io/v2
kind: Workspace
metadata:
  name: web-team
  namespace: gloo-mesh
spec:
  workloadClusters:
  - name: 'mgmt'               # Gloo configuration only namespace in the management plane
    namespaces:
    - name: 'web-team-config'
  - name: '*'                  # Any cluster that has the namespace 'web-ui'
    namespaces:
    - name: 'web-ui'
    configEnabled: false       # Prevent Gloo from reading configuration in data plane
---
apiVersion: admin.gloo.solo.io/v2
kind: Workspace
metadata:
  name: apis-team
  namespace: gloo-mesh
spec:
  workloadClusters:
  - name: 'mgmt'               # Gloo configuration only namespace in the management plane
    namespaces:
    - name: 'apis-team-config'
  - name: 'cluster1'           # APIs namespace in cluster1
    namespaces:
    - name: 'apis'
    configEnabled: false       # Prevent Gloo from reading configuration in data plane

Examples of common workspaces

Review the following examples of common workspaces. Also, refer to the Example workspace YAML configuration.

A simple workspace scenario is to use a workspace for each team. But what if a team owns several different apps in different production environments? You can create a Gloo workspace for each app. This approach might mean that multiple microservices and Kubernetes namespaces are grouped together into a single workspace.

If you want to apply policies and control access for each microservice, you might even make a separate Gloo workspace for each Kubernetes namespace.

In the following example:

  • The web-team workspace includes only one data plane namespace, web-ui. This namespace has all the resources to run the team’s frontend app.
  • The backend-apis-team workspace includes several data plane namespaces: ads, cart, catalog, currency, and recs. Each of these namespaces has a separate microservice. Together, these microservices provide all of the backend functionality that the frontend app serves to the end users. By grouping all of these microservices into one Gloo workspace, the backend team can consistently configure sharing and security features with the frontend team.
  apiVersion: admin.gloo.solo.io/v2
kind: Workspace
metadata:
  name: web-team
  namespace: gloo-mesh
spec:
  workloadClusters:
  - name: 'mgmt'
    namespaces:
    - name: web-team
  - name: '*'
    namespaces:
    - name: web-ui
---
apiVersion: admin.gloo.solo.io/v2
kind: Workspace
metadata:
  name: backend-apis-team
  namespace: gloo-mesh
spec:
  workloadClusters:
  - name: 'mgmt'
    namespaces:
    - name: backend-apis-team
  - name: '*'
    namespaces:
    - name: ads
    - name: cart
    - name: catalog
    - name: currency
    - name: recs

To use Gloo to configure ingress traffic, the Istio ingress gateway must be part of a workspace. Usually, you want the ingress gateway in a separate workspace from your apps so that you can manage the ingress gateway separately.

  apiVersion: admin.gloo.solo.io/v2
kind: Workspace
metadata:
  name: ops-team
  namespace: gloo-mesh
spec:
  workloadClusters:
  - name: 'cluster1'            # istio-ingress and istio-eastwest in cluster1
    namespaces:
    - name: 'istio-ingress'

Istio east-west gateways are used for multicluster routing within Gloo Mesh. Because these gateways are for internal-only traffic, you might not need the east-west gateways in the same workspace as your ingress gateway. You might create a separate east-west gateway workspace, or create an internal or Ops team workspace for the east-west gateway and other internal-only components like Gloo add-ons.

  apiVersion: admin.gloo.solo.io/v2
kind: Workspace
metadata:
  name: gloo-internal
  namespace: gloo-mesh
spec:
  workloadClusters:
  - name: '*'            # selects istio-eastwest namespaces in all clusters
    namespaces:
    - name: 'istio-eastwest'

Gloo add-ons are a set of services that add functionality to your Gloo environment. These services include rate limiting and external auth such as OpenID Connect (OIDC). Typically, these services are deployed on each workload cluster in the gloo-mesh-addons namespace. However, you might also deploy your own.

No matter your setup, certain Gloo policies depend on rate limiting and external auth services. To use these policies, the add-ons must be in a workspace. If you have several teams and workspaces, create a separate workspace for the add-ons, or include the add-ons in an internal or Ops team workspace.

  apiVersion: admin.gloo.solo.io/v2
kind: Workspace
metadata:
  name: gloo-addons
  namespace: gloo-mesh
spec:
  workloadClusters:
  - name: '*'               ## any cluster that has the namespace 'gloo-mesh-addons’
    namespaces:
    - name: 'gloo-mesh-addons'

The following example creates the global workspace. You might use a global workspace when you are first trying out or migrating to Gloo Mesh Enterprise. For production use cases, however, create separate workspaces.

Note that you must name the global workspace global and create it in the gloo-mesh namespace of the management cluster.

  • Clusters: The workspace includes all workload clusters by using a regex to match all cluster names.
  • Namespaces: The workspace includes all namespaces in all workload clusters. To avoid performance impacts, do not use wildcard selectors for production scenarios.
  kubectl apply --context $MGMT_CONTEXT -n gloo-mesh -f- <<EOF
apiVersion: admin.gloo.solo.io/v2
kind: Workspace
metadata:
  name: global
  namespace: gloo-mesh
spec:
  workloadClusters:
  - name: '*'
    namespaces:
    - name: '*'
EOF

Note: In the next section, you configure workspace settings. To create global workspace settings that apply to all namespaces in all clusters, you can create a WorkspaceSettings resource that is also named global in the gloo-mesh namespace of the management cluster.

Workspace settings configuration

As the platform admin, you can delegate workspace settings to each team owner. This way, each team can make its own decisions about service isolation, east-west gateways, and resource sharing. The team owners can work together to determine import and export settings so that they can share resources with one another.

Service isolation

To reduce network traffic and “noisy neighbor” concerns, you might want to isolate your services. In Kubernetes, your services are not isolated by default, but instead can communicate with other services on the same node. Similarly, namespaces are also not isolated from other namespaces in a cluster. To isolate services, you must set up additional resources like network policies from the container networking interface (CNI) plug-in.

With Gloo workspaces, you can quickly enforce service isolation at the workspace level. The workspace becomes the networking boundary. For even more fine-grained support, you can disable service isolation and use Gloo access policies to set up a zero-trust architecture instead.

Enabling service isolation at the workspace level

When you enable service isolation, Gloo creates Istio authorization policies and peer authentication resources in each namespace of the workspace. These resources allow communication across services within the workspace, but deny all other communication from outside the workspace. Even more convenient, if you set up import and export with other workspaces, Gloo allows communication from those workspaces automatically.

In a service mesh environment, service isolation also lets you quickly set up traffic encryption for all of your services within the workspace. Connections across the services are secured via mutual TLS (mTLS).

Service isolation config example: For more information, see the API docs.

  apiVersion: admin.gloo.solo.io/v2
kind: WorkspaceSettings
metadata:
  name: apis-team
  namespace: apis-team-config
spec:
  exportTo:
  - workspaces:
    - name: web-team
  options:
    serviceIsolation:
      enabled: true

Disabling service isolation as part of a zero-trust architecture

Instead of creating a “blanket” isolation policy for your entire workspace, you might want more fine-grained access control. For example, if you want to build a zero-trust architecture from the ground up, you want to start with a “deny-all” policy and then allow access for one resource at a time.

  1. Disable service isolation at the workspace level.
      apiVersion: admin.gloo.solo.io/v2
kind: WorkspaceSettings
metadata:
  name: apis-team
  namespace: apis-team-config
spec:
  exportTo:
  - workspaces:
    - name: web-team
  options:
    serviceIsolation:
      enabled: false
  2. Use Gloo access policies to scope access to your resources.

Proxy trimming

Service isolation discovers and creates a lot of Istio resources pretty quickly. As such, you might notice performance impacts. For large environments, you can trim all the Istio sidecar proxy configs within the workspace to eliminate all outbound destinations.

You can enable proxy trimming with the serviceIsolation.trimProxyConfig setting at the workspace level to add in all destinations, or by selecting specific workloads in a TrimProxyConfigPolicy. For more information, see the trim proxy config policy guide.

East-west gateway selection

In a standard Gloo deployment, you create an Istio east-west gateway per workload cluster to enable multicluster routing. In more advanced scenarios, you might have multiple east-west gateways per cluster, such as to further isolate traffic to certain apps. In such cases, you can choose the east-west gateways that Gloo uses for multicluster routing for a workspace.

By default, Gloo looks for a port named tls on the east-west gateway. However, you can override this behavior in the workspace settings, such as in the following example.

  apiVersion: admin.gloo.solo.io/v2
kind: WorkspaceSettings
metadata:
  name: web-team
  namespace: web-team-config
spec:
  options:
    eastWestGateways:
    - selector:
        labels:
          istio: eastwestgateway       # select any gateway with this label
        # port:
        #   name: cross-cluster-mtls   # optional override of named port for mutual tls

Resource sharing with import and export settings

By default, Gloo resources apply throughout the workspace. For example, you might have a failover policy that you want all of the apps in the Web team’s workspace to use. Each app in the Web team’s workspace can use the same failover policy, even if the app runs in a different namespace and cluster than where the failover policy is created.

But what if you need to use services in a different team’s workspace?

With workspace settings, you can easily set up which services in your workload clusters can talk to each other and how you want to share Gloo resources. Both workspaces must agree to share resources by configuring the import and export fields in each workspace settings. For more information, see Import and export resources across workspaces.

Example workspace YAML configuration

The following diagram and GitHub Gist example configures the settings for three workspaces. For more information, see Import and export resources across workspaces.

Figure: Workspaces as configuration boundaries.
Figure: Workspaces as configuration boundaries.
  • Ops team: The Ops team manages the gateways and addons such as external auth and rate limiting servers. As such, it exports the addon services to all workspaces. The Ops team imports from the web team only, because it does not need to use the backend microservices. Service isolation is enabled. Federation is disabled because virtual destinations are the preferred way to set up multicluster routing.
  • Web team: The Web team imports from both the Ops and Backend API teams, but only exports to the Ops team. Service isolation is enabled. Federation is disabled because virtual destinations are the preferred way to set up multicluster routing.
  • Backend API team: The Backend API team imports from only the Ops team, as it does not need the Web team’s microservices. The Backend API team does export to the Web team, which uses the backend microservices to run the web app. Service isolation is enabled. Federation is disabled because virtual destinations are the preferred way to set up multicluster routing.