Workspace configuration
You configure two Gloo custom resources to achieve multitenancy: Workspace and WorkspaceSettings.
About workspace configuration
The WorkspaceSettings custom resource is separate from the Workspace so that you can more easily delegate its configuration to each team in your org. Settings include import and export, service isolation, and east-west gateway selection, if applicable.
- Workspace: A workspace groups together the Kubernetes namespaces and clusters that a team needs access to. The Gloo, Istio, and Kubernetes resources that your team creates in a workspace can be used by all the namespaces in the workspace, across clusters. The platform admin typically creates the workspace.
- WorkspaceSettings: In the workspace settings, your team configures how its services within the workspace can communicate with each other. The settings help control how the team’s resources interact with resources from other workspaces.
Multitenancy feature by custom resource
The following table summarizes the features you configure by custom resource.
Feature to set up | Workspace | WorkspaceSettings |
---|---|---|
Workspace name | ✅ | - |
Cluster the workspace includes | ✅ | - |
Namespace the workspace includes | ✅ | - |
Gloo CR restrictions by namespace | ✅ | - |
Service isolation within workspace | - | ✅ |
East-west gateway selection | - | ✅ |
Resource sharing across workspaces with import and export | - | ✅ |
Keep reading this page for best practices about workspace, workspace settings, and example YAML configuration files.
Example persona-driven diagram
The following diagram shows how different personas might configure workspaces and workspace settings.
- The platform admin creates the Ops and App team workspaces in the management cluster. She also creates an
ops-mgmt
andapp-mgmt
namespace in the management cluster for the teams’ Gloo configuration. - The Ops team owner creates the Ops team’s workspace settings and other Gloo custom resources in the
ops-mgmt
namespace in the management cluster. The Ops workspace is set up to manage resources like the Istio ingress and east-west gateways and Gloo add-ons like the external auth and rate limiting services for each cluster. - The App team owner creates the App team’s workspace settings and other Gloo custom resources in the
app-mgmt
namespace in the management cluster. The App workspace is set up to manage the frontend and backend apps.
Workspace configuration
Review the configuration guidance for creating the workspace, selecting namespaces and clusters, and commonly shared workspaces. For more information, see the API docs.
Workspace metadata
Name your workspace for the tenant that it represents, such as the Ops team, Finance department, or Web apps. You create all of your workspaces in the gloo-mesh
namespace of the management cluster.
Cluster and namespace boundaries for workspaces
A Kubernetes namespace can belong to only one workspace. A single workspace, however, can have Kubernetes namespaces from many different clusters. In the workspace configuration, you group together the Kubernetes namespaces and clusters for the workspace to manage. Review the following selection options.
Conflicting workspaces
Workspaces can select multiple Kubernetes namespaces. But what happens when different workspaces select the same Kubernetes namespace? In general, the oldest workspace by creation timestamp takes precedence and gets the namespace. The newer conflicting workspaces do not get a namespace, which means that they might not translate any Gloo custom resources or exhibit other such unexpected behavior until the conflict is resolved. Review the following example scenarios and ways to avoid conflicting workspaces.
Avoiding conflicts
To avoid unexpected results from conflicting workspaces, consider the following best practices.
- Create your most critical workspaces in order of importance. This way, the critical workspaces take precedence in case of conflict.
- When practical, avoid selecting namespaces by wildcards or labels. Instead, specify each namespace by name. At a minimum, consider scoping any wildcards and enforcing naming conventions across your workspace teams. For example, you might have all namespaces that belong to a Web Team begin with
frontend
. Then, the workspace uses afrontend*
wildcard. - Coordinate across workspace teams so that others are aware of any reserved names, reserved labels, or other naming conventions.
Workspace configuration in the management plane
Consider keeping all your Gloo configuration in a single management cluster. This way, only users with access to the management cluster can create Gloo configuration, which simplifies access control. Also, if an issue occurs, the Gloo resources are easier to find.
- Management plane: The management cluster has a config namespace for each workspace. The team creates all their Gloo custom resources in this config namespace.
- For the
web-team
, theweb-team-config
namespace on themgmt
cluster has all the Gloo config. - For the
apis-team
, theapis-team-config
namespace on themgmt
cluster has all the Gloo config.
- For the
- Data plane: For their application workloads, the team deploys their Kubernetes resources in the workload clusters, not the management cluster.
- For the
web-team
, theweb-ui
namespace in any cluster runs their workloads. - For the
apis-team
, theapis
namespace only incluster1
runs their workloads. - The
configEnabled: false
setting is used to stop Gloo from reading Gloo configuration in the data plane namespaces. This way, the teams must create their Gloo configurations in the management plane.
- For the
apiVersion: admin.gloo.solo.io/v2
kind: Workspace
metadata:
name: web-team
namespace: gloo-mesh
spec:
workloadClusters:
- name: 'mgmt' # Gloo configuration only namespace in the management plane
namespaces:
- name: 'web-team-config'
- name: '*' # Any cluster that has the namespace 'web-ui'
namespaces:
- name: 'web-ui'
configEnabled: false # Prevent Gloo from reading configuration in data plane
---
apiVersion: admin.gloo.solo.io/v2
kind: Workspace
metadata:
name: apis-team
namespace: gloo-mesh
spec:
workloadClusters:
- name: 'mgmt' # Gloo configuration only namespace in the management plane
namespaces:
- name: 'apis-team-config'
- name: 'cluster1' # APIs namespace in cluster1
namespaces:
- name: 'apis'
configEnabled: false # Prevent Gloo from reading configuration in data plane
Examples of common workspaces
Review the following examples of common workspaces. Also, refer to the Example workspace YAML configuration.
Workspace settings configuration
As the platform admin, you can delegate workspace settings to each team owner. This way, each team can make its own decisions about service isolation, east-west gateways, and resource sharing. The team owners can work together to determine import and export settings so that they can share resources with one another.
If you created a global
workspace in the gloo-mesh
namespace of the management cluster and you want to create global workspace settings that apply to all namespaces in all clusters, you must create a WorkspaceSettings
resource that is also named global
in the gloo-mesh
namespace of the management cluster.
Service isolation
To reduce network traffic and “noisy neighbor” concerns, you might want to isolate your services. In Kubernetes, your services are not isolated by default, but instead can communicate with other services on the same node. Similarly, namespaces are also not isolated from other namespaces in a cluster. To isolate services, you must set up additional resources like network policies from the container networking interface (CNI) plug-in.
With Gloo workspaces, you can quickly enforce service isolation at the workspace level. The workspace becomes the networking boundary. For even more fine-grained support, you can disable service isolation and use Gloo access policies to set up a zero-trust architecture instead.
Enabling service isolation at the workspace level
When you enable service isolation, Gloo creates Istio authorization policies and peer authentication resources in each namespace of the workspace. These resources allow communication across services within the workspace, but deny all other communication from outside the workspace. Even more convenient, if you set up import and export with other workspaces, Gloo allows communication from those workspaces automatically.
In a service mesh environment, service isolation also lets you quickly set up traffic encryption for all of your services within the workspace. Connections across the services are secured via mutual TLS (mTLS).
Service isolation config example: For more information, see the API docs.
apiVersion: admin.gloo.solo.io/v2
kind: WorkspaceSettings
metadata:
name: apis-team
namespace: apis-team-config
spec:
exportTo:
- workspaces:
- name: web-team
options:
serviceIsolation:
enabled: true
Disabling service isolation as part of a zero-trust architecture
Instead of creating a “blanket” isolation policy for your entire workspace, you might want more fine-grained access control. For example, if you also have a Gloo Mesh Enterprise license, you can build a zero-trust architecture from the ground up by starting with a “deny-all” policy and then allowing access for one resource at a time.
- Disable service isolation at the workspace level.
apiVersion: admin.gloo.solo.io/v2 kind: WorkspaceSettings metadata: name: apis-team namespace: apis-team-config spec: exportTo: - workspaces: - name: web-team options: serviceIsolation: enabled: false
- Use Gloo access policies to scope access to your resources.
Proxy trimming
Service isolation discovers and creates a lot of Istio resources pretty quickly. As such, you might notice performance impacts. For large environments, you can trim all the Istio sidecar proxy configs within the workspace to eliminate all outbound destinations.
You can enable proxy trimming with the serviceIsolation.trimProxyConfig
setting at the workspace level to add in all destinations, or by selecting specific workloads in a TrimProxyConfigPolicy
. For more information, see the Trim proxy config guide in the Gloo Mesh Enterprise documentation.
To use this feature, you must have a Gloo Mesh Enterprise license in addition to your Gloo Mesh Gateway license.
You can trim the Istio sidecar proxies for standard Kubernetes workloads, such as Deployments, DaemonSets, and StatefulSets. Proxy trimming is not supported for workloads from other tooling providers’ custom resource definitions (CRDs), such as DeploymentConfigs in OpenShift.
East-west gateway selection
In a standard Gloo deployment, you create an Istio east-west gateway per workload cluster to enable multicluster routing. In more advanced scenarios, you might have multiple east-west gateways per cluster, such as to further isolate traffic to certain apps. In such cases, you can choose the east-west gateways that Gloo uses for multicluster routing for a workspace.
By default, Gloo looks for a port named tls
on the east-west gateway. However, you can override this behavior in the workspace settings, such as in the following example.
apiVersion: admin.gloo.solo.io/v2
kind: WorkspaceSettings
metadata:
name: web-team
namespace: web-team-config
spec:
options:
eastWestGateways:
- selector:
labels:
istio: eastwestgateway # select any gateway with this label
# port:
# name: cross-cluster-mtls # optional override of named port for mutual tls
Resource sharing with import and export settings
By default, Gloo resources apply throughout the workspace. For example, you might have a failover policy that you want all of the apps in the Web team’s workspace to use. Each app in the Web team’s workspace can use the same failover policy, even if the app runs in a different namespace and cluster than where the failover policy is created.
But what if you need to use services in a different team’s workspace?
With workspace settings, you can easily set up which services in your workload clusters can talk to each other and how you want to share Gloo resources. Both workspaces must agree to share resources by configuring the import and export fields in each workspace settings. For more information, see Import and export resources across workspaces.
Example workspace YAML configuration
The following diagram and GitHub Gist example configures the settings for three workspaces. For more information, see Import and export resources across workspaces.
- Ops team: The Ops team manages the gateways and addons such as external auth and rate limiting servers. As such, it exports the addon services to all workspaces. The Ops team imports from the web team only, because it does not need to use the backend microservices. Service isolation is enabled. Federation is disabled because virtual destinations are the preferred way to set up multicluster routing.
- Web team: The Web team imports from both the Ops and Backend API teams, but only exports to the Ops team. Service isolation is enabled. Federation is disabled because virtual destinations are the preferred way to set up multicluster routing.
- Backend API team: The Backend API team imports from only the Ops team, as it does not need the Web team’s microservices. The Backend API team does export to the Web team, which uses the backend microservices to run the web app. Service isolation is enabled. Federation is disabled because virtual destinations are the preferred way to set up multicluster routing.