On this page

Tutorial: Federate clusters and isolate workloads for multitenancy

Use the Bookinfo sample app to try out multitenancy, federation, and isolation across multiple clusters.

Multitenancy: Gloo workspaces let you delegate management and policy decisions to your teams. You create workspaces, and select the appropriate namespaces and clusters for the workspaces. Note that in multicluster setups, a workspace can even include namespaces across clusters. Afterwards, the rest of your teams access only the namespaces in the clusters that they need.
Federation: With Gloo, you can federate trust so that services in different clusters can communicate with each other. In this example, Gloo handles the certificates for you, but you can optionally bring your own. Federation lets you easily reuse policies and other resources across your multi-mesh environment.
Isolation: To enforce a zero trust model from the start, you can enable isolation for services. Isolated services cannot communicate with services outside the mesh or in another workspace, and mTLS is used for communication across services. Teams must set up their workspaces to import and export the services that they need.

The following figure depicts the multi-mesh architecture created by this guide.

Figure of multicluster Bookinfo traffic in a Gloo Mesh (Gloo Platform APIs) quick-start architecture.

Before you begin

Set up multitenancy by creating workspaces for your workloads

Gloo introduces a new concept for Kubernetes-based multitenancy, the Workspace custom resource. A workspace consists of one or more Kubernetes namespaces that are in one or more clusters. Think of a workspace as the boundary of your team’s resources. To get started, you can create a workspace for each of your teams. Your teams might start with their apps in a couple Kubernetes namespaces in a single cluster. As your teams scale across namespaces and clusters, their workspaces scale with them. Note that for simplicity, you create all your Gloo custom resources only in the management cluster. For more information, see the Multitenancy concept.

Create a bookinfo workspace that spans across all your clusters, and includes only the bookinfo namespaces in each cluster. Note that you must create the workspace resource in the gloo-mesh namespace of the management cluster. For more information about setting up workspaces, see Create a workspace.
```
  kubectl apply --context ${MGMT_CONTEXT} -f- <<EOF
apiVersion: admin.gloo.solo.io/v2
kind: Workspace
metadata:
  name: bookinfo
  namespace: gloo-mesh
spec:
  workloadClusters:
    - name: '*'
      namespaces:
      - name: bookinfo
EOF
  
```
Configure settings for the bookinfo workspace, which include:
- Enabled federation: Services in different clusters can communicate with each other. For example, even though the services in the bookinfo workspace are in different clusters, they are able to communicate with each other. In subsequent steps, you create routes in a route table to forward traffic between Bookinfo services across clusters.
- Enabled service isolation: Services are isolated and cannot communicate with services outside the mesh or in another workspace by default. mTLS is used for communication across services.
- Exporting: Because service isolation is enabled, the resources in this workspace are exported to the istio-system workspace. This setting enables the ingress gateway in the istio-system workspace to use the resources in the bookinfo workspace.
```
  kubectl apply --context ${MGMT_CONTEXT} -f- <<EOF
apiVersion: admin.gloo.solo.io/v2
kind: WorkspaceSettings
metadata:
  name: bookinfo-settings
  namespace: bookinfo
spec:
  exportTo:
  - workspaces:  
    - name: istio-system
  options:
    serviceIsolation:
      enabled: true
    federation:
      enabled: false
      serviceSelector:
        - {}
      hostSuffix: 'global'
EOF
  
```

Create an istio-system workspace that spans across clusters, and includes the istio-system and istio-eastwest namespaces in each cluster. This ensures that the istiod control plane components as well as the gateways are included in the same workspace.

  kubectl apply --context ${MGMT_CONTEXT} -f- <<EOF
apiVersion: admin.gloo.solo.io/v2
kind: Workspace
metadata:
  name: istio-system
  namespace: gloo-mesh
spec:
  workloadClusters:
    - name: '*'
      namespaces:
      - name: istio-system
      - name: istio-eastwest
EOF

Configure settings for the istio-system workspace. Resources are imported for use from the bookinfo workspace.

  kubectl apply --context ${MGMT_CONTEXT} -f- <<EOF
apiVersion: admin.gloo.solo.io/v2
kind: WorkspaceSettings
metadata:
  name: istio-system-settings
  namespace: istio-system
spec:
  importFrom:
  - workspaces:
    - name: bookinfo
EOF

Modify the default workspace that you created in the quick-start guide to become a management-only workspace.

  kubectl apply --context ${MGMT_CONTEXT} -f- <<EOF
apiVersion: admin.gloo.solo.io/v2
kind: Workspace
metadata:
  name: ${MGMT_CLUSTER}
  namespace: gloo-mesh
spec:
  workloadClusters:
    - name: '${MGMT_CLUSTER}'
      namespaces:
        - name: 'gloo-mesh'
EOF

check_circle

As you create more workspaces, you can create global workspace settings in the management cluster that apply by default to each workspace in your Gloo Mesh environment. For more information, see Configure workspace settings.

Test service isolation by blocking access from services

In the previous section, you enabled service isolation for the Bookinfo workspace. When service isolation is enabled:

Services in the mesh cannot communicate with services outside of the mesh.
Communication between services within the mesh are permitted only when services are in the same workspace or when their workspaces have rules to import/export resources between them.

To verify that services within the workspace are isolated, deploy two demo apps to an isolation-test namespace in cluster1, which is not part of the bookinfo workspace. One version of the app is included in the Istio service mesh, and one version is not included in the mesh. Then, attempt to curl the Bookinfo services from the demo apps to test service isolation.

Create the isolation-test namespace on cluster1. Do not label this namespace for Istio injection.
```
  kubectl create ns isolation-test --context ${REMOTE_CONTEXT1}
  
```

Create an app named in-mesh that is part of the Istio service mesh.

  kubectl -n isolation-test apply -f https://raw.githubusercontent.com/solo-io/gloo-mesh-use-cases/main/policy-demo/httpbin-in-mesh-gloo.yaml --context ${REMOTE_CONTEXT1}

Create another app named not-in-mesh that is not part of the Istio service mesh.

  kubectl -n isolation-test apply -f https://raw.githubusercontent.com/solo-io/gloo-mesh-use-cases/main/policy-demo/httpbin-not-in-mesh.yaml --context ${REMOTE_CONTEXT1}

Verify that the app pods have a status of running.

  kubectl -n isolation-test get pods --context ${REMOTE_CONTEXT1}

Curl the reviews service from the in-mesh service. This curl returns a 403 response code because the sidecar of the reviews service blocks the request. Even though the services are both in the same Istio service mesh, they cannot communicate because they are not in the same workspace, and the isolation-test namespace is not in a workspace with import/export rules for the Bookinfo workspace.
```
  pod=$(kubectl --context ${REMOTE_CONTEXT1} -n isolation-test get pods -l app=in-mesh -o jsonpath='{.items[0].metadata.name}')
kubectl --context ${REMOTE_CONTEXT1} -n isolation-test debug -i ${pod} --image=curlimages/curl -- curl -s -o /dev/null -w "%{http_code}" http://reviews.bookinfo.svc.cluster.local:9080/reviews/0
  
```

Curl the reviews service from the not-in-mesh service. This curl returns a 000 response code because communication cannot be established. Because the Bookinfo services are isolated, they cannot communicate with services outside of the same Istio service mesh.

  pod=$(kubectl --context ${REMOTE_CONTEXT1} -n isolation-test get pods -l app=not-in-mesh -o jsonpath='{.items[0].metadata.name}')
kubectl --context ${REMOTE_CONTEXT1} -n isolation-test debug -i ${pod} --image=curlimages/curl -- curl -s -o /dev/null -w "%{http_code}" http://reviews.bookinfo.svc.cluster.local:9080/reviews/0

Optional: For additional testing, you can change the Bookinfo workspace settings to disable service isolation. Then, when you repeat the commands in steps 5 - 6, the curl commands from both the in-mesh and not-in-mesh services now receive 200 response codes. Services within the same mesh and outside of the mesh can access the Bookinfo services because they are no longer isolated.

  kubectl apply --context ${MGMT_CONTEXT} -f- <<EOF
apiVersion: admin.gloo.solo.io/v2
kind: WorkspaceSettings
metadata:
  name: bookinfo-settings
  namespace: bookinfo
spec:
  exportTo:
  - workspaces:
     - name: 'istio-system'
  options:
    serviceIsolation:
      enabled: false
    federation:
      enabled: false
      serviceSelector:
        - {}
      hostSuffix: 'global'
EOF

Cleanup

You can optionally remove the resources that you set up as part of this guide.

Remove the demo app components.

  kubectl -n isolation-test delete -f in-mesh.yaml --context ${REMOTE_CONTEXT1}
kubectl -n isolation-test delete -f https://raw.githubusercontent.com/solo-io/gloo-mesh-use-cases/main/policy-demo/httpbin-not-in-mesh.yaml --context ${REMOTE_CONTEXT1}

Delete the isolation-test namespace.

  kubectl delete ns isolation-test --context ${REMOTE_CONTEXT1}

To revert your workspace setup to the single default workspace:

Modify the default workspace to select all clusters and namespaces again.

  kubectl apply --context ${MGMT_CONTEXT} -f- <<EOF
apiVersion: admin.gloo.solo.io/v2
kind: Workspace
metadata:
  name: ${MGMT_CLUSTER}
  namespace: gloo-mesh
spec:
  workloadClusters:
    - name: '*'
      namespaces:
        - name: '*'
EOF

Delete the bookinfo and istio-system workspaces and workspace settings.

  kubectl delete WorkspaceSettings istio-system-settings --context ${MGMT_CONTEXT}
kubectl delete Workspace istio-system --context ${MGMT_CONTEXT}
kubectl delete WorkspaceSettings bookinfo-settings --context ${MGMT_CONTEXT}
kubectl delete Workspace bookinfo --context ${MGMT_CONTEXT}

If you no longer need this quick-start Gloo Mesh environment, you can deregister workload clusters, uninstall management plane components from the management cluster, and uninstall Istio resources from the workload clusters by following the steps in Uninstalling Gloo Mesh and Istio.

Tutorial: Federate clusters and isolate workloads for multitenancy

Before you begin link

Set up multitenancy by creating workspaces for your workloads link

Test service isolation by blocking access from services link

Cleanup link

Before you begin

Set up multitenancy by creating workspaces for your workloads

Test service isolation by blocking access from services

Cleanup