Navigation :

Route within or across clusters

You can easily set up intelligent routing for active-active and active-passive workloads in one service mesh or across service meshes by using virtual destinations.

Single-cluster routing: Although services can directly communicate with each other within a service mesh, virtual destinations allow you to apply routing rules for service-to-service communication to each destination in your servce mesh. For example, by creating a virtual destination for a service, you can apply rules to it that designate routing to different versions of the service by use of weighted subsets.
Multicluster routing: With virtual destinations, you can define unique internal hostnames for apps that are spread across multiple clusters, and enable service discovery for these apps within the mesh by adding the hostnames to the service mesh registry.

Multicluster routing with Gloo Mesh

For more information, see the following resources:

The instructions in this guide assume that the services you want to route requests to are included in your service mesh. If you want to route to services or endpoints that are not included in your service mesh, see Route to services external to the mesh.

Before you begin

This guide assumes that you use the same names for components like clusters, workspaces, and namespaces as in the getting started. If you have different names, make sure to update the sample configuration files in this guide.

Complete the multicluster getting started guide to set up the following testing environment.
- Three clusters along with environment variables for the clusters and their Kubernetes contexts.
- The Gloo Platform CLI, meshctl, along with other CLI tools such as kubectl and istioctl.
- The Gloo management server in the management cluster, and the Gloo agents in the workload clusters.
- Istio installed in the workload clusters.
- A simple Gloo workspace setup.
Install Bookinfo and other sample apps.
In the workspace settings for each workspace that your services are in, ensure that you configure the following:
- Enable federation so that services in different clusters can communicate with each other.
- If you enable service isolation, services cannot communicate with services outside the mesh or in another workspace by default. If you want to set up multicluster routing across multiple workspaces, be sure to export the required resources from one workspace to the other workspaces that must access them. For example, if your ingress gateway is in a different workspace than your apps, be sure to export the app workspace resources to the gateway workspace.

Create a Gloo root trust policy to ensure that services in one cluster securely communicate with the services in other clusters. The root trust policy sets up the domain and certificates to establish a shared trust model across multiple clusters in your service mesh.

kubectl apply --context=$MGMT_CONTEXT -f - <<EOF
apiVersion: admin.gloo.solo.io/v2
kind: RootTrustPolicy
metadata:
  name: root-trust
  namespace: gloo-mesh
spec:
  config:
    autoRestartPods: true
    mgmtServerCa:
      generated: {}
EOF

Follow the other guides in this routing section to plan your routing table setup. For example, you might check out the path matching guide to decide how to match the incoming requests to your service paths, the redirect guide to set up any path or host rewrites, or the sub-table delegation guide to nest and sort multiple route tables. Note: Be sure that each route for one host is unique, such as by using prefix matching to determine which requests to the host should be forwarded to which destinations.

Set up routing for east-west traffic

To route in-mesh requests from one app to another app that is deployed to one cluster or that is deployed across clusters, you create a virtual destination for your destination app. Then, you create a route table that forwards east-west traffic from the initiator app to that virtual destination.

Create a virtual destination resource for your destination app that receives requests from the initiator app. This virtual destination is configured to listen for incoming traffic on the internal-only, arbitrary hostname destination-app.mesh.internal.com:8080. Incoming requests can then be routed to any service instances with the label app: destination-app on port 9080. Note that because virtual destinations are dynamic, the east-west gateway that handles the request routes it to the closest healthy app instance.

kubectl apply --context $REMOTE_CONTEXT1 -n global -f- <<EOF
apiVersion: networking.gloo.solo.io/v2
kind: VirtualDestination
metadata:
  name: destination-app-vd
  namespace: global
spec:
  hosts:
  # Arbitrary, internal-only hostname assigned to the endpoint
  - destination-app.mesh.internal.com
  ports:
  - number: 8080
    protocol: HTTP
    targetPort:
      number: 9080
  services:
    - labels:
        app: destination-app
EOF

Route requests initiated from your initiator app to your destination app by using the virtual destination hostname. The routing method depends on whether your team can change the address that the initiator app sends requests to.

Change initiator app to route to new hostname (preferred)
Reroute existing traffic patterns

If you can change the initiator app's code, edit the app to call the destination app using the virtual destination hostname instead of the service's DNS entry. In this setup, no route table is needed, because the initiator app can now directly call the virtual destination. However, if you want to also add routing rules for this virtual destination, you can create a route table, such as the following.

For hosts, specify the virtual destination hostname that you created in the previous step. In this example, destination-app.mesh.internal.com is used.
For workloadSelectors, specify the initiator app that initiates the request to the destination app. To instead select all workloads, do not specify this section. In this example, an app with the label app: initiator-app is selected.
For the http route, specify the virtual destination for the destination app, and any routing rules you want to apply. For example, you might use weighted routing to the subset of app versions.

kubectl apply --context $REMOTE_CONTEXT1 -n global -f- <<EOF
apiVersion: networking.gloo.solo.io/v2
kind: RouteTable
metadata:
  name: destination-app-routes
  namespace: global
spec:
  hosts:
  # Applies to the VD hostname that destination-app listens on
    - 'destination-app.mesh.internal.com'
  # Applies to requests sent by this initiator app
  workloadSelectors:
  - selector:
      labels:
        app: initiator-app
  http:
    # Route for the destination-app service
    - name: destination-app
      # Prefix matching
      matchers:
      - uri:
          prefix: /destination-app
      # Forwarding directive
      forwardTo:
        destinations:
          # Reference to the virtual destination that directs 15% of reviews traffic to destination-app-v1
          - ref:
              name: destination-app-vd
            kind: VIRTUAL_DESTINATION
            port:
              number: 8080
            subset:
              version: v1
            weight: 15
          # Reference to the virtual destination that directs 85% of reviews traffic to destination-app-v2
          - ref:
              name: destination-app-vd
            kind: VIRTUAL_DESTINATION
            port:
              number: 8080
            subset:
              version: v2
            weight: 85
EOF

If you cannot change the initiator app to call the destination app using the virtual destination hostname instead of the service's DNS entry, you can create a route table that defines how east-west requests within your mesh from the initiator app to the virtual destination for the destination app should be routed.

For hosts, specify the internal DNS entry that the destination app listens on, formatted such as <service-name>.<namespace-name>.svc.cluster.local. In this example, destination-app.global.svc.cluster.local is used. The east-west gateway in your mesh does the work of taking requests made to the destination-app.global.svc.cluster.local hostname and routing them to the destination-app.mesh.internal.com virtual destination hostname that you specified in the previous step.
For workloadSelectors, specify the initiator app that initiates the request to the destination app. To instead select all workloads, do not specify this section. In this example, an app with the label app: initiator-app is selected.
For the http route, specify the virtual destination for the destination app.

kubectl apply --context $REMOTE_CONTEXT1 -n global -f- <<EOF
apiVersion: networking.gloo.solo.io/v2
kind: RouteTable
metadata:
  name: destination-app-routes
  namespace: global
spec:
  hosts:
  # Applies to the internal hostname that destination-app listens on
    - 'destination-app.global.svc.cluster.local'
  # Applies to requests sent by this initiator app
  workloadSelectors:
  - selector:
      labels:
        app: initiator-app
  http:
    # Route for the destination-app service
    - name: destination-app
      # Forwarding directive
      forwardTo:
        destinations:
          # Reference to the virtual destination for the destination-app svcs
          - ref:
              name: destination-app-vd
            kind: VIRTUAL_DESTINATION
            port:
              number: 8080
EOF

Test the route from the initiator app to your destination app. For example, log in to your initiator app and run nslookup destination-app.mesh.internal.com to verify that the destination app is reachable through the virtual destination hostname. Or, if you can access your initiator app externally, you can curl the ingress gateway address and the path for your initiator app, and verify that information from the destination app is being successfully returned.

When you use a virtual destination, any request to the virtual destination is load balanced across healthy app instance without leaving the service mesh. You can apply policies to further control which app instance returns a response. For example, you might use an outlier detection policy to force traffic to local apps, or failover and fault injection policies to purposefully fail the connection to in the first cluster.

Next steps

If you haven't already, follow the other guides in the routing section to plan your routing table setup. For example, you might check out the prefix matching guide to decide how to match the incoming requests to your service paths, the redirect guide to set up any path or prefix rewrites, or the sub-table delegation guide to nest and sort multiple route tables.
Configure additional route settings, such as weighted routing to version subsets or adding and removing headers.
Apply a policy to your service or route. For example, you might set a connection policy to keep alive connections that the load balancer in your infrastructure provider might otherwise periodically close.