Multicluster routing with virtual destinations

You can easily set up intelligent multicluster routing for active-active and active-passive workloads in your environment by using virtual destinations. Virtual destination allow you to define unique internal hostnames for apps that are spread across multiple clusters, and route to these apps via these hostnames.

For more information about how to set up routing to multiple service across clusters, see Route across clusters.

How does it work?

Virtual destinations provide a more flexible abstraction for the services that back up the routes in your route table. Kubernetes services might change as you deploy different versions and updates to your apps. By using a virtual destination, you can consistently select the appropriate backing services with Kubernetes labels. This approach helps your setup dynamically scale and adjust to underlying application changes.

Virtual services also automatically set up intelligent multicluster routing. In virtual destinations, you define unique internal hostnames for apps that are spread across multiple clusters. To reach a multicluster endpoint, a client sends a request to an external address, which Gloo Gateway translates into the internal hostname for the apps. Then Gloo Gateway uses the inetrnal hostname to route the request to one of the app instances. The client does not need to know where the target app is deployed within the cluster environment. Instead, Gloo Gateway takes into account the context of the client request, such as the client location, where the target app is deployed within the cluster environment, and the routing rules that you defined to ensure that the request is routed in an efficient and secure manner.

Let’s take a look at an example multicluster setup to see how routing is done for active-active workloads in Kubernetes and how you can improve this scenario by using Gloo Gateway.

Multicluster routing in Kubernetes

In the following example, an external client wants to send a request to app B. The app B deployment consists of multiple instances that are spread across multiple clusters. To achieve multicluster routing in Kubernetes, all participating clusters must be added to a global load balancer. The global load balancer is assigned a custom domain, such as mydomain.com and each app in the cluster must be configured to serve on a specific path, such as app-b. For the client to reach app B, the client must send a request over the internet to the global load balancer by using the mydomain.com/app-b address. The global load balancer uses round-robin to select a cluster and routes the request to the cluster's ingress gateway, which then forwards the request to app B.

Figure: Traditional multicluster routing in Kubernetes.

A major challenge in this setup is knowing where app B is deployed. The global load balancer is aware of the ingress gateway IP addresses and whether or not a certain cluster can be reached. However, the global load balancer does not know if app B is actually deployed in cluster 2 or 3, or whether app B is healthy to accept requests.

Multicluster routing with Gloo Gateway

Now let’s look at how you can improve this scenario with virtual destinations in Gloo Gateway. To expose app B within the cluster environment, you define a unique internal hostname, such as app-b.internal, and assign a target port to it, such as 80. Then, you use labels to select the services in your clusters that expose app B. Gloo Gateway automatically discovers the app B service instances and creates the corresponding Istio custom resources, such as service entries, virtual services, and destination rules to enable multicluster routing. Finally, you create a route table that routes incoming requests for mydomain.com/app-b:80 to the app-b.internal hostname.

Because Gloo Gateway is aware of where app B is deployed and whether an instance is healthy to accept incoming requests, any request to mydomain.com/app-b:80 is routed through app-b.internal:80. Gloo Gateway routes the request to the closest app B instance. For example, if an instance of app B exists in the same cluster as the gateway, Gloo Gateway routes the request to that instance. In the following diagram, the closest instance is deployed within the same cluster.

Figure: Multicluster routing with Gloo Gateway.

But what happens if no local app B instance exists or the local instance becomes unavailable? Gloo Gateway automatically updates the routing rules and gives priority to the next closest app B instance in another cluster. In the following diagram, the next closest app B instance is deployed to cluster 2. Instead of leaving the cluster environment and going to the global load balancer, the request is now routed to the east-west gateway of cluster 2 directly, which forwards the request to app B.

Figure: Multicluster routing with Gloo Gateway in a failover scenario.

With location-aware priority-based routing, you ensure that requests are always routed in an efficient and secure manner improving network latency, response time, availability, and reliability for your services within the cluster environment. Instead of duplicating the same deployments across all the clusters as you would do in a traditional Kubernetes setup, you can optimze your clusters and deploy your apps where you need them without configuring and updating custom routing rules for each cluster.

When Gloo Gateway routes incoming requests across clusters through the east-west gateway, the communication from Gloo Gateway to the east-west gateway is secured with mTLS. However, when your app is deployed without Istio sidecars, the east-west gateway uses plaintext to route the request to your app. To secure communications to your apps with mTLS instead, consider using Gloo Mesh Enterprise alongside Gloo Gateway to set up an Istio service mesh for your workloads.

Additionally, cross-cluster routing through the east-west gateway in Gloo Gateway is supported only for incoming requests from a client that is external to your cluster environment. You can use Gloo Mesh Enterprise to also route from service-to-service within your cluster environment by using mTLS connections through the east-west gateway.

Applying overlapping virtual destination configuration

In some cases, you might have overlapping configuration across multiple virtual destinations. Gloo Gateway validates the hosts that you specify in each virtual destination, and returns an error if a virtual destination that defines the same host already exists in your Gloo Gateway environment. However, you can apply virtual destinations that define the same host in different workspaces, if the workspaces do not import or export the same virtual destination resources between them. For example, you might represent the same backend apps with the same host by applying a virtual destination in a staging workspace and by applying a virtual destination in a prod workspace. Because you do not import or export the virtual destination resources between these two workspaces, the virtual destination resources can be safely applied.