Route across clusters with east-west gateways
Enable cross-cluster routing between apps in a multicluster Gloo Mesh Enterprise setup.
If you have a multicluster Gloo Mesh Enterprise setup, deploy an Istio east-west gateway into each workload cluster. An east-west gateway lets services in one mesh communicate with services in another.
Before you begin
Save the kubeconfig contexts for your clusters. The examples in this guide assume one management cluster, and two workload clusters that run Istio service meshes.
export MGMT_CONTEXT=<management-cluster-context> export REMOTE_CONTEXT1=<remote-cluster1-context> export REMOTE_CONTEXT2=<remote-cluster2-context>
If you have not already, set environment variables for the Solo distribution of Istio that you want to install. You can find these values in the Istio images built by Solo.io support article.
# Solo distrubution of Istio patch version # in the format 1.x.x, with no tags export ISTIO_VERSION=1.24.2 # Repo key for the minor version of the Solo distribution of Istio # This is the 12-character hash at the end of the repo URL: 'us-docker.pkg.dev/gloo-mesh/istio-<repo-key>' export REPO_KEY=<repo_key> # Solo distrubution of Istio patch version and Solo tag # Optionally append other Solo tags as needed export ISTIO_IMAGE=${ISTIO_VERSION}-solo # Solo distribution of Istio image repo export REPO=us-docker.pkg.dev/gloo-mesh/istio-${REPO_KEY}
Get the revision that you used for your installation. Typically, this is
main
for a Helm installation, orgloo
for a Gloo operator installation.export REVISION=$(kubectl get pod -L app=istiod -n istio-system -o jsonpath='{.items[0].metadata.labels.istio\.io/rev}') echo ${REVISION}
Deploy east-west gateways in each workload cluster
Prepare a Helm values file for the Istio east-west gateway. This sample command downloads an example file,
eastwest-gateway.yaml
, and updates the environment variables with the values that you previously set. You can further edit the file to provide your own details for production-level settings.curl -0L https://raw.githubusercontent.com/solo-io/gloo-mesh-use-cases/main/gloo-mesh-enterprise/istio-install/manual-helm/eastwest-gateway-1.24+.yaml > eastwest-gateway.yaml envsubst < eastwest-gateway.yaml > eastwest-gateway-values.yaml
Create an east-west gateway in each cluster.
helm upgrade --install istio-eastwestgateway istio/gateway \ --version ${ISTIO_VERSION} \ --namespace istio-eastwest \ --create-namespace \ --kube-context ${REMOTE_CONTEXT1} \ --wait \ -f eastwest-gateway-values.yaml helm upgrade --install istio-eastwestgateway istio/gateway \ --version ${ISTIO_VERSION} \ --namespace istio-eastwest \ --create-namespace \ --kube-context ${REMOTE_CONTEXT2} \ --wait \ -f eastwest-gateway-values.yaml
Verify that the east-west gateway pods are running and the load balancer service is assigned an external address.
kubectl get pods,svc -n istio-eastwest --context ${REMOTE_CONTEXT1} kubectl get pods,svc -n istio-eastwest --context ${REMOTE_CONTEXT2}
Example output:
NAME READY STATUS RESTARTS AGE istio-eastwestgateway-7f6f8f7fc7-ncrzq 1/1 Running 0 11s istio-eastwestgateway-7f6f8f7fc7-ncrzq 1/1 Running 0 48s NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE istio-eastwestgateway LoadBalancer 10.96.166.166 <externalip> 15021:32343/TCP,80:31685/TCP,443:30877/TCP,31400:31030/TCP,15443:31507/TCP,15012:30668/TCP,15017:30812/TCP 13s
AWS clusters only: For the Elastic Load Balancer (ELB) instance that is automatically created for you to back the east-west gateway service, verify that the health check shows a healthy state. Gloo Mesh configures the east-west gateway to listen on HTTPS port 15443. However, when the ELB is created, the first port that is defined in the Kubernetes service manifest is used to perform the health check. This port might be different from the port that Gloo Mesh configures. For your ELB health check to pass, you might need to configure the load balancer to run the health check on port 15443.
Set up routing across clusters
After you create east-west gateways in each workload cluster, you can then set up cross-cluster routing to your in-mesh apps. The steps in this section use the Bookinfo sample app as an example.
Create a Gloo root trust policy to ensure that services in
cluster1
securely communicate with thereviews
service incluster2
. The root trust policy sets up the domain and certificates to establish a shared trust model across multiple clusters in your service mesh.kubectl apply --context $MGMT_CONTEXT -f - <<EOF apiVersion: admin.gloo.solo.io/v2 kind: RootTrustPolicy metadata: name: root-trust namespace: gloo-mesh spec: config: mgmtServerCa: generated: {} EOF
Restart istiod and the sample apps that you deployed earlier to apply the updated certificates.
kubectl rollout restart deployment istiod-main -n istio-system --context ${REMOTE_CONTEXT1} kubectl rollout restart deployment istiod-main -n istio-system --context ${REMOTE_CONTEXT2} kubectl rollout restart deployment details-v1 productpage-v1 ratings-v1 reviews-v1 reviews-v2 -n bookinfo --context ${REMOTE_CONTEXT1} kubectl rollout restart deployment ratings-v1 reviews-v3 -n bookinfo --context ${REMOTE_CONTEXT2} kubectl rollout restart deployment httpbin -n httpbin --context ${REMOTE_CONTEXT1} kubectl rollout restart deployment helloworld-v1 helloworld-v2 -n helloworld --context ${REMOTE_CONTEXT1} kubectl rollout restart deployment helloworld-v3 helloworld-v4 -n helloworld --context ${REMOTE_CONTEXT2}
Create a virtual destination resource and define a unique hostname that in-mesh gateways can use to send requests to the
reviews
app. This virtual destination is configured to listen for incoming traffic on the internal-only, arbitrary hostnamereviews.mesh.internal.com:8080
.kubectl apply --context $MGMT_CONTEXT -n bookinfo -f- <<EOF apiVersion: networking.gloo.solo.io/v2 kind: VirtualDestination metadata: name: reviews-vd namespace: bookinfo spec: hosts: # Arbitrary, internal-only hostname assigned to the endpoint - reviews.mesh.internal.com ports: - number: 9080 protocol: HTTP services: - labels: app: reviews EOF
Create a route table that defines how east-west requests within your mesh are routed from the
productpage
service to thereviews-vd
virtual destination. When you apply this route table, requests fromproductpage
to/reviews
now route to one of the three reviews versions across clusters. The east-west gateway in your mesh does the work of taking requests made to thereviews.bookinfo.svc.cluster.local
hostname and routing them to thereviews.mesh.internal.com
virtual destination hostname that you specified in the previous step.kubectl apply --context $MGMT_CONTEXT -n bookinfo -f- <<EOF apiVersion: networking.gloo.solo.io/v2 kind: RouteTable metadata: name: bookinfo-east-west namespace: bookinfo spec: hosts: - 'reviews.bookinfo.svc.cluster.local' workloadSelectors: - selector: labels: app: productpage http: - name: reviews matchers: - uri: prefix: /reviews forwardTo: destinations: - ref: name: reviews-vd kind: VIRTUAL_DESTINATION port: number: 9080 labels: route: reviews EOF
In your terminal, open the Bookinfo product page from your local host.
- Enable port-forwarding on the product page deployment.
kubectl --context ${REMOTE_CONTEXT1} -n bookinfo port-forward deployment/productpage-v1 9080:9080
- Open your browser to http://localhost:9080/productpage?u=normal.
- Enable port-forwarding on the product page deployment.
Refresh the page a few times again. Now, the red stars for
reviews-v3
are shown in the book reviews.Figure: Bookinfo product page UI with red star reviews Figure: Bookinfo product page UI with red star reviews Bookinfo services in
cluster1
are now successfully accessing the Bookinfo services incluster2
!
Next
- Review your options for exposing apps in your service mesh with an ingress gateway.
- If you haven’t already, install Gloo Mesh Enterprise so that Gloo Mesh Enterprise can manage your Istio resources. You don’t need to directly configure any Istio resources going forward.
- Review how Gloo Mesh custom resources are automatically translated into Istio resources.
- Apply Gloo policies to manage the security and resiliency of your service mesh environment.
- Launch the Gloo UI to review the Istio insights that were captured for your service mesh setup. Gloo Mesh Enterprise comes with an insights engine that automatically analyzes your Istio setups for health issues. These issues are displayed in the UI along with recommendations to harden your Istio setups. The insights give you a checklist to address issues that might otherwise be hard to detect across your environment. For more information, see Insights.
- Monitor and observe your Istio environment with Gloo Mesh Enterprise’s built-in telemetry tools.
- When it’s time to upgrade your service mesh, you can perform a safe in-place upgrade by using the Gloo operator or Helm.