Ingress to Istio sidecar mesh with virtual destinations (GME)

About Gloo Mesh (Gloo Platform APIs) link

Gloo Mesh (Gloo Platform APIs) is a service mesh management plane that is based on hardened, open-source projects like Envoy and Istio. With Gloo Mesh, you can unify the configuration, operation, and visibility of service-to-service connectivity across your distributed applications. These apps can run in different virtual machines (VMs) or Kubernetes clusters on premises or in various cloud providers, and even in different service meshes.

About VirtualDestinations link

A VirtualDestination is a traffic management concept in Gloo Mesh (Gloo Platform APIs) that allows you to define unique internal hostnames for services that are spread across a multicluster service mesh. Without VirtualDestinations, you must update the IP addresses of your services every time the IP address changes, such as when you update your app or deploy a new version. Because a VirtualDestination uses labels to select the backing services, it can automatically detect IP address changes and update them for you.

To learn more about VirtualDestinations, see the VirtualDestinations conceptual overview and multicluster routing guide in the Gloo Mesh (Gloo Platform APIs) documentation.

About this guide link

In this guide, you explore how to use Gloo Mesh (Gloo Platform APIs) and Gloo Gateway together to route traffic to Bookinfo services that are spread across clusters. Bookinfo is a sample microservices-based app that is provided by Istio and composed of 4 different microservices that interact with each other. The app is commonly used to demonstrate Istio’s service mesh features and capabilities.

To accomplish multicluster routing with Gloo Gateway and Gloo Mesh (Gloo Platform APIs), you leverage VirtualDestinations. You can use VirtualDestinations in multiple ways in Gloo Gateway. This guide provides steps to accomplish the following tasks:

Expose the product page app on a gateway, which is part of a service mesh that Gloo Mesh (Gloo Platform APIs) manages. The product page app uses a VirtualDestination to round robin requests to reviews app instances that are spread across multiple clusters. The VirtualDestination is not exposed on the gateway directly, but rather used within the service mesh only.

Expose the reviews VirtualDestination on the gateway directly. This way, you can explore how to use a unique internal hostname to round robin traffic to reviews app instances that are spread across clusters.

Step 1: Install Gloo Mesh (Gloo Platform APIs) link

1. Install Gloo Mesh (Gloo Platform APIs) by following the multicluster getting started tutorial in the Gloo Mesh (Gloo Platform APIs) documentation. This guide creates a three-cluster setup with one management cluster that runs the Gloo Mesh (Gloo Platform APIs) control plane and two workload clusters that run the data plane. You also install a Solo distribution of Istio in both of your workload clusters by using the Gloo Operator.

2. Deploy the sample Bookinfo app. You use this app to demonstrate traffic routing in a multicluster service mesh. You can optionally install other sample apps, such as httpbin or helloworld. However, these apps are not used in this guide.

3. Set up multicluster routing for the reviews app by using a VirtualDestination and RouteTable resource. Make sure that the productpage app can route traffic to all versions of the reviews app before you continue with installing Gloo Gateway.

4. Review the VirtualDestination that you created in the previous step. The VirtualDestination specifies the reviews.mesh.internal.com internal hostname that services in the mesh can use to reach all instances of the reviews app.

     kubectl get virtualdestination reviews-vd --context $MGMT_CONTEXT -n bookinfo -o yaml
     

   Example output:

     apiVersion: networking.gloo.solo.io/v2
   kind: VirtualDestination
   metadata:
     name: reviews-vd
     namespace: bookinfo
   spec:
     hosts:
     # Arbitrary, internal-only hostname assigned to the endpoint
     - reviews.mesh.internal.com
     ports:
     - number: 9080
       protocol: HTTP
     services:
       - labels:
           app: reviews
     

Step 2: Install Gloo Gateway link

After you successfully installed Gloo Mesh (Gloo Platform APIs) and confirmed that you can route traffic from the product page app to all versions of the reviews app, you can now continue to install Gloo Gateway. Gloo Gateway must be installed in a cluster that also has Istio installed. You can choose to install Gloo Gateway in one or each of your workload clusters.

This guide assumes that you want to install Gloo Gateway in workload cluster 1.

1. Set your Gloo Gateway license key as an environment variable. If you do not have one, contact an account representative.

     export GLOO_GATEWAY_LICENSE_KEY=<license-key>
     

2. Install the custom resources of the Kubernetes Gateway API. To use VirtualDestinations with Gloo Gateway, you must install the experimental channel of version 1.3.0.

     kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.3.0/experimental-install.yaml --context $REMOTE_CONTEXT1
     

   Example output:

     customresourcedefinition.apiextensions.k8s.io/backendlbpolicies.gateway.networking.k8s.io created
   customresourcedefinition.apiextensions.k8s.io/backendtlspolicies.gateway.networking.k8s.io created
   customresourcedefinition.apiextensions.k8s.io/gatewayclasses.gateway.networking.k8s.io created
   customresourcedefinition.apiextensions.k8s.io/gateways.gateway.networking.k8s.io created
   customresourcedefinition.apiextensions.k8s.io/grpcroutes.gateway.networking.k8s.io created
   customresourcedefinition.apiextensions.k8s.io/httproutes.gateway.networking.k8s.io created
   customresourcedefinition.apiextensions.k8s.io/referencegrants.gateway.networking.k8s.io created
   customresourcedefinition.apiextensions.k8s.io/tcproutes.gateway.networking.k8s.io created
   customresourcedefinition.apiextensions.k8s.io/tlsroutes.gateway.networking.k8s.io created
   customresourcedefinition.apiextensions.k8s.io/udproutes.gateway.networking.k8s.io created
     

3. Get the name of the istiod service. Depending on how you set up Istio, you might see a revisionless service name (istiod) or a service name with a revision, such as istiod-main.

     kubectl get services -n istio-system --context $REMOTE_CONTEXT1
     

   Example output:

     NAME          TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                                 AGE
   istiod-main   ClusterIP   34.118.238.13   <none>        15010/TCP,15012/TCP,443/TCP,15014/TCP   3d2h
     

4. Derive the Kubernetes service address for your istiod deployment. The service address uses the format <service-name>.<namespace>.svc:15012. For example, if your service name is istiod-main, the full service address is istiod-main.istio-system.svc:15012.

5. Install Gloo Gateway and enable the Istio integration.

   1. Add the Helm repository for Gloo Gateway Enterprise Edition.

        helm repo add glooe https://storage.googleapis.com/gloo-ee-helm
      helm repo update
        

   2. Install Gloo Gateway Enterprise Edition. This command creates the gloo-system namespace and installs the Gloo Gateway control plane into it. Replace the following values:

      • istioDiscoveryAddress: Set <istiod-service-address> to the istiod service address that you retrieved earlier.
      • istioMetaClusterId: The trustDomain that was set in the IstioOperator for istiod. If you followed the getting started tutorial in the Gloo Mesh (Gloo Platform APIs) documentation, you must set this value to the name of your workload cluster.
      • istioMetaMeshId: The meshID that was set in the IstioOperator for istiod. If you followed the getting started tutorial in the Gloo Mesh (Gloo Platform APIs) documentation, you must set this value to the name of your workload cluster.

        helm install -n gloo-system gloo-gateway glooe/gloo-ee \
      --kube-context $REMOTE_CONTEXT1 \
      --create-namespace \
      --version 1.20.3 \
      --set-string license_key=$GLOO_GATEWAY_LICENSE_KEY \
      -f -<<EOF
      gloo:
        gatewayProxies:
          gatewayProxy:
            disabled: true
        kubeGateway:
          enabled: true
          gatewayParameters:
            glooGateway:
              istio:
                istioProxyContainer:
                  istioDiscoveryAddress: <istiod-service-address>
                  istioMetaClusterId: <trustDomain>
                  istioMetaMeshId: <meshID>
        gloo:
          disableLeaderElection: true
      gloo-fed:
        enabled: false
        glooFedApiserver:
          enable: false
      grafana:
        defaultInstallationEnabled: false
      observability:
        enabled: false
      prometheus:
        enabled: false
      global:
        istioSDS:
          enabled: true
        istioIntegration:
          enabled: true
          enableAutoMtls: true
      EOF
        

      Example output:

        NAME: gloo-gateway
      LAST DEPLOYED: Thu Dec 9 11:50:39 2024
      NAMESPACE: gloo-system
      STATUS: deployed
      REVISION: 2
      TEST SUITE: None
        

6. Verify that the Gloo Gateway control plane is up and running.

     kubectl get pods -n gloo-system --context $REMOTE_CONTEXT1
     

   Example output:

     NAME                                READY   STATUS      RESTARTS   AGE
   extauth-64458459c8-9q9kq            1/1     Running     0          26s
   gateway-certgen-qpv5q               0/1     Completed   0          31s
   gloo-68846f8459-nbzxp               1/1     Running     0          26s
   gloo-resource-rollout-check-xskxt   0/1     Completed   0          25s
   gloo-resource-rollout-n9kvf         0/1     Completed   0          26s
   rate-limit-5d8f859465-qhgsg         1/1     Running     0          26s
   redis-9f4f98445-5fh69               1/1     Running     0          26s
     

7. Verify that the gloo-gateway GatewayClass is created. You can optionally take a look at how the gateway class is configured by adding the -o yaml option to your command.

     kubectl get gatewayclass gloo-gateway --context $REMOTE_CONTEXT1
     

   Example output:

     NAME           CONTROLLER             ACCEPTED   AGE
   gloo-gateway   solo.io/gloo-gateway   True       56s
     

Step 3: Create a gateway proxy link

1. Create a gateway resource and configure an HTTP listener. The following gateway can serve HTTP resources from all namespaces.

     kubectl apply --context $REMOTE_CONTEXT1 -n gloo-system -f- <<EOF
   kind: Gateway
   apiVersion: gateway.networking.k8s.io/v1
   metadata:
     name: http
   spec:
     gatewayClassName: gloo-gateway
     listeners:
     - protocol: HTTP
       port: 8080
       name: http
       allowedRoutes:
         namespaces:
           from: All
   EOF
     

2. Verify that the gateway is created successfully. You can also review the external address that is assigned to the gateway. Note that depending on your environment it might take a few minutes for the load balancer service to be assigned an external address.

     kubectl get gateway http -n gloo-system --context $REMOTE_CONTEXT1
     

   Example output:

     NAME   CLASS          ADDRESS                                                                  PROGRAMMED   AGE
   http   gloo-gateway   a3a6c06e2f4154185bf3f8af46abf22e-139567718.us-east-2.elb.amazonaws.com   True         93s
     

3. Check the gateway proxy pod that is created for you. Make sure that you see three containers in your pod (istio-proxy, sds, and gloo-gateway).

   info

   If your gateway proxy is stuck and goes into a CrashLoopBackoff status, review the logs of the istio-proxy and sds containers. You might need to change the Istio values that you provided in your Gloo Gateway Helm installation to point to the correct addresses.

     kubectl get pod -n gloo-system --context $REMOTE_CONTEXT1 | grep gloo-proxy-http  
     

   Example output:

     gloo-proxy-http-bc64bfbd4-qkgvh   3/3     Running   0          124m
     

4. Get the address of your gateway proxy and save it in an environment variable.

     export INGRESS_GW_ADDRESS=$(kubectl get svc --context $REMOTE_CONTEXT1 -n gloo-system gloo-proxy-http -o=jsonpath="{.status.loadBalancer.ingress[0]['hostname','ip']}")
   echo $INGRESS_GW_ADDRESS
     

   Port-forward the gloo-proxy-http pod on port 8080.

     kubectl port-forward deployment/gloo-proxy-http -n gloo-system --context $REMOTE_CONTEXT1 8080:8080
     

Step 4: Expose the product page app link

In this step, you expose the product page app in the gateway proxy. The product page app uses the VirtualDestination that you created as part of step 1 to route traffic to all instances of the reviews app.

1. Set up an HTTPRoute that exposes a route to the productpage app.

     kubectl apply --context $REMOTE_CONTEXT1 -f - <<EOF
   apiVersion: gateway.networking.k8s.io/v1
   kind: HTTPRoute
   metadata:
     name: productpage
     namespace: bookinfo
   spec:
     parentRefs:
       - name: http
         namespace: gloo-system
     rules:
       - matches:
         - path:
             type: Exact
             value: /productpage
         - path: 
             type: PathPrefix
             value: /static
         backendRefs: 
           - name: productpage
             port: 9080
             namespace: bookinfo
   EOF
     

2. Open the product page in your web browser.

     open http://$INGRESS_GW_ADDRESS:8080/productpage 
     

3. Refresh the page a few times. Because you have the Gloo Mesh (Gloo Platform APIs) RouteTable in place, the product page round robins through all the reviews instances in workload clusters 1 and 2 by using the VirtualDestination. Make sure that you see all three versions of the reviews app: black stars, no stars, and red stars.

Step 5: Route to a VirtualDestination link

In this step, you expose the VirtualDestination for the reviews app on your gateway proxy directly. When you send a request to the reviews app, the gateway proxy now uses the internal hostname that was defined in the VirtualDestination to reach all instances of the reviews app.

1. Create an HTTPRoute for the reviews app that references the VirtualDestination in your backendRefs. In order for the HTTPRoute to match the VirtualDestination, you use the hostname that your VirtualDestination is exposed on. Note that this hostname must match the hostname that is defined in the ServiceEntry resource that is automatically created by Gloo Mesh (Gloo Platform APIs) during the translation of your VirtualDestination.

   info

   To view the ServiceEntry that was created for you, run kubectl get serviceentry -n bookinfo -o yaml. Note that the ServiceEntry must list the gloo-system namespace in the spec.exportTo settings for the VirtualDestination integration to work properly. If you do not see the gloo-system namespace, check the Gloo Mesh (Gloo Platform APIs) workspace and workspace settings on your cluster.

     kubectl apply --context $REMOTE_CONTEXT1 -f - <<EOF
   apiVersion: gateway.networking.k8s.io/v1
   kind: HTTPRoute
   metadata:
     name: reviews
     namespace: gloo-system
   spec:
     parentRefs:
       - name: http
         namespace: gloo-system
     rules:
       - matches:
         - path: 
             type: PathPrefix
             value: /reviews
         backendRefs: 
           - name: reviews.mesh.internal.com 
             port: 9080
             kind: Hostname
             group: networking.istio.io
   EOF
     

2. Send multiple requests to the reviews app directly. Make sure that you get back a response from all the reviews app instances.

     for i in {1..5}; do curl -v http://$INGRESS_GW_ADDRESS:8080/reviews/0 ; done 
     

     for i in {1..5};, do curl -i localhost:8080/reviews/0; done
     

   Example output:

     ...
   {"id": "0","podname": "reviews-v1-7cb6f8bdb9-dwxfk","clustername": "null","reviews": [{  "reviewer": "Reviewer1",  "text": "An extremely entertaining play by Shakespeare. The slapstick humour is refreshing!"},{  "reviewer": "Reviewer2",  "text": "Absolutely fun and entertaining. The play lacks thematic depth when compared to other plays by Shakespeare."}]}* 
   ...
   {"id": "0","podname": "reviews-v2-778547d5c9-qqgv4","clustername": "null","reviews": [{  "reviewer": "Reviewer1",  "text": "An extremely entertaining play by Shakespeare. The slapstick humour is refreshing!", "rating": {"stars": 5, "color": "black"}},{  "reviewer": "Reviewer2",  "text": "Absolutely fun and entertaining. The play lacks thematic depth when compared to other plays by Shakespeare.", "rating": {"stars": 4, "color": "black"}}]}*  
   ...
   {"id": "0","podname": "reviews-v3-66d547bb6f-fzzqt","clustername": "null","reviews": [{  "reviewer": "Reviewer1",  "text": "An extremely entertaining play by Shakespeare. The slapstick humour is refreshing!", "rating": {"stars": 5, "color": "red"}},{  "reviewer": "Reviewer2",  "text": "Absolutely fun and entertaining. The play lacks thematic depth when compared to other plays by Shakespeare.", "rating": {"stars": 4, "color": "red"}}]}%  
     

Great job! You successfully exposed a VirtualDestination on your gateway and routed traffic to app instances that are spread across clusters.

Cleanup link

You can optionally remove the resources that you created.

1. Follow the Uninstall guide in the Gloo Mesh (Gloo Platform APIs) documentation to remove Gloo Mesh (Gloo Platform APIs).

2. Follow the upgrade guide to upgrade your Gloo Gateway Helm installation values and disable the Istio integration.

3. Remove the HTTPRoutes for the product page and reviews app.

     kubectl delete httproute productpage reviews -n bookinfo