Ingress with Gloo Mesh virtual destinations

About Gloo Mesh (Gloo Platform APIs) link

Gloo Mesh (Gloo Platform APIs) is a service mesh management plane that is based on hardened, open-source projects like Envoy and Istio. With Gloo Mesh, you can unify the configuration, operation, and visibility of service-to-service connectivity across your distributed applications. These apps can run in different virtual machines (VMs) or Kubernetes clusters on premises or in various cloud providers, and even in different service meshes.

About VirtualDestinations link

A VirtualDestination is a traffic management concept in Gloo Mesh (Gloo Platform APIs) that allows you to define unique internal hostnames for services that are spread across a multicluster service mesh. Without VirtualDestinations, you must update the IP addresses of your services every time the IP address changes, such as when you update your app or deploy a new version. Because a VirtualDestination uses labels to select the backing services, it can automatically detect IP address changes and update them for you.

To learn more about VirtualDestinations, see the VirtualDestinations conceptual overview and multicluster routing guide in the Gloo Mesh (Gloo Platform APIs) documentation.

About this guide link

In this guide, you explore how to use Gloo Mesh (Gloo Platform APIs) and Gloo Gateway together to route traffic to Bookinfo services that are spread across clusters. Bookinfo is a sample microservices-based app that is provided by Istio and composed of 4 different microservices that interact with each other. The app is commonly used to demonstrate Istio’s service mesh features and capabilities.

To accomplish multicluster routing with Gloo Gateway and Gloo Mesh (Gloo Platform APIs), you leverage VirtualDestinations. You can use VirtualDestinations in multiple ways in Gloo Gateway. This guide provides steps to accomplish the following tasks:

Expose the product page app on a gateway, which is part of a service mesh that Gloo Mesh (Gloo Platform APIs) manages. The product page app uses a VirtualDestination to round robin requests to reviews app instances that are spread across multiple clusters. The VirtualDestination is not exposed on the gateway directly, but rather used within the service mesh only.

Expose the reviews VirtualDestination on the gateway directly. This way, you can explore how to use a unique internal hostname to round robin traffic to reviews app instances that are spread across clusters.

Step 1: Install Gloo Mesh (Gloo Platform APIs) link

1. Install Gloo Mesh (Gloo Platform APIs) by following the multicluster getting started tutorial in the Gloo Mesh (Gloo Platform APIs) documentation. This guide creates a three-cluster setup with one management cluster that runs the Gloo Mesh (Gloo Platform APIs) control plane and two workload clusters that run the data plane. You also install a Solo distribution of Istio in both of your workload clusters by using the Gloo Operator.

2. Deploy the sample Bookinfo app. You use this app to demonstrate traffic routing in a multicluster service mesh. You can optionally install other sample apps, such as httpbin or helloworld. However, these apps are not used in this guide.

3. Set up multicluster routing for the reviews app by using a VirtualDestination and RouteTable resource. Make sure that the productpage app can route traffic to all versions of the reviews app before you continue with installing Gloo Gateway.

4. Review the VirtualDestination that you created in the previous step. The VirtualDestination specifies the reviews.mesh.internal.com internal hostname that services in the mesh can use to reach all instances of the reviews app.

     kubectl get virtualdestination reviews-vd --context $MGMT_CONTEXT -n bookinfo -o yaml
     

   Example output:

     apiVersion: networking.gloo.solo.io/v2
   kind: VirtualDestination
   metadata:
     name: reviews-vd
     namespace: bookinfo
   spec:
     hosts:
     # Arbitrary, internal-only hostname assigned to the endpoint
     - reviews.mesh.internal.com
     ports:
     - number: 9080
       protocol: HTTP
     services:
       - labels:
           app: reviews
     

Step 2: Install Gloo Gateway link

After you successfully installed Gloo Mesh (Gloo Platform APIs) and confirmed that you can route traffic from the product page app to all versions of the reviews app, you can now continue to install Gloo Gateway. Gloo Gateway must be installed in a cluster that also has Istio installed. You can choose to install Gloo Gateway in one or each of your workload clusters.

Install Gloo Gateway in workload cluster 1 with the Istio integration enabled.

Example commands:

  helm upgrade -i -n gloo-system gloo-gateway oci://us-docker.pkg.dev/solo-public/gloo-gateway/charts/gloo-gateway \
--version 2.0.2 \
--set controller.extraEnv.KGW_ENABLE_ISTIO_AUTO_MTLS=true
  

  
controller:
  extraEnv:
    KGW_ENABLE_ISTIO_AUTO_MTLS: true
  

Revisioned istiod and custom Istio meta mesh settings link

If you installed Istio with a revision, or you set up a custom Istio meta cluster ID and meta mesh, you must create a GatewayParameters resource to change the default Istio settings on your gateway proxy.

If you have a revisionless istiod setup and did not customize the Istio meta cluster ID and meta mesh ID, you can skip to the next step.

1. Get the name of the istiod service. Depending on how you set up Istio, you might see a revisionless service name (istiod) or a service name with a revision, such as istiod-main.

     kubectl get services -n istio-system --context $REMOTE_CONTEXT1
     

   Example output:

     NAME          TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                                 AGE
   istiod-main   ClusterIP   34.118.238.13   <none>        15010/TCP,15012/TCP,443/TCP,15014/TCP   3d2h
     

2. Derive the Kubernetes service address for your istiod deployment. The service address uses the format <service-name>.<namespace>.svc:15012. For example, if your service name is istiod-main, the full service address is istiod-main.istio-system.svc:15012.

3. Create a GatewayParameters resource to configure the revisioned istiod service address and any custom values for the Istio meta cluster ID and meta mesh ID. Note: If you do not set custom values for the cluster ID and mesh ID, you can omit these values. Then, the default values of Kubernetes for the cluster ID and cluster.local for the mesh ID are used.

     kubectl apply -f- <<EOF
   apiVersion: gateway.kgateway.dev/v1alpha1
   kind: GatewayParameters
   metadata:
     name: custom-gw-params
     namespace: gloo-system
   spec:
     kube: 
       istio:
         istioProxyContainer:
           istioDiscoveryAddress: <istiod-service-address> # such as istiod-main.istio-system.svc:15012
           istioMetaClusterId: <cluster-ID> ## such as the cluster name
           istioMetaMeshId: <meta-mesh-ID> ## such as the cluster name
       sdsContainer:
         image:
           registry: cr.kgateway.dev/kgateway-dev
           repository: sds
           tag: v2.1.0-main
   EOF
     

Step 3: Create a gateway proxy link

Create or update a Gateway that includes the Istio proxy.

1. Change the http gateway from the getting started tutorial to apply the custom settings of the GatewayParameters resource.

     kubectl apply -f- <<EOF
   kind: Gateway
   apiVersion: gateway.networking.k8s.io/v1
   metadata:
     name: http
     namespace: gloo-system
   spec:
     gatewayClassName: gloo-gateway-v2
     infrastructure:
       parametersRef:
         name: custom-gw-params
         group: gateway.kgateway.dev
         kind: GatewayParameters
     listeners:
     - protocol: HTTP
       port: 8080
       name: http
       allowedRoutes:
         namespaces:
           from: All
   EOF
     

2. Verify that the gateway proxy restarts with three containers. You might need to manually restart the proxy to apply the latest Gateway settings.

     kubectl get pods -n gloo-system -l gateway.networking.k8s.io/gateway-name=http \
     --context $REMOTE_CONTEXT1 \
     -o jsonpath='{range .items[*]}Pod: {.metadata.name} | Status: {.status.phase}{"\n"}Containers:{"\n"}{range .spec.containers[*]}- {.name}{"\n"}{end}{"\n"}{end}'
     

   Example output: Note that the pod is running and has three containers, including the istio-proxy.

     Pod: http-f7c7f4b78-pwgnt | Status: Running
   Containers:
   - kgateway-proxy
   - sds
   - istio-proxy
     

Step 4: Expose the productpage app link

In this step, you expose the product page app on the gateway proxy. The product page app uses the VirtualDestination that you created as part of step 1 to route traffic to all instances of the reviews app.

1. Set up an HTTPRoute that exposes a route to the productpage app.

     kubectl apply --context $REMOTE_CONTEXT1 -f - <<EOF
   apiVersion: gateway.networking.k8s.io/v1
   kind: HTTPRoute
   metadata:
     name: productpage
     namespace: bookinfo
   spec:
     parentRefs:
       - name: http
         namespace: gloo-system
     rules:
       - matches:
         - path:
             type: Exact
             value: /productpage
         - path: 
             type: PathPrefix
             value: /static
         backendRefs: 
           - name: productpage
             port: 9080
             namespace: bookinfo
   EOF
     

2. Open the product page in your web browser.

     http://$INGRESS_GW_ADDRESS:8080/productpage 
     

   localhost:8080/productpage

3. Refresh the page a few times. Because you have the Gloo Mesh (Gloo Platform APIs) RouteTable in place, the product page round robins through all the reviews instances in workload clusters 1 and 2 by using the VirtualDestination. Make sure that you see all three versions of the reviews app: black stars, no stars, and red stars.

Step 5: Route to a VirtualDestination link

In this step, you expose the VirtualDestination for the reviews app on your gateway proxy directly. When you send a request to the reviews app, the gateway proxy now uses the internal hostname that was defined in the VirtualDestination to reach all instances of the reviews app.

1. Create an HTTPRoute for the reviews app that references the VirtualDestination in your backendRefs. In order for the HTTPRoute to match the VirtualDestination, you use the hostname that your VirtualDestination is exposed on. Note that this hostname must match the hostname that is defined in the ServiceEntry resource that is automatically created by Gloo Mesh (Gloo Platform APIs) during the translation of your VirtualDestination.

   info

   To view the ServiceEntry that was created for you, run kubectl get serviceentry -n bookinfo -o yaml. Note that the ServiceEntry must list the gloo-system namespace in the spec.exportTo settings for the VirtualDestination integration to work properly. If you do not see the gloo-system namespace, check the Gloo Mesh (Gloo Platform APIs) workspace and workspace settings on your cluster.

     kubectl apply --context $REMOTE_CONTEXT1 -f - <<EOF
   apiVersion: gateway.networking.k8s.io/v1
   kind: HTTPRoute
   metadata:
     name: reviews
     namespace: gloo-system
   spec:
     parentRefs:
       - name: http
         namespace: gloo-system
     rules:
       - matches:
         - path: 
             type: PathPrefix
             value: /reviews
         backendRefs: 
           - name: reviews.mesh.internal.com 
             port: 9080
             kind: Hostname
             group: networking.istio.io
   EOF
     

2. Send multiple requests to the reviews app directly. Make sure that you get back a response from all the reviews app instances.

     for i in {1..5}; do curl -v http://$INGRESS_GW_ADDRESS:8080/reviews/0 ; done 
     

     for i in {1..5};, do curl -i localhost:8080/reviews/0; done
     

   Example output:

     ...
   {"id": "0","podname": "reviews-v1-7cb6f8bdb9-dwxfk","clustername": "null","reviews": [{  "reviewer": "Reviewer1",  "text": "An extremely entertaining play by Shakespeare. The slapstick humour is refreshing!"},{  "reviewer": "Reviewer2",  "text": "Absolutely fun and entertaining. The play lacks thematic depth when compared to other plays by Shakespeare."}]}* 
   ...
   {"id": "0","podname": "reviews-v2-778547d5c9-qqgv4","clustername": "null","reviews": [{  "reviewer": "Reviewer1",  "text": "An extremely entertaining play by Shakespeare. The slapstick humour is refreshing!", "rating": {"stars": 5, "color": "black"}},{  "reviewer": "Reviewer2",  "text": "Absolutely fun and entertaining. The play lacks thematic depth when compared to other plays by Shakespeare.", "rating": {"stars": 4, "color": "black"}}]}*  
   ...
   {"id": "0","podname": "reviews-v3-66d547bb6f-fzzqt","clustername": "null","reviews": [{  "reviewer": "Reviewer1",  "text": "An extremely entertaining play by Shakespeare. The slapstick humour is refreshing!", "rating": {"stars": 5, "color": "red"}},{  "reviewer": "Reviewer2",  "text": "Absolutely fun and entertaining. The play lacks thematic depth when compared to other plays by Shakespeare.", "rating": {"stars": 4, "color": "red"}}]}%  
     

Great job! You successfully exposed a VirtualDestination on your gateway and routed traffic to app instances that are spread across clusters.

Cleanup link

You can optionally remove the resources that you created.

1. Follow the Uninstall guide in the Gloo Mesh (Gloo Platform APIs) documentation to remove Gloo Mesh (Gloo Platform APIs).

2. Follow the upgrade guide to upgrade your Gloo Gateway Helm installation values and disable the Istio integration.

3. Remove the HTTPRoutes for the product page and reviews app.

     kubectl delete httproute productpage reviews -n bookinfo --context $REMOTE_CONTEXT1 
   kubectl delete gatewayparameters custom-gw-params -n gloo-system --context $REMOTE_CONTEXT1
     

4. Restore the http Gateway from the getting started tutorial.

     kubectl apply -f- --context $REMOTE_CONTEXT1 <<EOF
   kind: Gateway
   apiVersion: gateway.networking.k8s.io/v1
   metadata:
     name: http
     namespace: gloo-system
   spec:
     gatewayClassName: gloo-gateway
     listeners:
     - protocol: HTTP
       port: 8080
       name: http
       allowedRoutes:
         namespaces:
           from: All
   EOF
     

5. Follow the Istio documentation to uninstall Istio and remove the Bookinfo sample app.