Route all egress traffic for an external service through an egress gateway and ensure that only certain services in the mesh can send requests to the external endpoint that it serves. All other services in the mesh are blocked from sending requests to the external endpoint. You can also use the egress traffic to perform TLS origination and secure the connection to the external endpoint. For more information, see Perform TLS origination with egress gateways.

Before you begin

  1. Set the names of your clusters from your infrastructure provider. If your clusters have different names, specify those names instead.

      export REMOTE_CLUSTER1=<cluster1>
    export REMOTE_CLUSTER2=<cluster2>
    ...
      
  2. Save the kubeconfig contexts for your clusters. Run kubectl config get-contexts, look for your cluster in the CLUSTER column, and get the context name in the NAME column. Note: Do not use context names with underscores. The generated certificate that connects workload clusters to the management cluster uses the context name as a SAN specification, and underscores in SAN are not FQDN compliant. You can rename a context by running kubectl config rename-context "<oldcontext>" <newcontext>.
      export MGMT_CONTEXT=<management-cluster-context>
    export REMOTE_CONTEXT1=<remote-cluster1-context>
    export REMOTE_CONTEXT2=<remote-cluster2-context>
    ...
      
  3. Decide on the outbound traffic policy for your service mesh. If you followed the steps to install Istio with the Istio lifecycle manager, the outbound traffic policy for your service mesh is set to ALLOW_ANY by default and allows your services in the mesh to reach any external endpoint. You can change this setting and instead block all egress traffic to hosts that are not part of your service mesh by changing the outbound traffic policy to REGISTRY_ONLY as shown in the following steps. Note that if you manually installed Istio, follow the steps in Upgrade Istio to update that setting.

    1. Open the gm-istiod-values.yaml file that you used to create the istiod control plane.
    2. Change the outboundTrafficPolicy field to REGISTRY_ONLY.
    3. Apply your changes by using the following command.
        kubectl apply -f gm-istiod-values.yaml --context $MGMT_CONTEXT
        
  4. Review Supported versions to choose the Solo distribution of Istio that you want to use, and save the version information in the following environment variables.

    • REPO: The repo key for the Solo distribution of Istio that you can get by logging in to the Support Center and reviewing the Istio images built by Solo.io support article.
    • ISTIO_IMAGE: The version that you want to use with the solo tag, such as 1.22.3-patch0-solo. You can optionally append other tags of Solo distributions of Istio as needed.
    • REVISION: Take the Istio major and minor versions and replace the periods with hyphens, such as 1-22.

      export REPO=<repo-key>
    export ISTIO_IMAGE=1.22.3-patch0-solo
    export REVISION=1-22
      

Block egress traffic with an egress gateway

  1. Prepare a GatewayLifecycleManager to deploy an egress gateway.

    1. Download the gm-egress-gateway.yaml example file.

        curl -0L https://raw.githubusercontent.com/solo-io/gloo-mesh-use-cases/main/gloo-mesh/istio-install/gm-managed/gm-egress-gateway.yaml > gm-egress-gateway.yaml
        
    2. Update the example file with the environment variables that you previously set, and optionally further edit the file to provide your own details. Save the updated file as gm-egress-gateway-values.yaml. For more information, see the API reference.

      • Tip: Instead of updating the file manually, try running a terminal command to substitute values, such as the following command.
          envsubst < gm-egress-gateway.yaml > gm-egress-gateway-values.yaml
        open gm-egress-gateway-values.yaml
          
    3. Apply the GatewayLifecycleManager resource to your management cluster.

        kubectl apply -f gm-egress-gateway-values.yaml --context $MGMT_CONTEXT
        
    4. Verify that the egress gateway is deployed in your workload clusters.

        kubectl get pods -n gloo-mesh-gateways --context $REMOTE_CONTEXT1
      kubectl get pods -n gloo-mesh-gateways --context $REMOTE_CONTEXT2
        
  2. Log in to the reviews app and verify that you cannot reach www.google.com.

      kubectl --context ${REMOTE_CONTEXT1} -n bookinfo debug -i pods/$(kubectl get pod --context ${REMOTE_CONTEXT1} -l app=reviews -A -o jsonpath='{.items[0].metadata.name}') --image=curlimages/curl -- curl -vik www.google.com
      

    Example output:

      * Mark bundle as not supporting multiuse
    < HTTP/1.1 502 Bad Gateway
    HTTP/1.1 502 Bad Gateway
    < date: Thu, 11 May 2023 17:07:43 GMT
    date: Thu, 11 May 2023 17:07:43 GMT
    < server: envoy
    server: envoy
    < content-length: 0
    content-length: 0
      
  3. Create the global namespace in one of your workload clusters.

      kubectl create namespace global --context $REMOTE_CONTEXT1
      
  4. Create a virtual gateway resource to configure your egress gateway going forward.

      kubectl apply --context $REMOTE_CONTEXT1 -f- <<EOF
    apiVersion: networking.gloo.solo.io/v2
    kind: VirtualGateway
    metadata:  
      annotations:
        cluster.solo.io/cluster: ""
      labels:
        app: istio-egressgateway
        istio: egressgateway
      name: egress-vg
      namespace: gloo-mesh-gateways
    spec:
      listeners:
      - http: {}
        appProtocol: HTTPS
        exposedExternalServices:
        - host: www.google.com
        port:
          number: 443
        tls:
          mode: ISTIO_MUTUAL
      workloads:
      - selector:
          labels:
            app: istio-egressgateway
            istio: egressgateway
    EOF
      
    SettingDescription
    spec.listeners.exposedExternalServices.hostSelect the external services that you want to serve on this egress gateway. To select the external services, you define the hostname that must be configured in the external service.
    spec.listeners.portEnter the port number that services in the mesh use in combination with the host to send a request to the egress gateway.
    spec.listeners.tlsSelect the TLS mode that you want to use to connect to the egress gateway from within the mesh. In this example, requests for the www.google.com host are rerouted to the egress gateway first by using mutual TLS. The incoming TLS connection is terminated at the egress gateway.
  5. Create an external service for www.google.com.

      kubectl apply --context $REMOTE_CONTEXT1 -f- <<EOF
    apiVersion: networking.gloo.solo.io/v2
    kind: ExternalService
    metadata:
      name: google
      namespace: global
    spec:
      hosts:
      - "www.google.com"
      ports:
        - clientsideTls: {}
          egressGatewayRoutes: 
            portMatch: 80
            virtualGatewayRefs:
              - cluster: $REMOTE_CLUSTER1
                name: egress-vg
                namespace: gloo-mesh-gateways
              - cluster: $REMOTE_CLUSTER2
                name: egress-vg
                namespace: gloo-mesh-gateways
          name: https
          number: 443
          protocol: HTTPS
    EOF
      
    SettingDescription
    spec.hostsEnter the external hostname that you want to allow access to for services in the mesh.
    spec.ports.clientsideTlsAllow TLS origination to the www.google.com endpoint.
    spec.ports.egressGatewayRoutes.virtualGatewayRefsEnter the details for your egress gateway. Make sure to enter the egress gateway details for both of your workload clusters.
    spec.ports.egressGatewayRoutes.portMatchSelect the port that clients use when sending a request to the external endpoint. Note that this port does not represent the port that the client’s sidecar proxy uses to connect to the egress gateway. Instead, make sure to enter the source port that is used in the original client request.
    spec.ports.nameEnter the port number that you want to use to connect to the external endpoint. When a client request is sent to the gateway, the client’s sidecar proxy intercepts this traffic and securely connects to the egress gateway via mutual TLS. The egress gateway then terminates the incoming TLS connection. Because this example defines an HTTPS port, a new TLS connection is originated at the egress gateway to securely connect to the www.google.com external endpoint.
  6. Verify that you can access www.google.com from the reviews app.

      kubectl --context ${REMOTE_CONTEXT1} -n bookinfo debug -i pods/$(kubectl get pod --context ${REMOTE_CONTEXT1} -l app=reviews -A -o jsonpath='{.items[0].metadata.name}') --image=curlimages/curl -- curl -vik www.google.com
      

    Example output:

      * Connected to www.google.com (240.240.59.19) port 80 (#0)
    > GET / HTTP/1.1
    > Host: www.google.com
    > User-Agent: curl/8.1.1-DEV
    > Accept: */*
    > 
    < HTTP/1.1 200 OK
    < date: Wed, 24 May 2023 20:23:50 GMT
    < expires: -1
    < cache-control: private, max-age=0
    < content-type: text/html; charset=ISO-8859-1
    < content-security-policy-report-only: object-src 'none';base-uri 'self';script-src 'nonce-k4p_jp_okP7dor-UgJJt5Q' 'strict-dynamic' 'report-sample' 'unsafe-eval' 'unsafe-inline' https: http:;report-uri https://csp.withgoogle.com/csp/gws/other-hp
    < p3p: CP="This is not a P3P policy! See g.co/p3phelp for more info."
    < server: envoy
    HTTP/1.1 200 OK
    ...
      
  7. Verify that traffic was routed through the egress gateway by looking at the egress gateway logs.

      kubectl logs $(kubectl get pod --context ${REMOTE_CONTEXT1} -l app=istio-egressgateway -A -o jsonpath='{.items[0].metadata.name}') -n gloo-mesh-gateways --context $REMOTE_CONTEXT1
      

    Example output:

      {"upstream_service_time":"55","response_code":200,"response_flags":"-","authority":"www.google.com","bytes_received":0,"duration":55,"method":"GET","request_id":"f0d0412a-a393-9190-a49b-23baaadf7d22","route_name":null,"requested_server_name":"www.google.com","upstream_host":"142.251.161.105:443","downstream_remote_address":"10.84.1.21:52038","downstream_local_address":"10.84.0.24:8443","start_time":"2023-05-24T20:16:57.753Z","bytes_sent":17238,"response_code_details":"via_upstream","upstream_cluster":"outbound|443||www.google.com","upstream_transport_failure_reason":null,"path":"/","x_forwarded_for":"10.84.1.21","user_agent":"curl/8.1.1-DEV","upstream_local_address":"10.84.0.24:57268","connection_termination_details":null,"protocol":"HTTP/1.1"}
      
  8. Get the virtual service that was created for your external service and make sure that it lists port 443 for the www.google.com destination. This setting ensures the egress gateway originates a new TLS connection before connecting to www.google.com.

      kubectl get virtualservices -n gloo-mesh-gateways --context $REMOTE_CONTEXT1 | grep externalservice-google
      
      kubectl get virtualservice <virtual-service-name> -n gloo-mesh-gateways --context $REMOTE_CONTEXT1 -o yaml
      

    Example output:

      ...
    spec:
      exportTo:
      - bookinfo
      - default
      - global
      - gloo-mesh
      - gloo-mesh-gateways
      - gm-iop-1-22
      - helloworld
      - httpbin
      - istio-system
      gateways:
      - virtualgateway-egress-vg-gloo-m-ef7dba3d044465c79ef179255be34f5
      - mesh
      hosts:
      - www.google.com
      http:
      - match:
        - gateways:
          - virtualgateway-egress-vg-gloo-m-ef7dba3d044465c79ef179255be34f5
          port: 8443
        route:
        - destination:
            host: www.google.com
            port:
              number: 443
    ...
      
  9. Create an access policy for your external service to allow only the product page app to access the www.google.com external endpoint.

      kubectl apply --context $REMOTE_CONTEXT1 -f- <<EOF
    apiVersion: security.policy.gloo.solo.io/v2
    kind: AccessPolicy
    metadata:
      annotations:
        cluster.solo.io/cluster: ""
      name: egress-access-policy
      namespace: gloo-mesh-gateways
    spec:
      applyToDestinations:
      - kind: EXTERNAL_SERVICE
        port:
          number: 443
        selector:
          cluster: $REMOTE_CLUSTER1
          name: google
          namespace: global
      config:
        authz:
          allowedClients:
          - serviceAccountSelector:
              cluster: $REMOTE_CLUSTER1
              labels:
                account: productpage
              namespace: bookinfo
    EOF
      
  10. Send a request to www.google.com from the reviews app and verify that this request is denied with a 403 HTTP response code.

      kubectl --context ${REMOTE_CONTEXT1} -n bookinfo debug -i pods/$(kubectl get pod --context ${REMOTE_CONTEXT1} -l app=reviews -A -o jsonpath='{.items[0].metadata.name}') --image=curlimages/curl -- curl -vik www.google.com
      

    Example output:

      * Connected to www.google.com (240.240.59.19) port 80 (#0)
    > GET / HTTP/1.1
    > Host: www.google.com
    > User-Agent: curl/8.1.1-DEV
    > Accept: */*
    > 
    < HTTP/1.1 403 Forbidden
    < content-length: 19
    < content-type: text/plain
    < date: Wed, 24 May 2023 20:58:55 GMT
    < server: envoy
    < x-envoy-upstream-service-time: 13
    < 
    { [19 bytes data]
    HTTP/1.1 403 Forbidden
    
    * Connection #0 to host www.google.com left intact
    content-length: 19
    content-type: text/plain
    date: Wed, 24 May 2023 20:58:55 GMT
    server: envoy
    x-envoy-upstream-service-time: 13
    
    RBAC: access denied%        
      
  11. Send another request to www.google.com from the product page app and verify that this request is accepted.

      kubectl --context ${REMOTE_CONTEXT1} -n bookinfo debug -i pods/$(kubectl get pod --context ${REMOTE_CONTEXT1} -l app=productpage -A -o jsonpath='{.items[0].metadata.name}') --image=curlimages/curl -- curl -vik www.google.com
      

    Example output:

      * Connected to www.google.com (240.240.59.19) port 80 (#0)
    > GET / HTTP/1.1
    > Host: www.google.com
    > User-Agent: curl/8.1.1-DEV
    > Accept: */*
    > 
    < HTTP/1.1 200 OK
    < date: Wed, 24 May 2023 21:01:44 GMT
    < expires: -1
    < cache-control: private, max-age=0
    < content-type: text/html; charset=ISO-8859-1
    HTTP/1.1 200 OK
      

Cleanup

You can optionally remove the resources that you set up as part of this guide.
  kubectl delete externalservice google -n global --context $REMOTE_CONTEXT1
kubectl delete virtualgateway egress-vg -n gloo-mesh-gateways --context $REMOTE_CONTEXT1
kubectl delete gatewaylifecyclemanager istio-egressgateway -n gloo-mesh --context $MGMT_CONTEXT
kubectl delete accesspolicy egress-access-policy -n gloo-mesh-gateways --context $REMOTE_CONTEXT1
kubectl delete ns global --context $REMOTE_CONTEXT1