Route all egress traffic for an external service through an egress gateway and ensure that only certain services in the mesh can send requests to the external endpoint that it serves. All other services in the mesh are blocked from sending requests to the external endpoint. You can also use the egress traffic to perform TLS origination and secure the connection to the external endpoint. For more information, see Perform TLS origination with egress gateways.

Before you begin

  1. Set the names of your clusters from your infrastructure provider. If your clusters have different names, specify those names instead.

      export REMOTE_CLUSTER1=<cluster1>
export REMOTE_CLUSTER2=<cluster2>
...
  2. Save the kubeconfig contexts for your clusters. Run kubectl config get-contexts, look for your cluster in the CLUSTER column, and get the context name in the NAME column. Note: Do not use context names with underscores. The generated certificate that connects workload clusters to the management cluster uses the context name as a SAN specification, and underscores in SAN are not FQDN compliant. You can rename a context by running kubectl config rename-context "<oldcontext>" <newcontext>.
      export MGMT_CONTEXT=<management-cluster-context>
export REMOTE_CONTEXT1=<remote-cluster1-context>
export REMOTE_CONTEXT2=<remote-cluster2-context>
...

  3. Decide on the outbound traffic policy for your service mesh. If you followed the steps to install Istio with the Istio lifecycle manager, the outbound traffic policy for your service mesh is set to ALLOW_ANY by default and allows your services in the mesh to reach any external endpoint. You can change this setting and instead block all egress traffic to hosts that are not part of your service mesh by changing the outbound traffic policy to REGISTRY_ONLY as shown in the following steps. Note that if you manually installed Istio, follow the steps in Upgrade Istio to update that setting.

    1. Open the gm-istiod-values.yaml file that you used to create the istiod control plane.
    2. Change the outboundTrafficPolicy field to REGISTRY_ONLY.
    3. Apply your changes by using the following command.
        kubectl apply -f gm-istiod-values.yaml --context $MGMT_CONTEXT

  4. Review Supported versions to choose the Solo distribution of Istio that you want to use, and save the version information in the following environment variables.

    • REPO: The repo key for the Solo distribution of Istio that you can get by logging in to the Support Center and reviewing the Istio images built by Solo.io support article.
    • ISTIO_IMAGE: The version that you want to use with the solo tag, such as 1.24.1-patch1-solo. You can optionally append other tags of Solo distributions of Istio as needed.
    • REVISION: Take the Istio major and minor versions and replace the periods with hyphens, such as 1-24.

      export REPO=<repo-key>
export ISTIO_IMAGE=1.24.1-patch1-solo
export REVISION=1-24

Block egress traffic with an egress gateway

  1. Prepare a GatewayLifecycleManager to deploy an egress gateway.

    1. Download the gm-egress-gateway.yaml example file.

        curl -0L https://raw.githubusercontent.com/solo-io/gloo-mesh-use-cases/main/gloo-mesh/istio-install/gm-managed/gm-egress-gateway.yaml > gm-egress-gateway.yaml

    2. Update the example file with the environment variables that you previously set, and optionally further edit the file to provide your own details. Save the updated file as gm-egress-gateway-values.yaml. For more information, see the API reference.

      • Tip: Instead of updating the file manually, try running a terminal command to substitute values, such as the following command.
          envsubst < gm-egress-gateway.yaml > gm-egress-gateway-values.yaml
open gm-egress-gateway-values.yaml

    3. Apply the GatewayLifecycleManager resource to your management cluster.

        kubectl apply -f gm-egress-gateway-values.yaml --context $MGMT_CONTEXT

    4. Verify that the egress gateway is deployed in your workload clusters.

        kubectl get pods -n gloo-mesh-gateways --context $REMOTE_CONTEXT1
kubectl get pods -n gloo-mesh-gateways --context $REMOTE_CONTEXT2

  2. Log in to the reviews app and verify that you cannot reach www.google.com.

      kubectl --context ${REMOTE_CONTEXT1} -n bookinfo debug -i pods/$(kubectl get pod --context ${REMOTE_CONTEXT1} -l app=reviews -A -o jsonpath='{.items[0].metadata.name}') --image=curlimages/curl -- curl -vik www.google.com

    Example output:

      * Mark bundle as not supporting multiuse
< HTTP/1.1 502 Bad Gateway
HTTP/1.1 502 Bad Gateway
< date: Thu, 11 May 2023 17:07:43 GMT
date: Thu, 11 May 2023 17:07:43 GMT
< server: envoy
server: envoy
< content-length: 0
content-length: 0

  3. Create the global namespace in one of your workload clusters.

      kubectl create namespace global --context $REMOTE_CONTEXT1

  4. Create a virtual gateway resource to configure your egress gateway going forward.

      kubectl apply --context $REMOTE_CONTEXT1 -f- <<EOF
apiVersion: networking.gloo.solo.io/v2
kind: VirtualGateway
metadata:  
  annotations:
    cluster.solo.io/cluster: ""
  labels:
    app: istio-egressgateway
    istio: egressgateway
  name: egress-vg
  namespace: gloo-mesh-gateways
spec:
  listeners:
  - http: {}
    appProtocol: HTTPS
    exposedExternalServices:
    - host: www.google.com
    port:
      number: 443
    tls:
      mode: ISTIO_MUTUAL
  workloads:
  - selector:
      labels:
        app: istio-egressgateway
        istio: egressgateway
EOF
    SettingDescription
    spec.listeners.exposedExternalServices.hostSelect the external services that you want to serve on this egress gateway. To select the external services, you define the hostname that must be configured in the external service.
    spec.listeners.portEnter the port number that services in the mesh use in combination with the host to send a request to the egress gateway.
    spec.listeners.tlsSelect the TLS mode that you want to use to connect to the egress gateway from within the mesh. In this example, requests for the www.google.com host are rerouted to the egress gateway first by using mutual TLS. The incoming TLS connection is terminated at the egress gateway.

  5. Create an external service for www.google.com.

      kubectl apply --context $REMOTE_CONTEXT1 -f- <<EOF
apiVersion: networking.gloo.solo.io/v2
kind: ExternalService
metadata:
  name: google
  namespace: global
spec:
  hosts:
  - "www.google.com"
  ports:
    - clientsideTls: {}
      egressGatewayRoutes: 
        portMatch: 80
        virtualGatewayRefs:
          - cluster: $REMOTE_CLUSTER1
            name: egress-vg
            namespace: gloo-mesh-gateways
          - cluster: $REMOTE_CLUSTER2
            name: egress-vg
            namespace: gloo-mesh-gateways
      name: https
      number: 443
      protocol: HTTPS
EOF
    SettingDescription
    spec.hostsEnter the external hostname that you want to allow access to for services in the mesh.
    spec.ports.clientsideTlsAllow TLS origination to the www.google.com endpoint.
    spec.ports.egressGatewayRoutes.virtualGatewayRefsEnter the details for your egress gateway. Make sure to enter the egress gateway details for both of your workload clusters.
    spec.ports.egressGatewayRoutes.portMatchSelect the port that clients use when sending a request to the external endpoint. Note that this port does not represent the port that the client’s sidecar proxy uses to connect to the egress gateway. Instead, make sure to enter the source port that is used in the original client request.
    spec.ports.nameEnter the port number that you want to use to connect to the external endpoint. When a client request is sent to the gateway, the client’s sidecar proxy intercepts this traffic and securely connects to the egress gateway via mutual TLS. The egress gateway then terminates the incoming TLS connection. Because this example defines an HTTPS port, a new TLS connection is originated at the egress gateway to securely connect to the www.google.com external endpoint.

  6. Verify that you can access www.google.com from the reviews app.

      kubectl --context ${REMOTE_CONTEXT1} -n bookinfo debug -i pods/$(kubectl get pod --context ${REMOTE_CONTEXT1} -l app=reviews -A -o jsonpath='{.items[0].metadata.name}') --image=curlimages/curl -- curl -vik www.google.com

    Example output:

      * Connected to www.google.com (240.240.59.19) port 80 (#0)
> GET / HTTP/1.1
> Host: www.google.com
> User-Agent: curl/8.1.1-DEV
> Accept: */*
> 
< HTTP/1.1 200 OK
< date: Wed, 24 May 2023 20:23:50 GMT
< expires: -1
< cache-control: private, max-age=0
< content-type: text/html; charset=ISO-8859-1
< content-security-policy-report-only: object-src 'none';base-uri 'self';script-src 'nonce-k4p_jp_okP7dor-UgJJt5Q' 'strict-dynamic' 'report-sample' 'unsafe-eval' 'unsafe-inline' https: http:;report-uri https://csp.withgoogle.com/csp/gws/other-hp
< p3p: CP="This is not a P3P policy! See g.co/p3phelp for more info."
< server: envoy
HTTP/1.1 200 OK
...

  7. Verify that traffic was routed through the egress gateway by looking at the egress gateway logs.

      kubectl logs $(kubectl get pod --context ${REMOTE_CONTEXT1} -l app=istio-egressgateway -A -o jsonpath='{.items[0].metadata.name}') -n gloo-mesh-gateways --context $REMOTE_CONTEXT1

    Example output:

      {"upstream_service_time":"55","response_code":200,"response_flags":"-","authority":"www.google.com","bytes_received":0,"duration":55,"method":"GET","request_id":"f0d0412a-a393-9190-a49b-23baaadf7d22","route_name":null,"requested_server_name":"www.google.com","upstream_host":"142.251.161.105:443","downstream_remote_address":"10.84.1.21:52038","downstream_local_address":"10.84.0.24:8443","start_time":"2023-05-24T20:16:57.753Z","bytes_sent":17238,"response_code_details":"via_upstream","upstream_cluster":"outbound|443||www.google.com","upstream_transport_failure_reason":null,"path":"/","x_forwarded_for":"10.84.1.21","user_agent":"curl/8.1.1-DEV","upstream_local_address":"10.84.0.24:57268","connection_termination_details":null,"protocol":"HTTP/1.1"}

  8. Get the virtual service that was created for your external service and make sure that it lists port 443 for the www.google.com destination. This setting ensures the egress gateway originates a new TLS connection before connecting to www.google.com.

      kubectl get virtualservices -n gloo-mesh-gateways --context $REMOTE_CONTEXT1 | grep externalservice-google
  
      kubectl get virtualservice <virtual-service-name> -n gloo-mesh-gateways --context $REMOTE_CONTEXT1 -o yaml

    Example output:

      ...
spec:
  exportTo:
  - bookinfo
  - default
  - global
  - gloo-mesh
  - gloo-mesh-gateways
  - gm-iop-1-24
  - helloworld
  - httpbin
  - istio-system
  gateways:
  - virtualgateway-egress-vg-gloo-m-ef7dba3d044465c79ef179255be34f5
  - mesh
  hosts:
  - www.google.com
  http:
  - match:
    - gateways:
      - virtualgateway-egress-vg-gloo-m-ef7dba3d044465c79ef179255be34f5
      port: 8443
    route:
    - destination:
        host: www.google.com
        port:
          number: 443
...

  9. Create an access policy for your external service to allow only the product page app to access the www.google.com external endpoint.

      kubectl apply --context $REMOTE_CONTEXT1 -f- <<EOF
apiVersion: security.policy.gloo.solo.io/v2
kind: AccessPolicy
metadata:
  annotations:
    cluster.solo.io/cluster: ""
  name: egress-access-policy
  namespace: gloo-mesh-gateways
spec:
  applyToDestinations:
  - kind: EXTERNAL_SERVICE
    port:
      number: 443
    selector:
      cluster: $REMOTE_CLUSTER1
      name: google
      namespace: global
  config:
    authz:
      allowedClients:
      - serviceAccountSelector:
          cluster: $REMOTE_CLUSTER1
          labels:
            account: productpage
          namespace: bookinfo
EOF

  10. Send a request to www.google.com from the reviews app and verify that this request is denied with a 403 HTTP response code.

      kubectl --context ${REMOTE_CONTEXT1} -n bookinfo debug -i pods/$(kubectl get pod --context ${REMOTE_CONTEXT1} -l app=reviews -A -o jsonpath='{.items[0].metadata.name}') --image=curlimages/curl -- curl -vik www.google.com

    Example output:

      * Connected to www.google.com (240.240.59.19) port 80 (#0)
> GET / HTTP/1.1
> Host: www.google.com
> User-Agent: curl/8.1.1-DEV
> Accept: */*
> 
< HTTP/1.1 403 Forbidden
< content-length: 19
< content-type: text/plain
< date: Wed, 24 May 2023 20:58:55 GMT
< server: envoy
< x-envoy-upstream-service-time: 13
< 
{ [19 bytes data]
HTTP/1.1 403 Forbidden

* Connection #0 to host www.google.com left intact
content-length: 19
content-type: text/plain
date: Wed, 24 May 2023 20:58:55 GMT
server: envoy
x-envoy-upstream-service-time: 13

RBAC: access denied%

  11. Send another request to www.google.com from the product page app and verify that this request is accepted.

      kubectl --context ${REMOTE_CONTEXT1} -n bookinfo debug -i pods/$(kubectl get pod --context ${REMOTE_CONTEXT1} -l app=productpage -A -o jsonpath='{.items[0].metadata.name}') --image=curlimages/curl -- curl -vik www.google.com

    Example output:

      * Connected to www.google.com (240.240.59.19) port 80 (#0)
> GET / HTTP/1.1
> Host: www.google.com
> User-Agent: curl/8.1.1-DEV
> Accept: */*
> 
< HTTP/1.1 200 OK
< date: Wed, 24 May 2023 21:01:44 GMT
< expires: -1
< cache-control: private, max-age=0
< content-type: text/html; charset=ISO-8859-1
HTTP/1.1 200 OK

Cleanup

You can optionally remove the resources that you set up as part of this guide.
  kubectl delete externalservice google -n global --context $REMOTE_CONTEXT1
kubectl delete virtualgateway egress-vg -n gloo-mesh-gateways --context $REMOTE_CONTEXT1
kubectl delete gatewaylifecyclemanager istio-egressgateway -n gloo-mesh --context $MGMT_CONTEXT
kubectl delete accesspolicy egress-access-policy -n gloo-mesh-gateways --context $REMOTE_CONTEXT1
kubectl delete ns global --context $REMOTE_CONTEXT1