☰
Navigation :

Block egress traffic with an egress gateway

Use an egress gateway to allow egress traffic to external endpoints for only certain services in the mesh. All other services in the mesh are blocked from sending requests to the external endpoint. You can also use the egress traffic to perform TLS origination and secure the connection to the external endpoint. For more information, see Perform TLS origination with egress gateways.

Blocking egress traffic with an egress gateway and access policies works only if clients send traffic to the egress gateway through the client's sidecar proxy. If the sidecar proxy is bypassed, requests to an external endpoint might still be allowed, which can be a risk for malicious attacks. To make sure that no egress traffic is allowed, even if the client's sidecar proxy is bypassed, create a network policy that blocks all egress traffic in addition to setting up an egress gateway. You can further reduce the surface for malicious attacks by deploying the egress gateway to a dedicated node. Also note that you cannot apply other policies, such as outlier detection or active healthchecks, to the egress gateway.

Before you begin

  1. Set the names of your clusters from your infrastructure provider. If your clusters have different names, specify those names instead.

    export REMOTE_CLUSTER1=<cluster1>
export REMOTE_CLUSTER2=<cluster2>
...
  2. Save the kubeconfig contexts for your clusters. Run kubectl config get-contexts, look for your cluster in the CLUSTER column, and get the context name in the NAME column. Note: Do not use context names with underscores. The context name is used as a SAN specification in the generated certificate that connects workload clusters to the management cluster, and underscores in SAN are not FQDN compliant. You can rename a context by running kubectl config rename-context "<oldcontext>" <newcontext>. 
    export MGMT_CONTEXT=<management-cluster-context>
export REMOTE_CONTEXT1=<remote-cluster1-context>
export REMOTE_CONTEXT2=<remote-cluster2-context>
...

  3. Decide on the outbound traffic policy for your service mesh. If you followed the steps to install Istio with the Istio lifecycle manager, the outbound traffic policy for your service mesh is set to ALLOW_ANY by default and allows your services in the mesh to reach any external endpoint. You can change this setting and instead block all egress traffic to hosts that are not part of your service mesh by changing the outbound traffic policy to REGISTRY_ONLY as shown in the following steps. Note that if you manually installed Istio, follow the steps in Upgrade Istio to update that setting.

    1. Open the gm-istiod-values.yaml file that you used to create the istiod control plane.
    2. Change the outboundTrafficPolicy field to REGISTRY_ONLY.
    3. Apply your changes by using the following command. 
      kubectl apply -f gm-istiod-values.yaml --context $MGMT_CONTEXT

  4. Choose the Istio version you want to use for the egress gateway.

    To use the default supported version of Solo Istio, set the REPO environment variable to auto. This setting automatically uses the default supported Solo Istio version for the image tag (such as 1.18.2-solo) and the revision (such as 1-18-2).

    export REPO=auto

    Note that in future upgrades, the version set by the auto setting does not change unless you manually specify different values for the image tag, revision, and image repository.

    Save the Istio version information as environment variables.

    • For REPO, use a Solo Istio repo key that you can get by logging in to the Support Center and reviewing the Istio images built by Solo.io support article. For more information, see Get the Solo Istio version that you want to use.
    • For ISTIO_IMAGE, save the version that you downloaded, such as 1.18.2, and append the solo tag, which is required to use many enterprise features. You can optionally append other Solo Istio tags, as described in About Solo Istio. If you downloaded a different version than the following, make sure to specify that version instead.
    • For REVISION, take the Istio major and minor version numbers and replace the period with a hyphen, such as 1-18-2.
    export REPO=<repo-key>
export ISTIO_IMAGE=1.18.2-solo
export REVISION=1-18-2

    Istio versions 1.17 and later do not support the Gloo legacy metrics pipeline. If you run the legacy metrics pipeline, before you upgrade or install Istio with version 1.17, be sure that you set up the Gloo OpenTelemetry (OTel) pipeline instead in your new or existing Gloo Mesh installation.

Block egress traffic with an egress gateway

  1. Prepare a GatewayLifecycleManager to deploy an egress gateway.

    1. Download the gm-egress-gateway.yaml example file.

      curl -0L https://raw.githubusercontent.com/solo-io/gloo-mesh-use-cases/main/gloo-mesh/istio-install/gm-managed/gm-egress-gateway.yaml > gm-egress-gateway.yaml

    2. Update the example file with the environment variables that you previously set, and optionally further edit the file to provide your own details. Save the updated file as gm-egress-gateway-values.yaml. For more information, see the API reference.

      • Tip: Instead of updating the file manually, try running a terminal command to substitute values, such as the following command. 
        envsubst < gm-egress-gateway.yaml > gm-egress-gateway-values.yaml
open gm-egress-gateway-values.yaml

    3. Apply the GatewayLifecycleManager resource to your management cluster.

      kubectl apply -f gm-egress-gateway-values.yaml --context $MGMT_CONTEXT

    4. Verify that the egress gateway is deployed in your workload clusters.

      kubectl get pods -n gloo-mesh-gateways --context $REMOTE_CONTEXT1
kubectl get pods -n gloo-mesh-gateways --context $REMOTE_CONTEXT2

  2. Log in to the reviews app and verify that you cannot reach www.google.com.

    kubectl --context ${REMOTE_CONTEXT1} -n bookinfo debug -i pods/$(kubectl get pod --context ${REMOTE_CONTEXT1} -l app=reviews -A -o jsonpath='{.items[0].metadata.name}') --image=curlimages/curl -- curl -vik www.google.com

    Example output:

    * Mark bundle as not supporting multiuse
< HTTP/1.1 502 Bad Gateway
HTTP/1.1 502 Bad Gateway
< date: Thu, 11 May 2023 17:07:43 GMT
date: Thu, 11 May 2023 17:07:43 GMT
< server: envoy
server: envoy
< content-length: 0
content-length: 0

  3. Create the global namespace in one of your workload clusters.

    kubectl create namespace global --context $REMOTE_CONTEXT1

  4. Create a virtual gateway resource to configure your egress gateway going forward.

    kubectl apply --context $REMOTE_CONTEXT1 -f- <<EOF
apiVersion: networking.gloo.solo.io/v2
kind: VirtualGateway
metadata:  
  annotations:
    cluster.solo.io/cluster: ""
  labels:
    app: istio-egressgateway
    istio: egressgateway
  name: egress-vg
  namespace: gloo-mesh-gateways
spec:
  listeners:
  - appProtocol: HTTPS
    exposedExternalServices:
    - host: www.google.com
    port:
      number: 443
    tls:
      mode: ISTIO_MUTUAL
  workloads:
  - selector:
      labels:
        app: istio-egressgateway
        istio: egressgateway
EOF
    Setting Description
    spec.listeners.exposedExternalServices.host Select the external services that you want to serve on this egress gateway. To select the external services, you define the hostname that must be configured in the external service.
    spec.listeners.port Enter the port number that services in the mesh use in combination with the host to send a request to the egress gateway.
    spec.listeners.tls Select the TLS mode that you want to use to connect to the egress gateway from within the mesh. In this example, requests for the www.google.com host are rerouted to the egress gateway first by using mutual TLS. The incoming TLS connection is terminated at the egress gateway.

  5. Create an external service for www.google.com.

    kubectl apply --context $REMOTE_CONTEXT1 -f- <<EOF
apiVersion: networking.gloo.solo.io/v2
kind: ExternalService
metadata:
  name: google
  namespace: global
spec:
  hosts:
  - "www.google.com"
  ports:
    - clientsideTls: {}
      egressGatewayRoutes: 
        portMatch: 80
        virtualGatewayRefs:
          - cluster: $REMOTE_CLUSTER1
            name: egress-vg
            namespace: gloo-mesh-gateways
          - cluster: $REMOTE_CLUSTER2
            name: egress-vg
            namespace: gloo-mesh-gateways
      name: https
      number: 443
      protocol: HTTPS
EOF
    Setting Description
    spec.hosts Enter the external hostname that you want to allow access to for services in the mesh.
    spec.ports.clientsideTls Allow TLS origination to the www.google.com endpoint.
    spec.ports.egressGatewayRoutes.virtualGatewayRefs Enter the details for your egress gateway. Make sure to enter the egress gateway details for both of your workload clusters.
    spec.ports.egressGatewayRoutes.portMatch Select the port that clients use when sending a request to the external endpoint. Note that this port does not represent the port that the client's sidecar proxy uses to connect to the egress gateway. Instead, make sure to enter the source port that is used in the original client request.
    spec.ports.name Enter the port number that you want to use to connect to the external endpoint. When a client request is sent to the gateway, the client's sidecar proxy intercepts this traffic and securely connects to the egress gateway via mutual TLS. The egress gateway then terminates the incoming TLS connection. Because this example defines an HTTPS port, a new TLS connection is originated at the egress gateway to securely connect to the www.google.com external endpoint.

  6. Verify that you can access www.google.com from the reviews app.

    kubectl --context ${REMOTE_CONTEXT1} -n bookinfo debug -i pods/$(kubectl get pod --context ${REMOTE_CONTEXT1} -l app=reviews -A -o jsonpath='{.items[0].metadata.name}') --image=curlimages/curl -- curl -vik www.google.com

    Example output:

    * Connected to www.google.com (240.240.59.19) port 80 (#0)
> GET / HTTP/1.1
> Host: www.google.com
> User-Agent: curl/8.1.1-DEV
> Accept: */*
> 
< HTTP/1.1 200 OK
< date: Wed, 24 May 2023 20:23:50 GMT
< expires: -1
< cache-control: private, max-age=0
< content-type: text/html; charset=ISO-8859-1
< content-security-policy-report-only: object-src 'none';base-uri 'self';script-src 'nonce-k4p_jp_okP7dor-UgJJt5Q' 'strict-dynamic' 'report-sample' 'unsafe-eval' 'unsafe-inline' https: http:;report-uri https://csp.withgoogle.com/csp/gws/other-hp
< p3p: CP="This is not a P3P policy! See g.co/p3phelp for more info."
< server: envoy
HTTP/1.1 200 OK
...

  7. Verify that traffic was routed through the egress gateway by looking at the egress gateway logs.

    kubectl logs $(kubectl get pod --context ${REMOTE_CONTEXT1} -l app=istio-egressgateway -A -o jsonpath='{.items[0].metadata.name}') -n gloo-mesh-gateways --context $REMOTE_CONTEXT1

    Example output:

    {"upstream_service_time":"55","response_code":200,"response_flags":"-","authority":"www.google.com","bytes_received":0,"duration":55,"method":"GET","request_id":"f0d0412a-a393-9190-a49b-23baaadf7d22","route_name":null,"requested_server_name":"www.google.com","upstream_host":"142.251.161.105:443","downstream_remote_address":"10.84.1.21:52038","downstream_local_address":"10.84.0.24:8443","start_time":"2023-05-24T20:16:57.753Z","bytes_sent":17238,"response_code_details":"via_upstream","upstream_cluster":"outbound|443||www.google.com","upstream_transport_failure_reason":null,"path":"/","x_forwarded_for":"10.84.1.21","user_agent":"curl/8.1.1-DEV","upstream_local_address":"10.84.0.24:57268","connection_termination_details":null,"protocol":"HTTP/1.1"}

  8. Get the virtual service that was created for your external service and make sure that it lists port 443 for the www.google.com destination. This setting ensures the egress gateway originates a new TLS connection before connecting to www.google.com.

    kubectl get virtualservices -n gloo-mesh-gateways --context $REMOTE_CONTEXT1 | grep externalservice-google

    kubectl get virtualservice <virtual-service-name> -n gloo-mesh-gateways --context $REMOTE_CONTEXT1 -o yaml

    Example output:

    ...
spec:
  exportTo:
  - bookinfo
  - default
  - global
  - gloo-mesh
  - gloo-mesh-addons
  - gloo-mesh-gateways
  - gm-iop-1-18-2
  - helloworld
  - httpbin
  - istio-system
  gateways:
  - virtualgateway-egress-vg-gloo-m-ef7dba3d044465c79ef179255be34f5
  - mesh
  hosts:
  - www.google.com
  http:
  - match:
    - gateways:
      - virtualgateway-egress-vg-gloo-m-ef7dba3d044465c79ef179255be34f5
      port: 8443
    route:
    - destination:
        host: www.google.com
        port:
          number: 443
...

  9. Create an access policy for your external service to allow only the product page app to access the www.google.com external endpoint.

    kubectl apply --context $REMOTE_CONTEXT1 -f- <<EOF
apiVersion: security.policy.gloo.solo.io/v2
kind: AccessPolicy
metadata:
  annotations:
    cluster.solo.io/cluster: ""
  name: egress-access-policy
  namespace: gloo-mesh-gateways
spec:
  applyToDestinations:
  - kind: EXTERNAL_SERVICE
    port:
      number: 443
    selector:
      cluster: $REMOTE_CLUSTER1
      name: google
      namespace: global
  config:
    authz:
      allowedClients:
      - serviceAccountSelector:
          cluster: $REMOTE_CLUSTER1
          labels:
            account: productpage
          namespace: bookinfo
EOF

  10. Send a request to www.google.com from the reviews app and verify that this request is denied with a 403 HTTP response code.

    kubectl --context ${REMOTE_CONTEXT1} -n bookinfo debug -i pods/$(kubectl get pod --context ${REMOTE_CONTEXT1} -l app=reviews -A -o jsonpath='{.items[0].metadata.name}') --image=curlimages/curl -- curl -vik www.google.com

    Example output:

    * Connected to www.google.com (240.240.59.19) port 80 (#0)
> GET / HTTP/1.1
> Host: www.google.com
> User-Agent: curl/8.1.1-DEV
> Accept: */*
> 
< HTTP/1.1 403 Forbidden
< content-length: 19
< content-type: text/plain
< date: Wed, 24 May 2023 20:58:55 GMT
< server: envoy
< x-envoy-upstream-service-time: 13
< 
{ [19 bytes data]
HTTP/1.1 403 Forbidden

* Connection #0 to host www.google.com left intact
content-length: 19
content-type: text/plain
date: Wed, 24 May 2023 20:58:55 GMT
server: envoy
x-envoy-upstream-service-time: 13

RBAC: access denied%

  11. Send another request to www.google.com from the product page app and verify that this request is accepted.

    kubectl --context ${REMOTE_CONTEXT1} -n bookinfo debug -i pods/$(kubectl get pod --context ${REMOTE_CONTEXT1} -l app=productpage -A -o jsonpath='{.items[0].metadata.name}') --image=curlimages/curl -- curl -vik www.google.com

    Example output:

    * Connected to www.google.com (240.240.59.19) port 80 (#0)
> GET / HTTP/1.1
> Host: www.google.com
> User-Agent: curl/8.1.1-DEV
> Accept: */*
> 
< HTTP/1.1 200 OK
< date: Wed, 24 May 2023 21:01:44 GMT
< expires: -1
< cache-control: private, max-age=0
< content-type: text/html; charset=ISO-8859-1
HTTP/1.1 200 OK

  12. Optional: Clean up the resources that you created as part of this guide.

    kubectl delete externalservice google -n global --context $REMOTE_CONTEXT1
kubectl delete virtualgateway egress-vg -n gloo-mesh-gateways --context $REMOTE_CONTEXT1
kubectl delete gatewaylifecyclemanager istio-egressgateway -n gloo-mesh --context $MGMT_CONTEXT
kubectl delete accesspolicy egress-access-policy -n gloo-mesh-gateways --context $REMOTE_CONTEXT1
kubectl delete ns global --context $REMOTE_CONTEXT1