What’s happening

You successfully installed Gloo Mesh Gateway in your cluster that is hosted on Amazon Web Services (AWS). However, the health check for the Elastic Load Balancer (ELB) that is automatically created for you to back the Kubernetes service that exposes the Istio ingress gateway, fails and reports an unhealthy state.

Why it’s happening

When you install Gloo Mesh Gateway, an Istio ingress gateway is automatically deployed for you and exposed with a Kubernetes service of type load balancer. In clusters that are hosted on AWS, an ELB is automatically created to back the ingress gateway service. Gloo Mesh Gateway configures the ingress gateway to listen on HTTPS port 15443. However, when the ELB is created, the first port that is defined in the Kubernetes service manifest is used to perform the ELB health check. This port might be different from the port that Gloo Mesh Gateway configures.

For example, the following Kubernetes service manifest defines multiple ports that the ingress gateway can listens on. Because port 80 is the first port that is defined in this list, the ELB health check is configured for port 80 instead of port 15443.

...
spec:
  clusterIP: 10.100.108.166
  externalTrafficPolicy: Cluster
  ports:
  - name: http2
    nodePort: 31143
    port: 80
    protocol: TCP
    targetPort: 8080
  - name: https
    nodePort: 30131
    port: 443
    protocol: TCP
    targetPort: 8443
  - name: tls
    nodePort: 32287
    port: 15443
    protocol: TCP
    targetPort: 15443
  selector:
    app: istio-ingressgateway
    istio: ingressgateway

How to fix it?

For your ELB health check to pass, you need to configure the load balancer to run the health check on port 15443.

  1. Edit the istio-ingressgateway load balancer service in your cluster.

    kubectl edit svc/istio-ingressgateway -n istio-system

  2. Move the tls port to the top of the list of ports, such as in this example YAML file.

    ...
spec:
  clusterIP: 10.100.108.166
  externalTrafficPolicy: Cluster
  ports:
  - name: tls
    nodePort: 32287
    port: 15443
    protocol: TCP
    targetPort: 15443
  - name: http2
    nodePort: 31143
    port: 80
    protocol: TCP
    targetPort: 8080
  - name: https
    nodePort: 30131
    port: 443
    protocol: TCP
    targetPort: 8443
  selector:
    app: istio-ingressgateway
    istio: ingressgateway

  3. Save your changes.

  4. Wait a few minutes and then verify that the ELB health checks pass successfully.