Istio ingress gateway

Review load balancing, failover, and outlier detection for external traffic entering the mesh through an Istio ingress gateway.

The Istio ingress gateway uses the HBONE protocol to talk directly to the backend or to the next hop, such as to a waypoint. On that leg of the path, traffic is not captured by the ztunnel, so Envoy performs L7 load balancing.

Traffic distribution still applies, such as on the backend service. When you need finer control than traffic distribution provides, you can apply an Istio DestinationRule, such as for HTTP-based outlier detection or a custom failover order. The rule is enforced at the ingress gateway and overrides the traffic distribution setting for that host.

Cluster scenarios

The following diagrams give examples of how the Istio ingress gateway handles load balancing, outlier detection, and failover in single-cluster and multicluster setups.

Single-cluster scenario

The request routes from an external client to a backend service in the mesh.
The backend service is enrolled in the ambient mesh and runs in a single cluster.
The backend service has multiple replicas distributed across different zones or regions within the cluster.
Optionally, the spec.trafficDistribution or networking.istio.io/traffic-distribution setting is configured on the backend service to control locality-based load balancing.

    graph LR
    External[External<br/>Client] -->|1. HTTPS| Ingress[Istio Ingress<br/>Gateway<br/>L7 proxy]
    Ingress -->|2. HBONE| Waypoint[Waypoint]
    Ingress -->|2. Direct if no waypoint| Backend1[Backend pod<br/>✓ Healthy]
    Waypoint -->|3. Load balancing| Backend1
    Waypoint -->|3. Load balancing| Backend2[Backend pod<br/>✓ Healthy]
    Waypoint -.->|"Outlier detection: ejected"| Backend3[Backend pod<br/>✗ Unhealthy]

    style Ingress fill:#2068F3,color:#fff
    style Waypoint fill:#2068F3,color:#fff

Multicluster scenario

The request routes from an external client to a backend service in the mesh.
The backend app is exposed as a global service with the solo.io/service-scope=global annotation.
The networking.istio.io/traffic-distribution annotation is set on the global service. Note that other service-scope annotations like solo.io/service-scope=global-only, solo.io/service-scope=segment, solo.io/service-scope=cluster, or solo.io/service-takeover: true, behave in the same way. Only the number of endpoints that can be addressed differs, such as only the endpoints within the same cluster or segment.

    graph LR
    External[External<br/>Client]

    subgraph Cluster1["Cluster 1"]
        Ingress[Istio ingress<br/>gateway]
        Waypoint[Waypoint]
        Backend1[Backend pod<br/>✓ Healthy]
        Ingress -->|2. HBONE| Waypoint
        Ingress -->|"2. If no waypoint, load balance to global service"| Backend1
        Waypoint -->|"3. Local endpoint<br/>(global service)"| Backend1
    end

    subgraph Cluster2["Cluster 2"]
        Backend2[Backend pod<br/>✓ Healthy]
        Backend3[Backend pod<br/>✗ Unhealthy]
    end

    External -->|1. HTTPS| Ingress
    Waypoint -->|"3. Cross-cluster<br/>(global service)"| Backend2
    Waypoint -.->|"Outlier detection: ejected"| Backend3

    style Ingress fill:#2068F3,color:#fff
    style Waypoint fill:#2068F3,color:#fff

Routing variations:

Client → ingress gateway → backend
Client → ingress gateway → waypoint → backend. In this case, the backend has the istio.io/ingress-use-waypoint=true service label to ensure that the ingress gateway routes the incoming request to the waypoint proxy.

Locality determination

Traffic flows through multiple paths when using an Istio ingress gateway, and locality is determined separately in each path.

Client → ingress gateway

Load balancer-specific settings might enforce locality rules, but no locality setting is determined by or for Istio components.

Ingress gateway → backend

Endpoints are determined by comparing the ingress gateway’s locality with the locality of the backend.
After locality is determined, the traffic distribution mode on the backend applies. For example, if the backend has the Any mode, the ingress gateway round robins between the backend endpoints.

Ingress gateway → waypoint (if applicable)

Endpoints are determined by comparing the ingress gateway’s locality with the locality of the waypoint.
After locality is determined, the traffic distribution mode on the waypoint applies. For example, the default PreferClose mode selects the waypoint that is in the same network, region, and zone as the ingress gateway.
The Istio ingress gateway uses the HBONE protocol to communicate with the waypoint directly. Because of this, no ztunnel is involved, and Envoy L7 load balancing rules apply. For information about how Envoy handles traffic under high load, see L7 load-based spillover.

Waypoint → backend (if applicable)

Endpoints are determined by comparing the waypoint’s locality with the locality of the backend.
After locality is determined, the traffic distribution mode on the backend applies. For example, if the backend has the Any mode, the waypoint round robins between the backend endpoints.

Load balancing

Ingress gateway → backend:

Traffic distribution is determined by the traffic distribution setting on the global service. The following precedence order applies:

Service spec.trafficDistribution (highest precedence, Kubernetes-native, supports PreferClose only)
Service networking.istio.io/traffic-distribution annotation (Istio-extended, supports all modes)
ServiceEntry networking.istio.io/traffic-distribution annotation (Istio-extended, supports all modes). Use this when you use a ServiceEntry.
UNSPECIFIED_MODE (default, lowest precedence). In this case, the behavior of the PreferNetwork mode is used.

Ingress gateway → waypoint:

Traffic distribution is determined by the traffic distribution on the waypoint. The following precedence order applies:

Service spec.trafficDistribution (highest precedence, Kubernetes-native, supports PreferClose only)
Service networking.istio.io/traffic-distribution annotation (Istio-extended, supports all modes)

If unset, the behavior of the PreferClose mode is used.

Waypoint → backend:

Traffic distribution is determined by the traffic distribution setting on the global service. The following precedence order applies:

Service spec.trafficDistribution (highest precedence, Kubernetes-native, supports PreferClose only)
Service networking.istio.io/traffic-distribution annotation (Istio-extended, supports all modes)
ServiceEntry networking.istio.io/traffic-distribution annotation (Istio-extended, supports all modes). Use this when you use a ServiceEntry.
UNSPECIFIED_MODE (default, lowest precedence). In this case, the behavior of the PreferNetwork mode is used.

Defaults: If you do not explicitly set traffic distribution modes, PreferClose is used for the waypoint, and PreferNetwork is used for the backend.

To review the available traffic distribution modes and how each mode prioritizes endpoints, see Traffic distribution modes and endpoint priority. Note that these modes apply only if no DestinationRule is in place.

L7 load-based spillover

Istio ingress gateways use the same spillover behavior as waypoints. For details, see L7 load-based spillover in the Waypoints section.

Outlier detection and failover

The traffic distribution settings that are previously described are the recommended method for failover. If these settings cannot meet your failover requirements, you can define an Istio DestinationRule with failover settings in the spec.trafficPolicy.loadBalancer.localityLbSetting section. To detect and handle unhealthy endpoints, you can also define outlier detection conditions in the spec.trafficPolicy.outlierDetection section of a DestinationRule.

The DestinationRule targets the backend service hostname, or a waypoint can enforce it for the service. In a multicluster setup, use the global service hostname (e.g., productpage.bookinfo.mesh.internal). In a single-cluster setup, use the standard Kubernetes service hostname (e.g., productpage.bookinfo.svc.cluster.local or productpage). If multiple services are served by a waypoint, you can specify the hostname of each service in the DestinationRule to set different behavior for each service.

Note that you cannot define a DestinationRule on the request path from the ingress gateway to a waypoint. However, the DestinationRule logic that you create for the backend hosts also applies to the ingress-to-waypoint path.

The endpoints in the Envoy cluster depend on the istio.io/ingress-use-waypoint label:

When set to true: Waypoints serve as the endpoints.
When set to false or when no waypoint exists: Backend pods serve as the endpoints.

In either case, the DestinationRule logic applies to the endpoints in the same manner.

For example configurations, see DestinationRule examples.

kgateway Sidecar