Waypoint

Learn how L7 load balancing, outlier detection, and failover work for waypoint proxies.

When a waypoint proxy is in the path, L7 load balancing applies. Waypoints use the HBONE protocol to talk directly to the backend or to the next hop. On that leg of the path, traffic is not captured by the ztunnel, so Envoy performs L7 load balancing.

Traffic distribution still applies, such as on the waypoint’s service and on the backend service. When you need finer control than traffic distribution provides, you can apply an Istio DestinationRule, such as for HTTP-based outlier detection or a custom failover order. The rule is enforced at the waypoint and overrides the traffic distribution setting for that host.

Solo Enterprise for agentgateway behaves the same as the Istio waypoint when used as a waypoint proxy.

Cluster scenarios

The following diagrams give examples of how the waypoint handles load balancing, outlier detection, and failover in single-cluster and multicluster setups.

Single-cluster scenario

The request routes from the client app to the backend app.
Both the client and backend services are enrolled in the ambient mesh and run in the same cluster.
The backend app has L7 policies defined that are enforced by a waypoint proxy.
The backend service has multiple replicas distributed across different zones or regions within the cluster.
Optionally, the spec.trafficDistribution or networking.istio.io/traffic-distribution setting is configured on the backend service to control locality-based load balancing.

    graph LR
    Client[Client pod] -->|1. L4| Ztunnel1[Client<br/>ztunnel]
    Ztunnel1 -->|2. HBONE| Waypoint[Waypoint<br/>L7 proxy]
    Waypoint -->|3. Load balancing| Backend1[Backend pod<br/>✓ Healthy]
    Waypoint -->|3. Load balancing| Backend2[Backend pod<br/>✓ Healthy]
    Waypoint -.->|"Outlier detection: ejected"| Backend3[Backend pod<br/>✗ Unhealthy]

    style Waypoint fill:#2068F3,color:#fff

Multicluster scenario

The request routes from the client app to the backend app.
Both the client and backend services are enrolled in the ambient mesh.
The backend app has L7 policies defined that are enforced by a waypoint proxy.
The backend app is exposed as a global service with the solo.io/service-scope=global annotation.
The networking.istio.io/traffic-distribution annotation is set on the global service. Note that other service-scope annotations like solo.io/service-scope=global-only, solo.io/service-scope=segment, solo.io/service-scope=cluster, or solo.io/service-takeover: true, behave in the same way. Only the number of endpoints that can be addressed differs, such as only the endpoints within the same cluster or segment.

    graph LR
    subgraph Cluster1["Cluster 1"]
        Client[Client pod]
        Ztunnel1[Client<br/>ztunnel]
        Waypoint[Waypoint]
        Backend1[Backend pod<br/>✓ Healthy]
        Client -->|1. L4| Ztunnel1
        Ztunnel1 -->|2. HBONE| Waypoint
        Waypoint -->|"3. Local endpoint<br/>(global service)"| Backend1
    end

    subgraph Cluster2["Cluster 2"]
        Backend2[Backend pod<br/>✓ Healthy]
        Backend3[Backend pod<br/>✗ Unhealthy]
    end

    Waypoint -->|"3. Cross-cluster<br/>(global service)"| Backend2
    Waypoint -.->|"Outlier detection: ejected"| Backend3

    style Waypoint fill:#2068F3,color:#fff

Locality determination

Traffic flows through two paths when using waypoints, and locality is determined separately in each path.

Client (ztunnel) → waypoint:

Endpoints are determined by comparing the client’s ztunnel locality with the locality labels on the waypoint.
After locality is determined, the traffic distribution mode on the waypoint applies. For example, the default PreferClose mode selects the waypoint that is in the same network, region, and zone as the client.
This behavior follows the L4 ztunnel failover behavior.

Waypoint → backend:

Endpoints are determined by comparing the waypoint’s locality with the locality of the backend.
After locality is determined, the traffic distribution mode on the backend applies. For example, if the backend has the Any mode, the waypoint round robins between the backend endpoints.
This behavior follows the L7 Envoy failover behavior.

Load balancing

Client (ztunnel) → waypoint:

Traffic distribution is enforced via the traffic distribution setting on the service that exposes the waypoint Gateway. The following precedence order applies:

Service spec.trafficDistribution (highest precedence, Kubernetes-native, supports PreferClose only)
Service networking.istio.io/traffic-distribution annotation (Istio-extended, supports all modes)

If unset, the behavior of the PreferClose mode is used.

Waypoint → backend:

Traffic distribution is enforced via the traffic distribution setting on the global service. The following precedence order applies:

Service spec.trafficDistribution (highest precedence, Kubernetes-native, supports PreferClose only)
Service networking.istio.io/traffic-distribution annotation (Istio-extended, supports all modes)
ServiceEntry networking.istio.io/traffic-distribution annotation (Istio-extended, supports all modes). Use this when you use a ServiceEntry.
UNSPECIFIED_MODE (default, lowest precedence). In this case, the behavior of the PreferNetwork mode is used.

Defaults: If you do not explicitly set traffic distribution modes, PreferClose is used for the waypoint, and PreferNetwork is used for the backend.

To review the available traffic distribution modes and how each mode prioritizes endpoints, see Traffic distribution modes and endpoint priority. Note that these modes apply only if no DestinationRule is in place.

L7 load-based spillover

When traffic distribution creates priority levels for endpoints, Envoy uses locality weighted load balancing to distribute requests across those priorities. Under high load, Envoy might send some traffic to lower-priority endpoints (farther localities) even when higher-priority endpoints (closer localities) are still healthy.

Envoy uses an overprovisioning factor (default 1.4) to determine when to spill over traffic. A priority level is not considered unhealthy until the percentage of healthy endpoints multiplied by the overprovisioning factor drops below 100. With the default factor, a locality is not considered unhealthy until healthy endpoints drop below approximately 72% (100 / 1.4). When endpoints in a preferred locality approach their capacity under heavy load but are still technically healthy, Envoy begins sending some requests to the next priority level to prevent overload.

This spillover behavior is capacity-based, not failure-based. It is distinct from traditional failover, which occurs when endpoints become unavailable or are ejected due to outlier detection. Spillover is a load-balancing mechanism to prevent hot spots and distribute load more evenly across available capacity.

Outlier detection and failover

The traffic distribution settings that are previously described are the recommended method for failover. If these settings cannot meet your failover requirements, you can define an Istio DestinationRule with failover settings in the spec.trafficPolicy.loadBalancer.localityLbSetting section. To detect and handle unhealthy endpoints, you can also define outlier detection conditions in the spec.trafficPolicy.outlierDetection section of a DestinationRule.

The DestinationRule is applied at the waypoint and targets the backend service hostname. In a multicluster setup, use the global service hostname (e.g., productpage.bookinfo.mesh.internal). In a single-cluster setup, use the standard Kubernetes service hostname (e.g., productpage.bookinfo.svc.cluster.local or productpage). If multiple services are served by the waypoint, you can specify the hostname of each service in the DestinationRule to set different behavior for each service.

Note that you cannot define a DestinationRule on the request path from the client to the waypoint. However, the logic in DestinationRule that you create for the backend hosts also applies to the path from the client to the waypoint.

For example configurations, see DestinationRule examples.

Ztunnel kgateway