Sidecar

Review load balancing, failover, and outlier detection when a client has a sidecar proxy.

Cluster scenarios

The following diagrams give examples of how sidecar proxies handle load balancing, outlier detection, and failover in single-cluster and multicluster setups.

Single-cluster scenario

• The client app in the ambient mesh also has a sidecar proxy.
• The client app routes to a backend app in the ambient mesh that uses a ztunnel.
• Both apps run in the same cluster.
• The backend service has multiple replicas distributed across different zones or regions within the cluster.
• Optionally, the spec.trafficDistribution or networking.istio.io/traffic-distribution setting is configured on the backend service to control locality-based load balancing.

    graph LR
    Sidecar[Sidecar<br/>Envoy proxy] -->|1. HBONE| Waypoint[Waypoint]
    Sidecar -->|1. If no waypoint, direct load balancing| Backend1[Backend pod<br/>✓ Healthy]
    Waypoint -->|2. Load balancing| Backend1
    Waypoint -->|2. Load balancing| Backend2[Backend pod<br/>✓ Healthy]
    Waypoint -.->|"Outlier detection: ejected"| Backend3[Backend pod<br/>✗ Unhealthy]

    style Sidecar fill:#2068F3,color:#fff
    style Waypoint fill:#2068F3,color:#fff
  

Multicluster scenario

• The client app in the ambient mesh also has a sidecar proxy.
• The client app routes to a backend app in the ambient mesh that uses a ztunnel.
• The backend app is exposed as a global service with the solo.io/service-scope=global annotation.
• The networking.istio.io/traffic-distribution annotation is set on the global service. Note that other service-scope annotations like solo.io/service-scope=global-only, solo.io/service-scope=segment, solo.io/service-scope=cluster, or solo.io/service-takeover: true, behave in the same way. Only the number of endpoints that can be addressed differs, such as only the endpoints within the same cluster or segment.

    graph LR
    subgraph Cluster1["Cluster 1"]
        Sidecar[Sidecar<br/>Envoy proxy]
        Waypoint[Waypoint]
        Backend1[Backend pod<br/>✓ Healthy]
        Sidecar -->|1. HBONE| Waypoint
        Sidecar -->|1. If no waypoint, load balance to global service| Backend1
        Waypoint -->|"2. Local endpoint<br/>(global service)"| Backend1
    end

    subgraph Cluster2["Cluster 2"]
        Backend2[Backend pod<br/>✓ Healthy]
        Backend3[Backend pod<br/>✗ Unhealthy]
    end

    Waypoint -->|"2. Cross-cluster<br/>(global service)"| Backend2
    Waypoint -.->|"Outlier detection: ejected"| Backend3

    style Sidecar fill:#2068F3,color:#fff
    style Waypoint fill:#2068F3,color:#fff
  

Routing variations:

• Client (sidecar) → backend
• Client (sidecar) → waypoint → backend

Locality determination

Locality is determined differently based on whether a waypoint serves the backend service or not.

Sidecar → backend

• Endpoints are determined by comparing the sidecar’s locality with the locality of the backend’s ztunnel.
• After locality is determined, the traffic distribution mode on the backend applies. For example, if the backend has the Any mode, the sidecar round robins between the backend endpoints.

Sidecar → waypoint (if applicable)

• Endpoints are determined by comparing the sidecar’s locality with the locality of the waypoint.
• After locality is determined, the traffic distribution mode on the waypoint applies. For example, the default PreferClose mode selects the waypoint that is in the same network, region, and zone as the sidecar.
• The sidecar proxy uses the HBONE protocol to communicate with the waypoint directly. Because of this, no ztunnel is involved, and Envoy L7 load balancing rules apply. Envoy can route traffic to lower priority endpoints if higher priority endpoints do not meet the load balancing threshold.

Waypoint → backend (if applicable)

• Endpoints are determined by comparing the waypoint’s locality with the locality of the backend.
• After locality is determined, the traffic distribution mode on the backend applies. For example, if the backend has the Any mode, the waypoint round robins between the backend endpoints.

Load balancing

Ingress gateway → backend:

Traffic distribution is determined by the traffic distribution setting on the global service. The following precedence order applies:

1. Service spec.trafficDistribution (highest precedence, Kubernetes-native, supports PreferClose only)
2. Service networking.istio.io/traffic-distribution annotation (Istio-extended, supports all modes)
3. ServiceEntry networking.istio.io/traffic-distribution annotation (Istio-extended, supports all modes). Use this when you use a ServiceEntry.
4. UNSPECIFIED_MODE (default, lowest precedence). In this case, the behavior of the PreferNetwork mode is used.

Ingress gateway → waypoint:

Traffic distribution is determined by the traffic distribution on the waypoint. The following precedence order applies:

1. Service spec.trafficDistribution (highest precedence, Kubernetes-native, supports PreferClose only)
2. Service networking.istio.io/traffic-distribution annotation (Istio-extended, supports all modes)

If unset, the behavior of the PreferClose mode is used.

Waypoint → backend:

Traffic distribution is determined by the traffic distribution setting on the global service. The following precedence order applies:

1. Service spec.trafficDistribution (highest precedence, Kubernetes-native, supports PreferClose only)
2. Service networking.istio.io/traffic-distribution annotation (Istio-extended, supports all modes)
3. ServiceEntry networking.istio.io/traffic-distribution annotation (Istio-extended, supports all modes). Use this when you use a ServiceEntry.
4. UNSPECIFIED_MODE (default, lowest precedence). In this case, the behavior of the PreferNetwork mode is used.

Defaults: If you do not explicitly set traffic distribution modes, PreferClose is used for the waypoint, and PreferNetwork is used for the backend.

To review the available traffic distribution modes and how each mode prioritizes endpoints, see Traffic distribution modes and endpoint priority. Note that these modes apply only if no DestinationRule is in place.

Outlier detection and failover

The traffic distribution settings that are previously described are the recommended method for failover. If these settings cannot meet your failover requirements, you can define an Istio DestinationRule with failover settings in the spec.trafficPolicy.loadBalancer.localityLbSetting section. To detect and handle unhealthy endpoints, you can also define outlier detection conditions in the spec.trafficPolicy.outlierDetection section of a DestinationRule.

The DestinationRule targets the backend service hostname, or a waypoint can enforce it for the service. In a multicluster setup, use the global service hostname (e.g., productpage.bookinfo.mesh.internal). In a single-cluster setup, use the standard Kubernetes service hostname (e.g., productpage.bookinfo.svc.cluster.local or productpage). If multiple services are served by a waypoint, you can specify the hostname of each service in the DestinationRule to set different behavior for each service.

For example configurations, see DestinationRule examples.