Waypoints (L7)

Set up L7 load balancing and failover with waypoint proxies, including custom DestinationRules for HTTP-aware behavior.

About this guide

Waypoint proxies enable Layer 7 (L7) policies in your ambient mesh, including Istio DestinationRules for HTTP-based outlier detection and custom failover priority. This guide shows you how to deploy a waypoint proxy, observe default L7 load balancing behavior, and configure a DestinationRule to customize failover based on HTTP health. For conceptual information about how L7 load balancing works with waypoints, see the load balancing and failover overview.

Before you begin

Set up an ambient mesh in one cluster by using the Gloo Operator or Helm.

Step 1: Deploy sample apps with a waypoint

Deploy the httpbin sample app and create a waypoint proxy for L7 policy enforcement.

Traffic flow: The following diagram shows how traffic flows when a waypoint proxy is configured. The client’s ztunnel routes to the waypoint at L4, and the waypoint enforces L7 policies before routing to backend endpoints.

    graph LR
    Client[client-in-ambient] -->|Request| Ztunnel1[Client ztunnel<br/>L4]
    Ztunnel1 -->|HBONE| Waypoint[Waypoint proxy<br/>L7 policy enforcement]
    Waypoint -->|L7 load balancing| Backend1[in-ambient pod 1]
    Waypoint -->|L7 load balancing| Backend2[in-ambient pod 2]
    Waypoint -->|L7 load balancing| Backend3[in-ambient pod 3]

    style Waypoint fill:#2068F3,color:#fff

Comparison with L4 flow: Without a waypoint, traffic flows directly from the client’s ztunnel to the backend endpoints using L4 load balancing. With a waypoint, the traffic takes an additional hop through the waypoint proxy, which enables L7 policies such as DestinationRules, outlier detection, and HTTP-aware failover.

Deploy the in-ambient httpbin sample app. This manifest creates the httpbin namespace with an in-ambient backend service. The app is already labeled for inclusion in the ambient mesh with istio.io/dataplane-mode: ambient.
```
kubectl apply -f https://raw.githubusercontent.com/solo-io/doc-examples/main/istio/sample-apps/in-ambient.yaml
```

Deploy the client-in-ambient app in the same namespace. This app is also already labeled for the ambient mesh.

kubectl apply -f https://raw.githubusercontent.com/solo-io/doc-examples/main/istio/sample-apps/client-in-ambient.yaml

Scale the in-ambient deployment to 3 replicas so that you can observe load balancing across multiple endpoints.
```
kubectl scale deployment in-ambient -n httpbin --replicas=3
```

Verify that all pods are running.

kubectl get pods -n httpbin

Example output, in which in-ambient runs as 3 replicas:

NAME                                 READY   STATUS    RESTARTS   AGE
client-in-ambient-6b5c96c4f8-x2j9k   1/1     Running   0          30s
in-ambient-7d8f9b6c54-abc12          1/1     Running   0          45s
in-ambient-7d8f9b6c54-def34          1/1     Running   0          20s
in-ambient-7d8f9b6c54-ghi56          1/1     Running   0          20s

Create a waypoint Gateway in the httpbin namespace. The waypoint enforces L7 policies for services in the namespace.

kubectl apply -f- <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: httpbin-waypoint
  namespace: httpbin
spec:
  gatewayClassName: istio-waypoint
  listeners:
  - name: mesh
    port: 15008
    protocol: HBONE
    allowedRoutes:
      namespaces:
        from: Same
EOF

Label the namespace to use the waypoint for all services.

kubectl label namespace httpbin istio.io/use-waypoint=httpbin-waypoint

Wait for the waypoint to be fully deployed.

kubectl -n httpbin rollout status deployment/httpbin-waypoint

Example output:

deployment "httpbin-waypoint" successfully rolled out

Verify that the waypoint pod is running.

kubectl get pods -n httpbin | grep waypoint

Example output:

NAME                                 READY   STATUS    RESTARTS   AGE
httpbin-waypoint-5cb48fc696-mq99j    1/1     Running   0          2m

Step 2: Observe default L7 load balancing

Send HTTP requests through the waypoint to observe the default load balancing behavior. Because you labeled the httpbin namespace with istio.io/use-waypoint=httpbin-waypoint in the previous step, all traffic to services in that namespace automatically routes through the waypoint - even without additional policies applied. Without a DestinationRule configured, the waypoint uses simple round-robin load balancing across all available endpoints.

Send multiple curl requests from the client to the in-ambient service. The /hostname endpoint returns the pod hostname of the replica that handled the request. Although the curl command is the same as in the L4 example, the traffic now flows through the waypoint proxy for L7 processing before reaching the backend pods.
```
kubectl exec -n httpbin deploy/client-in-ambient -- sh -c "
for i in \$(seq 1 12); do
  curl -s http://in-ambient:8000/hostname
done"
```
Example output:
```
in-ambient-7d8f9b6c54-abc12
in-ambient-7d8f9b6c54-def34
in-ambient-7d8f9b6c54-ghi56
in-ambient-7d8f9b6c54-abc12
...
```

Review the waypoint logs to see HTTP requests being processed at L7.

kubectl logs -n httpbin deploy/httpbin-waypoint | tail -20

Example output showing L7 HTTP request handling:

[2025-03-06T17:15:23.456Z] "GET /hostname HTTP/1.1" 200 - via_upstream - "-" 0 32 5 4 "-"
"curl/7.88.1" "abc123-def456" "in-ambient.httpbin.svc.cluster.local:8000"
"10.10.0.15:8080" inbound-vip|8000|http|in-ambient.httpbin.svc.cluster.local
10.10.0.14:45678 10.96.45.123:8000 10.10.0.14:45678 - default

The waypoint logs show:

HTTP method and path (GET /hostname)
HTTP response code (200)
Request headers and timing information

Compare with ztunnel logs to see how traffic flows differently than the L4 example.

kubectl logs -n istio-system -l app=ztunnel --tail=20 | grep "in-ambient"

Example output showing ztunnel routing to the waypoint:

2025-03-06T17:15:23.450Z  info  access  connection complete  src.addr=10.10.0.14:40292
src.workload="client-in-ambient-6b5c96c4f8-x2j9k" src.namespace="httpbin"
dst.addr=10.10.0.20:15008 dst.service="httpbin-waypoint.httpbin.svc.cluster.local"
dst.workload="httpbin-waypoint-7b8c9d5e6f-abc12" direction="outbound"
bytes_sent=156 bytes_recv=89 duration="8ms"

Unlike the L4 example where ztunnel routes directly to backend pods, here ztunnel routes to the waypoint (httpbin-waypoint). The waypoint then performs L7 load balancing to the backend pods.

Step 3: Apply a DestinationRule for custom failover

Create an Istio DestinationRule with outlier detection policy to customize HTTP-aware failover behavior. The DestinationRule is enforced at the waypoint.

How a DestinationRule affects traffic: The following diagram shows how the waypoint enforces the DestinationRule. The waypoint performs HTTP-aware outlier detection and can quickly eject unhealthy endpoints based on HTTP response codes (5xx errors), providing faster and more accurate failover than L4 TCP-based health checks.

    graph LR
    Client[client-in-ambient] -->|Request| Ztunnel1[Client ztunnel]
    Ztunnel1 -->|HBONE| Waypoint[Waypoint proxy<br/>Enforces DestinationRule]
    Waypoint -->|HTTP health check| Backend1[in-ambient pod 1<br/>✓ Healthy]
    Waypoint -->|HTTP health check| Backend2[in-ambient pod 2<br/>✓ Healthy]
    Waypoint -->|HTTP health check| Backend3[in-ambient pod 3<br/>✓ Healthy]
    Waypoint -.->|Ejects after 3x 5xx| BackendX[Unhealthy Pod<br/>Ejected for 30s]

    style Waypoint fill:#2068F3,color:#fff

Create a DestinationRule for the in-ambient service to configure the following load balancing and failover policies:

Load balancing: Load balance requests evenly across endpoints by using a round-robin distribution pattern.
Outlier detection: Eject endpoints after 3 consecutive 5xx errors, with a 30-second base ejection time.

kubectl apply -f- <<EOF
apiVersion: networking.istio.io/v1
kind: DestinationRule
metadata:
  name: in-ambient-failover
  namespace: httpbin
spec:
  host: in-ambient
  trafficPolicy:
    loadBalancer:
      simple: ROUND_ROBIN
    outlierDetection:
      consecutive5xxErrors: 3
      interval: 10s
      baseEjectionTime: 30s
      maxEjectionPercent: 50
EOF

Verify that the DestinationRule is applied.

kubectl get destinationrule -n httpbin

Example output:

NAME                  HOST         AGE
in-ambient-failover   in-ambient   30s

When a DestinationRule exists for a host, it overrides the traffic distribution setting for that host. The waypoint enforces the rule in both single cluster and multicluster meshes.

Step 4: Test HTTP-aware failover

Test how the waypoint handles failover when a backend returns errors. This demonstration uses the httpbin /status/{code} endpoint to simulate a failing service that returns 5xx errors.

Get the name of one of the in-ambient pods to make it “unhealthy” by having it return 5xx errors.

POD_TO_FAIL=$(kubectl get pod -n httpbin -l app=in-ambient -o jsonpath='{.items[0].metadata.name}')
echo "Pod to simulate failure: $POD_TO_FAIL"

Simulate a failing endpoint by patching the selected pod to run a command that makes it unresponsive. This creates a scenario where requests timeout, triggering HTTP 503 responses from the waypoint.
```
kubectl exec -n httpbin $POD_TO_FAIL -- pkill -9 gunicorn
```
Send multiple requests to trigger the outlier detection. Because one pod is unresponsive, some requests will timeout and the waypoint will return 503 errors for those attempts. Send enough requests to trigger the consecutive 5xx threshold (3 consecutive errors as configured in the DestinationRule).
```
kubectl exec -n httpbin deploy/client-in-ambient -- sh -c "
for i in \$(seq 1 15); do
  curl -s --max-time 2 http://in-ambient:8000/hostname || echo 'request failed'
  sleep 0.5
done"
```
Example output showing initial failures followed by successful responses after ejection:
```
in-ambient-7d8f9b6c54-abc12
request failed
request failed
request failed
in-ambient-7d8f9b6c54-def34
in-ambient-7d8f9b6c54-ghi56
in-ambient-7d8f9b6c54-def34
in-ambient-7d8f9b6c54-ghi56
in-ambient-7d8f9b6c54-def34
in-ambient-7d8f9b6c54-ghi56
in-ambient-7d8f9b6c54-def34
in-ambient-7d8f9b6c54-ghi56
```
Notice how after the first few failures, the unhealthy pod is ejected and traffic is distributed only among the remaining healthy pods.

Review the waypoint logs to observe outlier detection in action.

kubectl logs -n httpbin deploy/httpbin-waypoint --tail=50 | grep -E "5xx|outlier|ejecting"

Example logs showing outlier detection:

[2025-03-11T19:45:12.234Z] "GET /hostname HTTP/1.1" 503 - via_upstream - "-" 0 95 2 1 "-"
"curl/7.88.1" "xyz789-abc123" "in-ambient.httpbin.svc.cluster.local:8000"
"10.10.0.16:8080" outbound|8000||in-ambient.httpbin.svc.cluster.local 10.10.0.20:52341
10.10.0.16:8080 10.10.0.14:40123 - -

[2025-03-11T19:45:12.235Z]  warn  envoy outlier  ejecting endpoint due to consecutive 5xx
endpoint=10.10.0.16:8080 workload="in-ambient-7d8f9b6c54-abc12" consecutive_5xx=3
ejection_time=30s

The logs confirm that:

The waypoint detected consecutive 5xx responses from the unhealthy endpoint
The endpoint was ejected for 30 seconds (as configured in the DestinationRule)
Subsequent traffic was routed only to healthy endpoints

Verify traffic distribution after ejection. Send more requests and confirm that the ejected pod is not receiving traffic.
```
kubectl exec -n httpbin deploy/client-in-ambient -- sh -c "
for i in \$(seq 1 10); do
  curl -s http://in-ambient:8000/hostname
done" | sort | uniq -c
```
Example output showing traffic distributed only among the two healthy pods:
```
5 in-ambient-7d8f9b6c54-def34
5 in-ambient-7d8f9b6c54-ghi56
```
Notice that the previously failing pod (abc12 in this example) is not included in the distribution, proving that the waypoint successfully ejected it from the load balancing pool.

Wait for the ejection period to expire (30 seconds as configured in the DestinationRule) or for the pod to recover, then verify that traffic is distributed across all three replicas again.

sleep 35
kubectl exec -n httpbin deploy/client-in-ambient -- sh -c "
for i in \$(seq 1 12); do
  curl -s http://in-ambient:8000/hostname
done" | sort | uniq -c

Example output showing all three pods receiving traffic again:

4 in-ambient-7d8f9b6c54-abc12
4 in-ambient-7d8f9b6c54-def34
4 in-ambient-7d8f9b6c54-ghi56

Cleanup

You can optionally remove the resources that you created in this guide. If you want to continue to the other load balancing and failover guides, you can keep the resources for use in those guides as well.

kubectl delete destinationrule in-ambient-failover -n httpbin
kubectl delete gateway httpbin-waypoint -n httpbin
kubectl delete namespace httpbin

Next steps

For load balancing with an ingress gateway, see L7 load balancing with Solo Enterprise for kgateway.
For multicluster failover scenarios, see Multicluster zone and region failover.

Ztunnel (L4)L7 load balancing with kgateway