For the complete documentation index, see llms.txt. Markdown versions of all docs pages are available by appending .md to any docs URL.

Configure and view service metrics from ztunnel and waypoint proxies in ambient mesh.

Metrics provide a way of monitoring and understanding behavior in aggregate. The two layers of ambient mesh generate different metrics:

• Ztunnel generates TCP metrics for all service traffic
• Waypoint proxies generate request and response metrics for HTTP, HTTP/2, and gRPC traffic.

To monitor service behavior, Istio generates metrics for all service traffic in, out, and within an Istio service mesh. These metrics provide information on behaviors such as the overall volume of traffic, the error rates within the traffic, and the response times for requests.

In addition to monitoring the behavior of services within a mesh, monitoring the behavior of the mesh itself is also important. Istio components export metrics on their own internal behaviors to provide insight on the health and function of the mesh control plane.

Metric sources

Istio provides a set of service-oriented metrics for monitoring service communications. These metrics cover the four basic service monitoring needs: latency, traffic, errors, and saturation.

Each proxy generates a rich set of metrics about all traffic passing through the proxy (both inbound and outbound). The proxies also provide detailed statistics about the administrative functions of the proxy itself, including configuration and health information.

Metrics are accessible by making an HTTP request to either /stats/prometheus or /metrics on port 15020.

Istio proxies are marked up with prometheus.io annotations, which causes Prometheus to scrape them by default.

To query and visualize metrics from your ambient mesh, see Prometheus and Grafana. For a curated metrics dashboard with no additional configuration, see Solo UI metrics.

Differences between the layers

ztunnel

Ztunnel exposes the following sets of metrics:

• TCP metrics from Istio’s standard metrics:
  • istio_tcp_sent_bytes_total: In a TCP connection, the total number of bytes that are sent in the response.
  • istio_tcp_received_bytes_total: In a TCP connection, the size of total bytes received during the request.
  • istio_tcp_connections_opened_total: The total number of open TCP connections.
  • istio_tcp_connections_closed_total: The total number of closed TCP connections.
• Various DNS and debugging metrics

Example istio_tcp_connections_opened_total metric for ztunnel:

istio_tcp_connections_opened_total{
   reporter="source",
   source_workload="productpage-v1",
   source_canonical_service="productpage",
   source_canonical_revision="v1",
   source_workload_namespace="default",
   source_principal="spiffe://cluster.local/ns/default/sa/bookinfo-productpage",
   source_app="productpage",
   source_version="v1",
   source_cluster="Kubernetes",
   destination_service="reviews.default.svc.cluster.local",
   destination_service_namespace="default",
   destination_service_name="reviews",
   destination_workload="waypoint",
   destination_canonical_service="waypoint",
   destination_canonical_revision="latest",
   destination_workload_namespace="default",
   destination_principal="spiffe://cluster.local/ns/default/sa/waypoint",
   destination_app="waypoint",
   destination_version="latest",
   destination_cluster="Kubernetes",
   request_protocol="tcp",
   response_flags="-",
   connection_security_policy="mutual_tls"
} 1

Waypoint proxies

Waypoint proxies expose the full set of Istio metrics.

An example waypoint metric:

istio_requests_total {
   reporter="waypoint",
   source_workload="productpage-v1",
   source_canonical_service="productpage",
   source_canonical_revision="v1",
   source_workload_namespace="default",
   source_principal="spiffe://cluster.local/ns/default/sa/bookinfo-productpage",
   source_app="productpage",
   source_version="v1",
   source_cluster="Kubernetes",
   destination_workload="details-v1",
   destination_workload_namespace="default",
   destination_principal="spiffe://cluster.local/ns/default/sa/bookinfo-details",
   destination_app="details",
   destination_version="v1",
   destination_service="details.default.svc.cluster.local",
   destination_canonical_service="details",
   destination_canonical_revision="v1",
   destination_service_name="details",
   destination_service_namespace="default",
   destination_cluster="Kubernetes",
   request_protocol="http",
   response_code="200",
   grpc_response_status="",
   response_flags="-",
   connection_security_policy="mutual_tls"
} 1

Waypoint proxies also expose Envoy statistics as metrics.