Request tracing

Sample request traces to monitor the traffic and health of your service mesh.

Distributed tracing helps you track requests across multiple services in your distributed service mesh. Sampling the requests in your mesh can give you insight into request latency, serialization, and parallelism. For more information, see the Istio tracing documentation.

To get started, enable the default Envoy distributed tracing in your Istio installation to trace requests across your service mesh, analyze service dependencies and find bottlenecks or high latency services in your mesh. Then, use a tracing provider, such as the built-in Jaeger platform, to collect and visualize the sampled spans.

1. Enable tracing in the Gloo OTel pipeline.

   This guide walks you through how to set up tracing with the built-in Jaeger platform, which can be used for testing or demo purposes. However, if you plan to set up tracing for your production environment, create your own custom tracing pipeline in the Gloo telemetry pipeline to forward traces to a tracing platform that is managed by your organization and hardened for production. Alternatively, you can send production traces to a SaaS backend. Use the steps in Bring your own Jaeger instance as a guide to integrate your own tracing platform.

2. Instrument the workloads in your service mesh to collect traces by updating your Istio installation. The following example meshConfig options set a sampling rate of 100% of requests and traces to the Gloo telemetry collector agent. You can add these meshConfig settings to new or existing Istio installations in the following ways:

   • Managed gateway proxies: See the upgrade guide for Istio settings in your Helm chart.
   • Manually installed gateway proxies: See the manual Istio upgrade guide.

   ...
   meshConfig:      
     enableTracing: true
     defaultConfig: 
       tracing: 
         sampling: 100
         zipkin:
           address: gloo-telemetry-collector.gloo-mesh.svc.cluster.local:9411
   ...
   

3. Restart the workloads in your service mesh that you want to collect traces for. For example, to restart the product page app, run the following command.

   kubectl rollout restart deployment productpage-v1 -n bookinfo --context $REMOTE_CONTEXT1  
   

4. Open the product page app and send a few sample requests by refreshing the page a couple of times. Each request produces Istio traces that are sent to the Jaeger instance that you configured.

   kubectl port-forward deployment/productpage-v1 -n bookinfo --context $REMOTE_CONTEXT1 9080
   

5. Open the Gloo UI and go to the Tracing menu to view your traces.

   

For more information about the sampling rate, custom tag, and maximum path length settings, see the Istio tracing configuration docs.