Gateway metrics
Use OpenTelemetry collectors to scrape metrics from your gateway proxies and visualize these metrics in Grafana.
This guide shows you how to set up an observability pipeline in Gloo Gateway OSS, or to customize a pipeline in Gloo Gateway Enterprise. For default installations of Gloo Gateway Enterprise, you can simply follow the steps to set up the UI, which includes all necessary observability components.
About
Metrics are essential to gain insight into the health and performance of your gateway proxies. OpenTelemetry is a flexible open source framework that provides a set of APIs, libraries, and instrumentation to help capture and export telemetry data, such as metrics. The framework can also be used to collect traces and logs from your apps. Then, you can use observability tools, such as Grafana or Prometheus, to visualize your metrics so that you can analyze the health of your gateway and troubleshoot issues more easily.
In this guide, you deploy an OpenTelemetry collector that scapes metrics from the Gloo Gateway proxies in the data plane, the Gloo Gateway pods in the control plane, and the external auth and rate limiting add-ons. The metrics that are collected by the OpenTelemetry collector are exposed in Prometheus format. To visualize these metrics, you also deploy a Grafana instance that scrapes the metrics from the OpenTelemetry collector.
If you do not want to deploy an OpenTelemetry collector and Grafana, you can quickly see the raw Prometheus metrics that are automatically exposed on the gateway proxy by accessing the Prometheus metrics on your gateway.
- Port-forward the gateway deployment on port 19000.
- Access the gateway metrics by reviewing the Prometheus statistics.
Before you begin
Follow the Get started guide to install Gloo Gateway, set up a gateway resource, and deploy the httpbin sample app.
Get the external address of the gateway and save it in an environment variable.
Set up an OpenTelemetry collector
Add the Helm repository for OpenTelemetry.
Install the OpenTelemetry collector in your cluster. This command sets up pipelines that scrape metrics from the gateway proxies, Gloo Gateway control plane, and external auth and rate limiting add-ons, and exposes them in Prometheus format.
Verify that the OpenTelemetry collector pod is running.
Example output:
Set up Grafana
Deploy Grafana and other Prometheus components in your cluster. The following example uses the kube-prometheus-stack community Helm chart to install these components.
Verify that the Prometheus stack’s components are up and running.
Example output:
Create a PodMonitor resource to scrape metrics from the OpenTelemetry collector.
Save the sample Grafana dashboard configuration as
envoy.json
.Import the Grafana dashboard.
Visualize metrics in Grafana
Generate traffic for the httpbin app.
Open Grafana and log in to Grafana by using the username
admin
and passwordprom-operator
.Go to Dashboards > Envoy to open the dashboard that you imported. Verify that you see the traffic that you generated for the httpbin app.
Cleanup
You can optionally remove the resources that you set up as part of this guide.Remove the configmap for the Envoy dashboard.
Remove the PodMonitor.
Uninstall Grafana.
Uninstall the OpenTelemetry collector.
Remove the
monitoring
andotel
namespaces.