Install the Gloo components required to run the developer portal with analytics.
Set up Portal with monitoring analytics so that you can review usage information about the API products that are exposed in your developer portal. You can review information such as total requests, total users, total services, and error rates.
Monitoring helps you verify that the different components you set up for Portal are running as expected. For example, you can monitor the status of the portal server, portal frontend app, ingress gateway, external auth server, rate limiter, and backing Redis database. Additionally, you can monitor the status of the Gloo custom resources you configured Portal with, including route tables and policies. You can also enable the ingress gateway to log analytics about your portal API products.
As an API product owner, you can collect analytics about your API usage. This way, you can identify ways to better monetize your API products based on actual usage. You can enable various Gloo Mesh Gateway features to enable analytics for API products that your portal exposes.
Review the following diagram and description of Portal API Analytics.
Figure: Portal analytics overview
When the end user queries one of your APIs, your gateway handles the request. You can enable the gateway to generate access logs. This way, the Open Telemetry (OTel) metrics pipeline that you set up can store the access logs in a Clickhouse storage database. As the Portal admin, you can then view usage information for your API products in the Gloo UI through a Grafana dashboard.
The API analytics you can collect from the access logs include the following:
Total requests of a consumer: Count of all requests from one user email address.
Average error rate: Percentage of requests with error status codes.
Total consumers: Count of all requests from distinct user email addresses.
Total services: Count of all services that receive traffic from users.
Create or use an existing Kubernetes or OpenShift cluster, and save the cluster name in an environment variable. Note: The cluster name must be alphanumeric with no special characters except a hyphen (-), lowercase, and begin with a letter (not a number).
export CLUSTER_NAME=<cluster_name>
Set your Gloo Mesh Gateway license key as an environment variable. If you do not have one, contact an account representative. If you prefer to specify license keys in a secret instead, see Licensing. To check your license’s validity, you can run meshctl license check --key $(echo ${GLOO_MESH_GATEWAY_LICENSE_KEY} | base64 -w0).
Set the Gloo Mesh Gateway version. This example uses the latest version. You can find other versions in the Changelog documentation. Append -fips for a FIPS-compliant image, such as 2.6.0-beta3-fips. Do not include v before the version number.
Choose from the following options to install the required Portal components to collect API usage analytics during an initial installation or upgrade of Gloo Mesh Gateway.
Apply the Gloo Mesh Gateway CRDs to your cluster by creating a gloo-platform-crds Helm release. Note: If you plan to manually deploy and manage your Istio installation in workload clusters rather than using Solo’s Istio lifecycle manager, include the --set installIstioOperator=false flag to ensure that the Istio operator CRD is not managed by this Gloo CRD Helm release.
Create a secret with the password to use to store access logs in Clickhouse. This example setup uses the base64-encoded password for the value of password. Note that this secret must be in each cluster and namespace where you deploy the Gloo OTel pipeline.
Elevate the permissions of the following service accounts that will be created. These permissions allow the to make use of a user ID that is normally restricted by OpenShift. For more information, see the Istio on OpenShift documentation.
Note: When you use the settings in this profile to install Gloo Mesh Gateway in OpenShift 4.11 and later, you might see warnings for the pods and containers which violate the OpenShift PodSecurity "restricted:v1.24" profile, due to the elevated permissions required by Istio. You can ignore these warnings. For more info, see this article.
Create another Helm values file for the add-ons that you need.
touch gloo-gateway-addons.yaml
open gloo-gateway-addons.yaml
In the Helm values file, include including the external auth service, rate limiter, and portal server, and save the file. The following example also sets up the local Redis instance to be used for backing storage for the servers. For more backing storage options such as to bring your own Redis with auth, see Portal server.
Verify that Portal and the related components are installed.
meshctl check
In the example output, make sure that the portal, external auth, and rate limiting servers and all of the core Gloo Mesh Gateway components are healthy.
🟢 Gloo Mesh Gateway License Status
INFO gloo-gateway enterprise license expires on 05 Nov 23 14:18 EST
🟢 CRD Version check
🟢 Gloo deployment status
Namespace | Name | Ready | Status
gloo-mesh | gloo-mesh-agent | 1/1 | Healthy
gloo-mesh | gloo-mesh-mgmt-server | 1/1 | Healthy
gloo-mesh | gloo-mesh-redis | 1/1 | Healthy
gloo-mesh | gloo-mesh-ui | 1/1 | Healthy
gloo-mesh | prometheus-server | 1/1 | Healthy
gloo-mesh-addons | ext-auth-service | 1/1 | Healthy
gloo-mesh-addons | gloo-mesh-portal-server | 1/1 | Healthy
gloo-mesh-addons | rate-limiter | 1/1 | Healthy
gloo-mesh-addons | redis | 1/1 | Healthy
gloo-mesh | gloo-telemetry-collector-agent | 3/3 | Healthy
🟢 Mgmt server connectivity to workload agents
Cluster | Registered | Connected Pod
cluster1 | true | gloo-mesh/gloo-mesh-mgmt-server-65bd557b95-v8qq6
Verify that the gateway proxy service is created and assigned an external IP address. It might take a few minutes for the load balancer to deploy.
kubectl get svc -n gloo-mesh-gateways
Example output:
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
istio-ingressgateway LoadBalancer 10.XX.XXX.XXX 35.XXX.XXX.XXX 15021:30826/TCP,80:31257/TCP,443:30673/TCP,15443:30789/TCP 48s
Now that your Gloo Mesh Gateway components for Portal are installed, set up Grafana.
Set up the required Portal components by upgrading your existing Gloo Mesh Gateway installation.
info
The following steps upgrade an existing Helm release to make sure that the required external auth, rate limiting, and portal servers are set up. The steps do not upgrade the Gloo Mesh Gateway management server or agent versions or otherwise change the components.
Check the Helm releases in your cluster. Depending on your installation method, you either have only a main installation release (such as gloo-platform), or a main installation and a separate add-ons release (such as gloo-agent-addons), in addition to your CRDs release.
helm list --all-namespaces
Example of separate releases for platform and add-ons:
If you have only one release for your installation, get those values. Note that your Helm release might have a different name.
helm get values gloo-platform -n gloo-mesh -o yaml > gloo-gateway-single.yaml
open gloo-gateway-single.yaml
If you have a separate add-ons release, get those values.
helm get values gloo-agent-addons -n gloo-mesh-addons -o yaml > gloo-agent-addons.yaml
open gloo-agent-addons.yaml
Add or edit the following settings for the required add-ons, including the external auth, rate limiting, and portal servers. The following example also sets up the local Redis instance to be used for backing storage for the servers. For more backing storage options, see Portal backing databases.
Note that the analytics enablement is only in the gloo-platform Helm values file.
gloo-platform Helm values file: Update the values file, such as to match the following example.
clickhouse to store the access logs.
istioInstallations with the Portal access log formats.
telemetryCollector to collect access logs.
telemetryCollectorCustomization to enable the Istio access logs pipeline.
telemetryGateway to enable the Gloo telemetry gateway, including to refer to the secret with the Clickhouse password that you previously created.
telemetryGatewayCustomization to set up the Clickhouse logs pipeline, including the Clickhouse password.
gloo-agent-addons Helm values file: Update the values file to include the external auth service, rate limiter, and portal server.
Verify that Portal and the related components are installed.
meshctl check
In the example output, make sure that the portal, external auth, and rate limiting servers and all of the core Gloo Mesh Gateway components are healthy.
🟢 Gloo Mesh Gateway License Status
INFO gloo-gateway enterprise license expires on 05 Nov 23 14:18 EST
🟢 CRD Version check
🟢 Gloo deployment status
Namespace | Name | Ready | Status
gloo-mesh | gloo-mesh-agent | 1/1 | Healthy
gloo-mesh | gloo-mesh-mgmt-server | 1/1 | Healthy
gloo-mesh | gloo-mesh-redis | 1/1 | Healthy
gloo-mesh | gloo-mesh-ui | 1/1 | Healthy
gloo-mesh | prometheus-server | 1/1 | Healthy
gloo-mesh-addons | ext-auth-service | 1/1 | Healthy
gloo-mesh-addons | gloo-mesh-portal-server | 1/1 | Healthy
gloo-mesh-addons | rate-limiter | 1/1 | Healthy
gloo-mesh-addons | redis | 1/1 | Healthy
gloo-mesh | gloo-telemetry-collector-agent | 3/3 | Healthy
🟢 Mgmt server connectivity to workload agents
Cluster | Registered | Connected Pod
cluster1 | true | gloo-mesh/gloo-mesh-mgmt-server-65bd557b95-v8qq6
Now that your Gloo Mesh Gateway components for Portal are installed, set up Grafana.
You installed or upgraded Gloo Mesh Gateway with the add-ons to run the developer portal and collect access logs for your APIs. Now, you can configure a Grafana instance to pull the data stored in Clickhouse.
Verify that the Clickhouse resources are healthy.
kubectl get all -A -l app.kubernetes.io/name=clickhouse
Install or upgrade Grafana to use the Clickhouse database, such as with the following commands. Note that the Clickhouse password matches the password of the secret that you previously created.
Log in to the Grafana dashboard with admin as the username, and prom-operator as the password. These are the default credentials that are set by the Prometheus community chart. You can change these credentials when you log in to Grafana.
Import the Gloo Mesh Gateway Portal API analytics dashboard.
Download the JSON file that holds the configuration for the Gloo Mesh Gateway Portal API analytics dashboard.
From the Grafana menu, select + > Import.
Click Upload JSON file and select the file for the API analytics dashboard that you downloaded.
Click Import to open the API analytics dashboard.
Good job! You set up the Grafana dashboard to monitor the API usage of your developer portal.
Step 4: Monitor API usage with the Grafana dashboard link
With Portal and the related logging components installed, you can monitor the API usage in developer portal.
Explore the API usage analytics, such as in the following example. You can filter by several properties, such as APIs, usage plans, methods, status codes, and more.