Install Portal with analytics

Set up Portal with monitoring analytics so that you can review usage information about the API products that are exposed in your developer portal. You can review information such as total requests, total users, total services, and error rates.

Monitoring helps you verify that the different components you set up for Portal are running as expected. For example, you can monitor the status of the portal server, portal frontend app, ingress gateway, external auth server, rate limiter, and backing Redis database. Additionally, you can monitor the status of the Gloo custom resources you configured Portal with, including route tables and policies. You can also enable the ingress gateway to log analytics about your portal API products.

About analytics link

As an API product owner, you can collect analytics about your API usage. This way, you can identify ways to better monetize your API products based on actual usage. You can enable various Gloo Mesh Gateway features to enable analytics for API products that your portal exposes.

Review the following diagram and description of Portal API Analytics.

When the end user queries one of your APIs, your gateway handles the request. You can enable the gateway to generate access logs. This way, the Open Telemetry (OTel) metrics pipeline that you set up can store the access logs in a Clickhouse storage database. As the Portal admin, you can then view usage information for your API products in the Gloo UI through a Grafana dashboard.

The API analytics you can collect from the access logs include the following:

• Total requests of a consumer: Count of all requests from one user email address.
• Average error rate: Percentage of requests with error status codes.
• Total consumers: Count of all requests from distinct user email addresses.
• Total services: Count of all services that receive traffic from users.

Before you begin link

1. Create or use an existing Kubernetes or OpenShift cluster, and save the cluster name in an environment variable. Note: The cluster name must be alphanumeric with no special characters except a hyphen (-), lowercase, and begin with a letter (not a number).

     export CLUSTER_NAME=<cluster_name>
     

2. Set your Gloo Mesh Gateway license key as an environment variable. If you do not have one, contact an account representative. If you prefer to specify license keys in a secret instead, see Licensing. To check your license’s validity, you can run meshctl license check --key $(echo ${GLOO_MESH_GATEWAY_LICENSE_KEY} | base64 -w0).

     export GLOO_MESH_GATEWAY_LICENSE_KEY=<license_key>
     

3. Set the Gloo Mesh Gateway version. This example uses the latest version. You can find other versions in the Changelog documentation. Append -fips for a FIPS-compliant image, such as 2.5.12-fips. Do not include v before the version number.

     export GLOO_VERSION=2.5.12
     

Set up Portal with analytics link

Complete the following steps to set up Gloo Mesh Gateway with the portal server and configure the ingress gateway to collect portal-related analytics.

Step 1: Install Portal link

Choose from the following options to install the required Portal components to collect API usage analytics during an initial installation or upgrade of Gloo Mesh Gateway.

• Initial installation
• Upgrade an existing installation

Installation option link

Set up the required Portal components as part of your initial Gloo Mesh Gateway installation.

1. Add and update the Helm repository for Gloo Mesh Gateway.

     helm repo add gloo-platform https://storage.googleapis.com/gloo-platform/helm-charts
   helm repo update
     

2. Apply the Gloo Mesh Gateway CRDs to your cluster by creating a gloo-platform-crds Helm release. Note: If you plan to manually deploy and manage your Istio installation in workload clusters rather than using Solo’s Istio lifecycle manager, include the --set installIstioOperator=false flag to ensure that the Istio operator CRD is not managed by this Gloo CRD Helm release.

     helm install gloo-platform-crds gloo-platform/gloo-platform-crds \
      --namespace=gloo-mesh \
      --create-namespace \
      --version $GLOO_VERSION
     

3. Create a secret with the password to use to store access logs in Clickhouse. This example setup uses the base64-encoded password for the value of password. Note that this secret must be in each cluster and namespace where you deploy the Gloo OTel pipeline.

     cat << EOF | kubectl apply -f -
   apiVersion: v1
   kind: Secret
   metadata:
     name: clickhouse-auth
     namespace: gloo-mesh
   type: Opaque
   data:
     # password = password
     password: cGFzc3dvcmQ=
   EOF
     

4. Create a Helm values file for your Gloo Mesh Gateway installation.

     touch gloo-gateway-single.yaml
   open gloo-gateway-single.yaml
     

5. In the Helm values file, include the components needed for Portal analytics and save the file.

     clickhouse:
     enabled: true
   glooAgent:
     enabled: true
     relay:
       serverAddress: gloo-mesh-mgmt-server.gloo-mesh:9900
   glooMgmtServer:
     serviceType: ClusterIP
     registerCluster: true
     enabled: true
     createGlobalWorkspace: true
   glooUi:
     enabled: true
   istioInstallations:
     controlPlane:
       enabled: true
       installations:
         - istioOperatorSpec:
             meshConfig:
               # Enable access logging to /dev/stdout
               accessLogFile: /dev/stdout
               # Encoding for the access log (TEXT or JSON). Default value is TEXT.
               accessLogEncoding: JSON
               # If empty, the default log format is used.
               # See the default log format at https://istio.io/latest/docs/tasks/observability/logs/access-log/#default-access-log-format
               # To change the format, see https://www.envoyproxy.io/docs/envoy/latest/configuration/observability/access_log/usage#format-rules
               accessLogFormat: |
                 {
                   "timestamp": "%START_TIME%",
                   "server_name": "%REQ(:AUTHORITY)%",
                   "response_duration": "%DURATION%",
                   "request_command": "%REQ(:METHOD)%",
                   "request_uri": "%REQ(X-ENVOY-ORIGINAL-PATH?:PATH)%",
                   "request_protocol": "%PROTOCOL%",
                   "status_code": "%RESPONSE_CODE%",
                   "client_address": "%DOWNSTREAM_REMOTE_ADDRESS_WITHOUT_PORT%",
                   "x_forwarded_for": "%REQ(X-FORWARDED-FOR)%",
                   "bytes_sent": "%BYTES_SENT%",
                   "bytes_received": "%BYTES_RECEIVED%",
                   "user_agent": "%REQ(USER-AGENT)%",
                   "downstream_local_address": "%DOWNSTREAM_LOCAL_ADDRESS%",
                   "requested_server_name": "%REQUESTED_SERVER_NAME%",
                   "request_id": "%REQ(X-REQUEST-ID)%",
                   "response_flags": "%RESPONSE_FLAGS%",
                   "route_name": "%ROUTE_NAME%",
                   "upstream_cluster": "%UPSTREAM_CLUSTER%",
                   "upstream_host": "%UPSTREAM_HOST%",
                   "upstream_local_address": "%UPSTREAM_LOCAL_ADDRESS%",
                   "upstream_service_time": "%REQ(x-envoy-upstream-service-time)%",
                   "upstream_transport_failure_reason": "%UPSTREAM_TRANSPORT_FAILURE_REASON%",
                   "correlation_id": "%REQ(X-CORRELATION-ID)%",
                   "user_id": "%DYNAMIC_METADATA(envoy.filters.http.ext_authz:userId)%",
                   "api_id": "%DYNAMIC_METADATA(io.solo.gloo.apimanagement:api_id)%",
                   "api_product_id": "%DYNAMIC_METADATA(io.solo.gloo.apimanagement:api_product_id)%",
                   "api_product_name": "%DYNAMIC_METADATA(io.solo.gloo.apimanagement:api_product_name)%",
                   "usage_plan": "%DYNAMIC_METADATA(envoy.filters.http.ext_authz:usagePlan)%",
                   "custom_metadata": "%DYNAMIC_METADATA(io.solo.gloo.apimanagement.custom_metadata)%"
                 }
           revision: auto
     enabled: true
     northSouthGateways:
       - enabled: true
         installations:
           - gatewayRevision: auto
             istioOperatorSpec: {}
         name: istio-ingressgateway
   telemetryCollector:
     enabled: true
     presets:
       logsCollection:
         enabled: true
         storeCheckpoints: true
     config:
       exporters:
         otlp:
           endpoint: gloo-telemetry-gateway.gloo-mesh:4317
   telemetryCollectorCustomization: 
     pipelines:
       logs/portal:
         enabled: true
     extraExporters:
       clickhouse:
         password: "${env:CLICKHOUSE_PASSWORD}"
   prometheus:
     enabled: true
   redis:
     deployment:
       enabled: true
   telemetryGateway:
     enabled: true
     service:
       type: ClusterIP
     extraEnvs:
     - name: CLICKHOUSE_PASSWORD
       valueFrom:
         secretKeyRef:
           key: password
           name: clickhouse-auth
     - name: KUBE_NODE_NAME
       valueFrom:
         fieldRef:
           fieldPath: spec.nodeName
     - name: KUBE_POD_NAME
       valueFrom:
         fieldRef:
           fieldPath: metadata.name
     - name: REDIS_USERNAME
       valueFrom:
         secretKeyRef:
           key: username
           name: redis-auth-secrets
           optional: true
     - name: REDIS_PASSWORD
       valueFrom:
         secretKeyRef:
           key: password
           name: redis-auth-secrets
           optional: true
   telemetryGatewayCustomization:
     pipelines:
       logs/clickhouse:
         enabled: true
     extraExporters:
       clickhouse:
         password: "${env:CLICKHOUSE_PASSWORD}"
     

   For more information, see the Istio on OpenShift documentation.

   1. Elevate the permissions of the following service accounts that will be created. These permissions allow the to make use of a user ID that is normally restricted by OpenShift. For more information, see the Istio on OpenShift documentation.

        oc adm policy add-scc-to-group anyuid system:serviceaccounts:gloo-mesh
      oc adm policy add-scc-to-group anyuid system:serviceaccounts:istio-system
      oc adm policy add-scc-to-group anyuid system:serviceaccounts:gloo-mesh-gateways
      oc adm policy add-scc-to-group anyuid system:serviceaccounts:gm-iop-1-20
        

   2. Create the gloo-mesh-gateways project, and create a NetworkAttachmentDefinition custom resource for the project.

        kubectl create ns gloo-mesh-gateways
      cat <<EOF | oc -n gloo-mesh-gateways create -f -
      apiVersion: "k8s.cni.cncf.io/v1"
      kind: NetworkAttachmentDefinition
      metadata:
        name: istio-cni
      EOF
        

   3. Elevate the permissions of the service account in each project where you want to deploy workloads.

        oc adm policy add-scc-to-group anyuid system:serviceaccounts:<project>
        

   4. Create a NetworkAttachmentDefinition custom resource for each project where you want to deploy workloads.

        cat <<EOF | oc -n <project> create -f -
      apiVersion: "k8s.cni.cncf.io/v1"
      kind: NetworkAttachmentDefinition
      metadata:
        name: istio-cni
      EOF
        

   5. Prepare the gloo-gateway-single values file.

        clickhouse:
        enabled: true
      glooAgent:
        enabled: true
        floatingUserId: true
        relay:
          serverAddress: gloo-mesh-mgmt-server.gloo-mesh:9900
      glooMgmtServer:
        enabled: true
        floatingUserId: true
        registerCluster: true
        serviceType: ClusterIP
        createGlobalWorkspace: true
      glooUi:
        enabled: true
        floatingUserId: true
      istioInstallations:
        controlPlane:
          enabled: true
          installations:
          - clusters: null
            istioOperatorSpec:
              components:
                # Openshift requires the Istio CNI feature to be enabled
                cni:
                  enabled: true
                  namespace: kube-system
                  k8s:
                    overlays:
                      - kind: DaemonSet
                        name: istio-cni-node
                        patches:
                          - path: spec.template.spec.containers[0].securityContext.privileged
                            value: true
                pilot:
                  k8s:
                    env:
                    - name: PILOT_ENABLE_K8S_SELECT_WORKLOAD_ENTRIES
                      value: "false"
                    - name: PILOT_SKIP_VALIDATE_TRUST_DOMAIN
                      value: "true"
              meshConfig:
                accessLogFile: /dev/stdout
                accessLogEncoding: JSON
                defaultConfig:
                  envoyAccessLogService:
                    address: gloo-mesh-agent.gloo-mesh:9977
                  holdApplicationUntilProxyStarts: true
                  proxyMetadata:
                    ISTIO_META_DNS_AUTO_ALLOCATE: "true"
                    ISTIO_META_DNS_CAPTURE: "true"
                outboundTrafficPolicy:
                  mode: ALLOW_ANY
                rootNamespace: istio-system
                accessLogFormat: |
                  {
                    "timestamp": "%START_TIME%",
                    "server_name": "%REQ(:AUTHORITY)%",
                    "response_duration": "%DURATION%",
                    "request_command": "%REQ(:METHOD)%",
                    "request_uri": "%REQ(X-ENVOY-ORIGINAL-PATH?:PATH)%",
                    "request_protocol": "%PROTOCOL%",
                    "status_code": "%RESPONSE_CODE%",
                    "client_address": "%DOWNSTREAM_REMOTE_ADDRESS_WITHOUT_PORT%",
                    "x_forwarded_for": "%REQ(X-FORWARDED-FOR)%",
                    "bytes_sent": "%BYTES_SENT%",
                    "bytes_received": "%BYTES_RECEIVED%",
                    "user_agent": "%REQ(USER-AGENT)%",
                    "downstream_local_address": "%DOWNSTREAM_LOCAL_ADDRESS%",
                    "requested_server_name": "%REQUESTED_SERVER_NAME%",
                    "request_id": "%REQ(X-REQUEST-ID)%",
                    "response_flags": "%RESPONSE_FLAGS%",
                    "route_name": "%ROUTE_NAME%",
                    "upstream_cluster": "%UPSTREAM_CLUSTER%",
                    "upstream_host": "%UPSTREAM_HOST%",
                    "upstream_local_address": "%UPSTREAM_LOCAL_ADDRESS%",
                    "upstream_service_time": "%REQ(x-envoy-upstream-service-time)%",
                    "upstream_transport_failure_reason": "%UPSTREAM_TRANSPORT_FAILURE_REASON%",
                    "correlation_id": "%REQ(X-CORRELATION-ID)%",
                    "user_id": "%DYNAMIC_METADATA(envoy.filters.http.ext_authz:userId)%",
                    "api_id": "%DYNAMIC_METADATA(io.solo.gloo.apimanagement:api_id)%",
                    "api_product_id": "%DYNAMIC_METADATA(io.solo.gloo.apimanagement:api_product_id)%",
                    "api_product_name": "%DYNAMIC_METADATA(io.solo.gloo.apimanagement:api_product_name)%",
                    "usage_plan": "%DYNAMIC_METADATA(envoy.filters.http.ext_authz:usagePlan)%",
                    "custom_metadata": "%DYNAMIC_METADATA(io.solo.gloo.apimanagement.custom_metadata)%"
                  }
          name: istio-ingressgateway
              namespace: istio-system
              # Openshift-specific installation (https://istio.io/latest/docs/setup/additional-setup/config-profiles/)
              # Note: This profile is currently required to install the istio-cni
              profile: openshift
              values:
                # CNI options for OpenShift
                cni:
                  cniBinDir: /var/lib/cni/bin
                  cniConfDir: /etc/cni/multus/net.d
                  chained: false
                  cniConfFileName: "istio-cni.conf"
                  excludeNamespaces:
                  - istio-system
                  - kube-system
                  logLevel: info
                sidecarInjectorWebhook:
                  injectedAnnotations:
                    k8s.v1.cni.cncf.io/networks: istio-cni
            revision: auto
        eastWestGateways: null
        enabled: true
        northSouthGateways:
        - enabled: true
          installations:
          - clusters: null
            gatewayRevision: auto
            istioOperatorSpec: {}
          name: istio-ingressgateway
          namespace: gloo-mesh-gateways
      prometheus:
        enabled: true
      redis:
        deployment:
          enabled: true
          floatingUserId: true
      telemetryCollector:
        config:
          exporters:
            otlp:
              endpoint: gloo-telemetry-gateway.gloo-mesh:4317
        enabled: true
        presets:
          logsCollection:
            enabled: true
            storeCheckpoints: true
      telemetryCollectorCustomization: 
        pipelines:
          logs/portal:
            enabled: true
        extraExporters:
          clickhouse:
            password: "${env:CLICKHOUSE_PASSWORD}"
      telemetryGateway:
        enabled: true
        service:
          type: ClusterIP
        extraEnvs:
        - name: CLICKHOUSE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: password
              name: clickhouse-auth
      telemetryGatewayCustomization:
        pipelines:
          logs/clickhouse:
            enabled: true
        extraExporters:
          clickhouse:
            password: "${env:CLICKHOUSE_PASSWORD}"
        

      Note: When you use the settings in this profile to install Gloo Mesh Gateway in OpenShift 4.11 and later, you might see warnings for the pods and containers which violate the OpenShift PodSecurity "restricted:v1.24" profile, due to the elevated permissions required by Istio. You can ignore these warnings. For more info, see this article.

6. Create another Helm values file for the add-ons that you need.

     touch gloo-gateway-addons.yaml
   open gloo-gateway-addons.yaml
     

7. In the Helm values file, include including the external auth service, rate limiter, and portal server, and save the file. The following example also sets up the local Redis instance to be used for backing storage for the servers. For more backing storage options such as to bring your own Redis with auth, see the Backing Redis databases guide.

     common:
     addonNamespace: gloo-mesh
   extAuthService:
     enabled: true
     extAuth:
       apiKeyStorage:
         name: redis
         enabled: true
         config:
           host: "redis.gloo-mesh:6379"
           db: 0
         secretKey: "ThisIsSecret"
   glooPortalServer:
     enabled: true
     apiKeyStorage:
       redis:
         enabled: true
         address: redis.gloo-mesh:6379
       configPath: /etc/redis-client-config/config.yaml
       secretKey: "ThisIsSecret"
   rateLimiter:
     enabled: true
     

8. Install the Gloo Mesh Gateway management plane and data plane components in your cluster, including the customizations in your Helm values file.

     helm install gloo-platform gloo-platform/gloo-platform \
      --namespace gloo-mesh \
      --version $GLOO_VERSION \
      --values gloo-gateway-single.yaml \
      --set common.cluster=$CLUSTER_NAME \
      --set licensing.glooGatewayLicenseKey=$GLOO_MESH_GATEWAY_LICENSE_KEY
     

9. Install the Gloo Mesh Gateway add-ons in a separate Helm release.

     helm install gloo-agent-addons gloo-platform/gloo-platform \
      --namespace gloo-mesh \
      --create-namespace \
      --version $GLOO_VERSION \
      --set common.cluster=$CLUSTER_NAME \
      --values gloo-gateway-addons.yaml
     

   1. Elevate the permissions of the gloo-telemetry-collector and the gloo-mesh service account that will be created. The telemetry collector agent requires the hostmount-anyuid permission to read the varlogpods, varlibdockercontainers, and cilium-run volumes.

        oc adm policy add-scc-to-group anyuid system:serviceaccounts:gloo-mesh
      oc adm policy add-scc-to-user hostmount-anyuid -z gloo-telemetry-collector -n gloo-mesh
        

   2. Create a NetworkAttachmentDefinition custom resource for the project.

        cat <<EOF | oc -n gloo-mesh create -f -
      apiVersion: "k8s.cni.cncf.io/v1"
      kind: NetworkAttachmentDefinition
      metadata:
        name: istio-cni
      EOF
        

   3. Create the add-ons release.

        helm install gloo-agent-addons gloo-platform/gloo-platform \
         --namespace gloo-mesh \
         --version $GLOO_VERSION \
         --set common.cluster=$CLUSTER_NAME \
         --values gloo-gateway-addons.yaml
        

10. Verify that Portal and the related components are installed.

      meshctl check
      

    In the example output, make sure that the portal, external auth, and rate limiting servers and all of the core Gloo Mesh Gateway components are healthy.

      🟢 Gloo Mesh Gateway License Status
    
     INFO  gloo-gateway enterprise license expires on 05 Nov 23 14:18 EST
    
    🟢 CRD Version check
    
    🟢 Gloo deployment status
    
    Namespace        | Name                           | Ready | Status 
    gloo-mesh        | ext-auth-service               | 1/1   | Healthy
    gloo-mesh        | gloo-mesh-agent                | 1/1   | Healthy
    gloo-mesh        | gloo-mesh-mgmt-server          | 1/1   | Healthy
    gloo-mesh        | gloo-mesh-portal-server        | 1/1   | Healthy
    gloo-mesh        | gloo-mesh-redis                | 1/1   | Healthy
    gloo-mesh        | gloo-mesh-ui                   | 1/1   | Healthy
    gloo-mesh        | prometheus-server              | 1/1   | Healthy
    gloo-mesh        | rate-limiter                   | 1/1   | Healthy
    gloo-mesh        | redis                          | 1/1   | Healthy
    gloo-mesh        | gloo-telemetry-collector-agent | 3/3   | Healthy
    
    🟢 Mgmt server connectivity to workload agents
    
    Cluster  | Registered | Connected Pod                                   
    cluster1 | true       | gloo-mesh/gloo-mesh-mgmt-server-65bd557b95-v8qq6
      

11. Verify that the gateway proxy service is created and assigned an external IP address. It might take a few minutes for the load balancer to deploy.

      kubectl get svc -n gloo-mesh-gateways
      

    Example output:

      NAME                   TYPE           CLUSTER-IP      EXTERNAL-IP      PORT(S)                                                      AGE
    istio-ingressgateway   LoadBalancer   10.XX.XXX.XXX   35.XXX.XXX.XXX   15021:30826/TCP,80:31257/TCP,443:30673/TCP,15443:30789/TCP   48s
      

Now that your Gloo Mesh Gateway components for Portal are installed, set up Grafana.

Upgrade option link

Set up the required Portal components by upgrading your existing Gloo Mesh Gateway installation.

1. Check the Helm releases in your cluster. Depending on your installation method, you either have only a main installation release (such as gloo-platform), or a main installation and a separate add-ons release (such as gloo-agent-addons), in addition to your CRDs release.

     helm list --all-namespaces
     

   Example of separate releases for platform and add-ons:

     NAME                 	NAMESPACE       	REVISION	UPDATED                             	STATUS  	CHART
   gloo-agent-addons    	gloo-mesh       	1       	2023-09-01 13:29:03.136686 -0400 EDT	deployed	gloo-platform-2.4.0
   gloo-platform        	gloo-mesh       	1      	  2023-09-01 13:26:56.061102 -0400 EDT	deployed	gloo-platform-2.4.0
   gloo-platform-crds   	gloo-mesh       	1       	2023-09-01 13:23:56.061102 -0400 EDT	deployed	gloo-platform-crds-2.4.0
     

2. Get your current installation values.

   • If you have only one release for your installation, get those values. Note that your Helm release might have a different name.

       helm get values gloo-platform -n gloo-mesh -o yaml > gloo-gateway-single.yaml
     open gloo-gateway-single.yaml
       

   • If you have a separate add-ons release, get those values.

       helm get values gloo-agent-addons -n gloo-mesh -o yaml > gloo-agent-addons.yaml
     open gloo-agent-addons.yaml
       

3. Add or edit the following settings for the required add-ons, including the external auth, rate limiting, and portal servers. The following example also sets up the local Redis instance to be used for backing storage for the servers. For more backing storage options, see the Backing Redis databases guide.

   Note that the analytics enablement is only in the gloo-platform Helm values file.

   • gloo-platform Helm values file: Download the GitHub example file. Update the values file, such as to match the following example.
     • clickhouse to store the access logs.
     • istioInstallations with the Portal access log formats.
     • telemetryCollector to collect access logs.
     • telemetryCollectorCustomization to enable the Istio access logs pipeline.
     • telemetryGateway to enable the Gloo telemetry gateway, including to refer to the secret with the Clickhouse password that you previously created.
     • telemetryGatewayCustomization to set up the Clickhouse logs pipeline, including the Clickhouse password.
   • gloo-agent-addons Helm values file: Download the GitHub example file. Update the values file to include the external auth service, rate limiter, and portal server.

   Download the GitHub example file. Update the values file, such as to match the following example.

   • clickhouse to store the access logs.
   • extAuthService to configure the external auth service to share the same backing Redis instance as the Portal server.
   • glooPortalServer to configure the Portal to share the same backing Redis instance as the external auth service.
   • istioInstallations with the Portal access log formats.
   • rateLimiter to enable for usage plans.
   • telemetryCollector to collect access logs.
   • telemetryCollectorCustomization to enable the Istio access logs pipeline.
   • telemetryGateway to enable the Gloo telemetry gateway, including to refer to the secret with the Clickhouse password that you previously created.
   • telemetryGatewayCustomization to set up the Clickhouse logs pipeline, including the Clickhouse password.

4. Upgrade your Helm release with the Helm values that you previously prepared.

   • If you have only one release for your installation, upgrade the gloo-platform release.

       helm upgrade gloo-platform gloo-platform/gloo-platform \
        --namespace gloo-mesh \
        -f gloo-gateway-single.yaml \
        --version $GLOO_VERSION
       

   • If you have a separate add-ons release, upgrade the gloo-agent-addons release as well.

       helm upgrade gloo-agent-addons gloo-platform/gloo-platform \
        --namespace gloo-mesh \
        -f gloo-agent-addons.yaml \
        --version $GLOO_VERSION
       

5. Verify that Portal and the related components are installed.

     meshctl check
     

   In the example output, make sure that the portal, external auth, and rate limiting servers and all of the core Gloo Mesh Gateway components are healthy.

     🟢 Gloo Mesh Gateway License Status
   
    INFO  gloo-gateway enterprise license expires on 05 Nov 23 14:18 EST
   
   🟢 CRD Version check
   
   🟢 Gloo deployment status
   
   Namespace        | Name                           | Ready | Status 
   gloo-mesh        | ext-auth-service               | 1/1   | Healthy
   gloo-mesh        | gloo-mesh-agent                | 1/1   | Healthy
   gloo-mesh        | gloo-mesh-mgmt-server          | 1/1   | Healthy
   gloo-mesh        | gloo-mesh-portal-server        | 1/1   | Healthy
   gloo-mesh        | gloo-mesh-redis                | 1/1   | Healthy
   gloo-mesh        | gloo-mesh-ui                   | 1/1   | Healthy
   gloo-mesh        | prometheus-server              | 1/1   | Healthy
   gloo-mesh        | rate-limiter                   | 1/1   | Healthy
   gloo-mesh        | redis                          | 1/1   | Healthy
   gloo-mesh        | gloo-telemetry-collector-agent | 3/3   | Healthy
   
   🟢 Mgmt server connectivity to workload agents
   
   Cluster  | Registered | Connected Pod                                   
   cluster1 | true       | gloo-mesh/gloo-mesh-mgmt-server-65bd557b95-v8qq6
     

Now that your Gloo Mesh Gateway components for Portal are installed, set up Grafana.

Step 2: Set up Grafana to use Clickhouse link

You installed or upgraded Gloo Mesh Gateway with the add-ons to run the developer portal and collect access logs for your APIs. Now, you can configure a Grafana instance to pull the data stored in Clickhouse.

1. Verify that the Clickhouse resources are healthy.

     kubectl get all -A -l app.kubernetes.io/name=clickhouse
     

2. Install or upgrade Grafana to use the Clickhouse database, such as with the following commands. Note that the Clickhouse password matches the password of the secret that you previously created.

     helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
   helm repo update
   helm upgrade --install kube-prometheus-stack \
   prometheus-community/kube-prometheus-stack \
   --version 30.0.3 \
   --namespace monitoring \
   --create-namespace \
   --set "grafana.additionalDataSources[0].secureJsonData.password=$(kubectl get secret clickhouse-auth -n gloo-mesh -o jsonpath='{.data.password}' | base64 -d)" \
   --values - <<EOF
   grafana:
     service:
       type: LoadBalancer
     plugins:
     - grafana-clickhouse-datasource
     additionalDataSources:
     - name: ClickHouse
       type: grafana-clickhouse-datasource
       isDefault: false
       uid: clickhouse-access-logs
       jsonData:
         defaultDatabase: default
         port: 9000
         server: clickhouse.gloo-mesh
         username: default
         tlsSkipVerify: true
   EOF
     

3. Verify that the Grafana deployment and the rest of the Prometheus stack is healthy.

     kubectl get pods -n monitoring
     

If Clickhouse or Grafana are not healthy, try the Debugging steps.

Now that your Gloo Mesh Gateway components are set up, you can add the Grafana dashboard for portal usage analytics.

Step 3: Add the portal usage analytics dashboard to Grafana link

You can use the sample Gloo Mesh Gateway Portal API analytics dashboard to monitor portal usage.

1. Set up port forwarding on your local machine to access the Grafana dashboard.

     kubectl port-forward $(kubectl get pods -n monitoring -o name | grep grafana) 8080:3000 -n monitoring
     

2. Open the Grafana dashboard in your web browser.
3. Log in to the Grafana dashboard with admin as the username, and prom-operator as the password. These are the default credentials that are set by the Prometheus community chart. You can change these credentials when you log in to Grafana.
4. Import the Gloo Mesh Gateway Portal API analytics dashboard.
   1. Download the JSON file that holds the configuration for the Gloo Mesh Gateway Portal API analytics dashboard.
   2. From the Grafana menu, select + > Import.
   3. Click Upload JSON file and select the file for the API analytics dashboard that you downloaded.
   4. Click Import to open the API analytics dashboard.

Good job! You set up the Grafana dashboard to monitor the API usage of your developer portal.

Step 4: Monitor API usage with the Grafana dashboard link

With Portal and the related logging components installed, you can monitor the API usage in developer portal.

1. Create your API products.
2. Prepare usage plans for your API products.
3. Configure a developer portal.
4. Generate traffic to your API products, either from your users or such as with the following example request to the Tracks API product .

     curl -v --resolve api.example.com:80:${INGRESS_GW_ADDRESS} http://api.example.com/trackapi/tracks
     

5. Explore the API usage analytics, such as in the following example. You can filter by several properties, such as APIs, usage plans, methods, status codes, and more.

Other monitoring tools link

To help you monitor your environment, Gloo Mesh Gateway provides several tools as follows:

• Gloo UI, which features tabs to help debug your gateways, APIs, portals, and policies.
• Global status reporting on Gloo custom resources, including whether the resource is accepted.
• Logs for the deployments, such as the portal server or ingress gateway.
• API usage analytics for the API products that are exposed in your developer portal.
• Troubleshooting documentation for Gloo Mesh Gateway, including a debug guide for portal API usage analytics.