Running into an issue while using Gloo Mesh (OSS APIs)? Try the following general debugging steps to find why things might not behave as expected.

Debug your environment

  1. Make sure that you run a supported version of Gloo, Istio, and Kubernetes and their related command line interface (CLI) tools. If not, upgrade your version and CLIs.
  2. Review your infrastructure provider for any errors, incidents, or outages that might impact your clusters, load balancers, and related infrastructure. Common configuration problems happen in the following resources:
    • Firewalls
    • Application load balancer (ALB) settings
    • Expired or malformatted certificates, tokens, and secrets
  3. Check your Gloo Mesh (OSS APIs) environment for errors, such as an expired license or an unhealthy pod.
      meshctl check --kubecontext ${MGMT_CONTEXT}

    • meshctl:

        meshctl dashboard --kubecontext ${MGMT_CONTEXT}

    • kubectl:

      1. Port-forward the gloo-mesh-ui service on port 8090.
          kubectl port-forward --kubecontext ${MGMT_CONTEXT} -n gloo-mesh svc/gloo-mesh-ui 8090:8090
      2. Open you browser and connect to http://localhost:8090.
  4. Depending on which component has issues, try one of the debugging guides:
  5. To help you debug issues in your environment, you can generate a .tar.gz file of your Gloo and Istio operational information. For more information, see the CLI reference.
      meshctl debug report
    • To generate a file for multiple clusters:
         meshctl debug report --kubecontext cluster1,cluster2,cluster3
    • To include information from application namespaces in addition to the gloo-mesh and Istio namespaces:
         meshctl debug report --include-namespaces app1,app2
    • To upload the debug information to a secure repo owned by Solo.io, you can set a folder structure that makes it easy to identify your upload.
         meshctl debug report --upload true --upload-dir <your_name>/<issue_name>

Debug with the UI

The Gloo UI provides a single pane of glass from which you can assess what might be wrong in your environment.

  1. Open the Gloo UI. The Gloo UI is served from the gloo-mesh-ui service on port 8090. You can connect by using the meshctl or kubectl CLIs.

    For more information, see the CLI documentation.

      meshctl dashboard --kubecontext $MGMT_CONTEXT
    1. Port-forward the gloo-mesh-ui service on 8090.
        kubectl port-forward -n gloo-mesh --context $MGMT_CONTEXT svc/gloo-mesh-ui 8090:8090
    2. Open your browser and connect to http://localhost:8090.

  2. Review the following sections to help you scan your environment for errors or warnings.

Dashboard

The Gloo and Mesh cards provide an at-a-glance status of the health of the Gloo management server and agents, and the status of the health of your Istio control and data planes. These cards display different information depending on whether you have a single or multicluster setup.

Single cluster

Gloo management server and agent components are installed in multicluster setups only. For this reason, the Gloo section of the card is empty, even though your other Gloo components (such as the telemetry collector) continue to run.

The Mesh section of the card shows the version of the Istio installation, and the status of the Istio control and data plane. If pods are unavailable, a warning is displayed. You can hover over the warning icon to see more information, such as the reported Istio insights.

Figure: Gloo and Mesh cards for a single cluster setup in the Dashboard page
Figure: Gloo and Mesh cards for a single cluster setup in the Dashboard page

Multicluster

The Gloo section of the card shows the installed Gloo version in your multicluster setups, the status of the telemetry gateway and management server in your management cluster, and the status of the Gloo agent in your workload clusters. The cluster each component is deployed to is listed under the deployment name.

The Mesh section of the card shows the overall status of all Istio control and data planes across the clusters in your environment. If pods are unavailable, a warning is displayed. You can hover over the warning icon to see more information, such as the reported Istio insights. To review the Istio versions, hover over the clusters, and click a cluster name to review that cluster’s Istio information.

Figure: Gloo and Mesh cards for a multicluster page in the Dashboard page
Figure: Gloo and Mesh cards for a multicluster page in the Dashboard page

Inventory

The Inventory section provides an at-a-glance look at the health of registered clusters and discovered services that make up your Gloo environment.

Clusters

On the Clusters page, review basic details of each cluster that you registered with the Gloo management plane.

  1. To filter clusters by the cluster’s Gloo Mesh (OSS APIs) installation health, click the Healthy and Unhealthy buttons. You can also use the Sort by Name dropdown or the search bar to filter clusters by name.

    Figure: Clusters page
    Figure: Clusters page
    Figure: Clusters page
    Figure: Clusters page

  2. Click More Details to see a more detailed dashboard for the cluster. This dashboard can help you find errors in your Gloo setup.

Figure: Cluster details page
Figure: Cluster details page

Resources

The Resources section provides the configuration of each Solo, Istio, Gateway API, and Kubernetes resource in your environment to help you locate and resolve configuration issues. With the Filter by… dropdown, you can filter services by resource type.

To debug specific resources, click View YAML to view the resource’s YAML configuration.

Figure: Istio resources page
Figure: Istio resources page
Figure: Istio resources page
Figure: Istio resources page