The Gloo agent keeps track of changes to Gloo custom resources in the cluster. It reports these changes back to the management server.
Verify that the Gloo agent pod is running.
kubectl get pods -n gloo-mesh
If not, describe the pod and look for error messages.
kubectl describe pod -n gloo-mesh -l app=gloo-mesh-agent
Check the logs of the Gloo agent in your workload cluster. To view logs recorded since a relative duration such as 5s, 2m, or 3h, you can specify the
meshctl logs agent -l error [--since DURATION]
Optionally, you can format the output with
jqor save it in a local file so that you can read and analyze the output more easily.
meshctl logs agent -l error > agent-logs.json
In the logs, look for
"err"messages. For example, you might see a message similar to the following.
Message Description Steps to resolve
"err": "RouteTable.networking.gloo.solo.io \"bookinfo\" not found",
Gloo expected to find a resource such as a route table named
bookinfo. You can check the
resourcefield to see which namespace the resource was expected in.
If you recently deleted the resource, wait to see if the error resolves itself. If not, try debugging the resource.
"err": "Operation cannot be fulfilled on virtualgateways.networking.gloo.solo.io \"istio-ingressgateway\": the object has been modified; please apply your changes to the latest version and try again
Gloo is trying to reconcile your changes to the resource, such as updating a virtual gateway to listen on a different port. If you recently updated the resource, wait to see if the error resolves itself. If not, try debugging the resource.
Waited for <time> due to client-side throttling, not priority and fairness, request
Gloo experienced a timeout when sending a request to the Kubernetes API server. For example, the Kubernetes etcd might be overloaded by the number of resources in the cluster. Wait to see if the error resolves as your Kubernetes cluster load reduces.
You can also check the logs for other all other log levels, such as
meshctl logs agent [--since DURATION]
If you continue to see error messages that indicate state reconciliation issues, try debugging Gloo resources.