Agents
The Gloo agent keeps track of changes to Gloo custom resources in the cluster. It reports these changes back to the management cluster.
-
Verify that the Gloo agent pod is running.
kubectl get pods -n gloo-mesh --context ${REMOTE_CONTEXT1}
If not, describe the pod and look for error messages.
kubectl describe pod -n gloo-mesh -l app=gloo-mesh-agent --context ${REMOTE_CONTEXT1}
-
Check the logs of the Gloo agent in your workload cluster.
kubectl logs --context ${REMOTE_CONTEXT1} -n gloo-mesh deploy/gloo-mesh-agent
Optionally, you can format the output with
jq
or save it in a local file so that you can read and analyze the output more easily.kubectl logs --context ${REMOTE_CONTEXT1} -n gloo-mesh deploy/gloo-mesh-agent | jq > agent-logs.json
-
In the logs, look for
"err"
,Err:
, orError
messages. For example, you might see a message similar to the following.Message Description Steps to resolve Err: connection error: desc = \"transport: Error while dialing dial tcp: missing address\"
The Gloo agent does not have the correct address set for the management server. In your Helm settings file for the agent Helm chart, compare the value for the serverAddress
setting with the IP address and port of the management server. If necessary, upgrade your agent installation with the correct address, such ashelm upgrade gloo-mesh-agent gloo-mesh-agent/gloo-mesh-agent --namespace gloo-mesh --kube-context=${REMOTE_CONTEXT} --version ${GLOO_VERSION} --set serverAddress=<mgmt_server_address>
."err": "RouteTable.networking.gloo.solo.io \"bookinfo\" not found",
Gloo expected to find a resource such as a route table named bookinfo
. You can check theresource
field to see which namespace the resource was expected in.If you recently deleted the resource, wait to see if the error resolves itself. If not, try debugging the resource. "err": "Operation cannot be fulfilled on virtualgateways.networking.gloo.solo.io \"istio-ingressgateway\": the object has been modified; please apply your changes to the latest version and try again
Gloo is trying to reconcile your changes to the resource, such as updating a virtual gateway to listen on a different port. If you recently updated the resource, wait to see if the error resolves itself. If not, try debugging the resource. -
If you see many error messages and are unsure of the timeline of events, you can try to reset the logs. To do so, you delete the agent pod. You cannot make changes to your Gloo resources while the pod is restarting.
kubectl delete pod -n gloo-mesh -l app=gloo-mesh-agent --context ${REMOTE_CONTEXT1}
-
Re-run the first command to get the logs of your new Gloo agent. Note that the agent might take a few minutes to produce logs.
-
If you continue to see error messages that indicate state reconciliation issues, try debugging Gloo resources.