Istio operator
Istio operators are used to install Istio, either as part of Gloo Platform's managed installation with Istio Lifecycle Manager or your own manual installation. If the Istio operator is configured incorrectly, the Gloo agent might not start or you might experience issues with other Istio components such as gateways.
Before reviewing these specific Istio operator topics, try Debugging Istio.
Conflicting IstioLifecycleManager errors
What's happening
- When you check the management logs, you see an Istio Lifecycle Manager error similar to the following:
failed to upsert snapshot for istio lifecycle manager","parent":"gloo-platform~gloo-mesh~cluster-1~admin.gloo.solo.io/v2, Kind=IstioLifecycleManager","err":"conflicting IOPs have been created from a different parent Istio Lifecycle Manager ...
- When you check the Istio Lifecycle Manager status, you see a conflicting message similar to the following:
kubectl get IstioLifecycleManager -A -o yaml
Example output:
status: clusters: cluster-1: installations: auto: message: 'Another conflicting IstioLifecycleManager has previously been used to install a IstioOperators in this cluster, please check on uninstall of : gm-iop-1-18-2.gloo-platform' observedRevision: 1-18-2 state: FAILED
Why it's happening
You might have a conflicting Istio Lifecycle Manager. For example, you might have uninstalled a previous Istio Lifecycle Manager that did not completely delete. This error can happen when the namespace is deleted before the Istio Lifecycle Manager is able to finish uninstalling.
How to fix it
- List the Istio Lifecycle Managers. If you have multiple in the same namespace for the same purpose, try uninstalling the one that you no longer need.
kubectl get IstioLifecycleManager -A
- If you already uninstalled the Istio Lifecycle Manager, try to manually replace the GatewayLifecycleManager CR.
Agent crashes with IstioOperator error
What's happening
Your Gloo agent does not start and is in a CrashLoopBackOff
state.
When you check the agent logs, you see an error similar to the following:
failed to list *v1alpha1.IstioOperator: unknown field \"target\" in v1alpha1.ResourceMetricSource
...
{"level":"error","ts":1678198470.502656,"logger":"controller.input-ConfigMap-cache","caller":"controller/controller.go:208","msg":"Could not wait for Cache to sync","error":"failed to wait for input-ConfigMap-cache caches to sync: timed out waiting for cache to be synced","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.2/pkg/internal/controller/controller.go:208\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.2/pkg/internal/controller/controller.go:234\nsigs.k8s.io/controller-runtime/pkg/manager.(*runnableGroup).reconcile.func1\n\t/go/pkg/mod/sigs.k8s.io/controller-runtime@v0.11.2/pkg/manager/runnable_group.go:218"}
Why it's happening
You might have an error in your Istio operator configuration, such as using a field that is deprecated or no longer supported.
By default, Gloo Platform expects the Istio ingress gateway to have the name istio-ingressgateway
. If you use a custom name for the ingress gateway, you cannot set up horizontal pod autoscaling (HPA) for the Istio gateways.
How to fix it
- Review your Istio operator configuration file for any deprecated or unsupported fields. For example, review the upstream Istio
operator.proto
file for unsupported fields in your version of Istio. - If you use a custom Istio ingress gateway name, remove the HPA section from your Istio operator gateway configuration file. The following example shows the
hpaSpec
in theistio-eastwestgateway
andistio-ingressgateway
sections.... components: ingressGateways: # Enable the default east-west gateway - name: <custom-name> # Deploy to the gloo-mesh-gateways namespace namespace: gloo-mesh-gateways enabled: true k8s: hpaSpec: maxReplicas: 5 metrics: - resource: name: cpu targetAverageUtilization: 60 type: Resource ... - name: <custom-name> # Deploy to the gloo-mesh-gateways namespace namespace: gloo-mesh-gateways enabled: true k8s: hpaSpec: maxReplicas: 5 metrics: - resource: name: cpu targetAverageUtilization: 60 type: Resource
- To add back similar HPA functionality, set autoscaling minimum and maximum values to the gateway configuration. The following example shows both the
istio-eastwestgateway
andistio-ingressgateway
sections. Note that thetargetAverageUtilization
field is also removed, because that field is deprecated in Istio 1.14 and later.... spec: values: gateways: istio-ingressgateway: autoscaleMin: 2 autoscaleMax: 5 ... istio-eastwestgateway: autoscaleMin: 2 autoscaleMax: 5
- Update your Istio operator configuration.