Resource validation
Prevent invalid configuration of Gloo custom resources (CRs) from being applied to your cluster.
By checking your Gloo CRs against defined configuration constraints and validation rules, any attempt to apply invalid configuration to your cluster is rejected.
About
In Gloo Mesh Enterprise 2.6 and later, the fields of select Gloo CRDs contain constraints and rules to ensure that only valid configuration is admitted to your Gloo Mesh Enterprise setup.
- Schema constraints: OpenAPI schema validation includes simple constraints for single fields, such as marking a field as required or setting a minimum and maximum for a value. These rules ensure syntactic validity.
- Common Expression Language (CEL) rules: In Kubernetes 1.25 and later, CEL rules provide more complex validation that can apply to multiple fields within a CRD, such as requiring a specific value for one field if a different field is also set.
Resource validation is automatically enabled for the following Gloo CRDs:
- ExtAuthPolicy
- FaultInjectionPolicy
- LoadBalancerPolicy
- OutlierDetectionPolicy
- RetryTimeoutPolicy
- RouteTable, including PortalMetadata
- VirtualDestination
- VirtualGateway
To see the constraints and rules for each field, check the API reference docs for each custom resource and look for the Configuration constraints. You can also run kubectl explain <CR> to see the rules for each field. You can check the top-level fields of the entire resource, such as kubectl explain RouteTable, or describe specific sections, such as kubectl explain RouteTable.status or kubectl explain RouteTable.spec.http.
Resource validation in Gloo Mesh Enterprise is performed automatically when you apply CRs, and can optionally be performed manually before you attempt to apply CRs.
Automatic admission validation
The resource validation feature is enabled by default when you install Gloo Mesh Enterprise at or upgrade your existing installation to version 2.6.0 or later. When you attempt to apply your Gloo CRs, such as by using kubectl apply, the CRs are automatically checked for validity. If the resource contains invalid schema or settings, the resource application is automatically rejected, and an error message is returned.
For example, you might try to apply the following invalid RetryTimeoutPolicy (attempts: -1 is invalid).
kubectl apply -f - << EOF
apiVersion: resilience.policy.gloo.solo.io/v2
kind: RetryTimeoutPolicy
metadata:
name: test
namespace: default
spec:
applyToRoutes:
- route:
labels:
app: test
config:
retries:
attempts: -1
EOF
The invalid resource is rejected, and the following error message is returned.
The RetryTimeoutPolicy "test" is invalid: spec.config.retries.attempts: Invalid value: -1: spec.config.retries.attempts in body should be greater than or equal to 0
Automatic admission validation is enabled by default in version 2.6 and later. If your resources contain invalid configuration, they are rejected. This can interrupt automated processes that apply CRs to your environment, such as GitOps pipelines. To disable this feature, upgrade your Gloo CRDs Helm chart to include the --set disableAdmissionValidation=true flag.
helm upgrade -i gloo-platform-crds gloo-platform/gloo-platform-crds \
--namespace=gloo-mesh \
--version=$GLOO_VERSION \
--set disableAdmissionValidation=true
Pre-admission validation with meshctl
Before you apply your Gloo CRs, you can check the validity of your configuration by using the meshctl x validate resources command. This method of resource validation is optional, but can be an important step in ensuring that you can later apply your resources smoothly to your Gloo Mesh Enterprise environment. For example, you can incorporate this CLI command into your GitOps workflow as a way to reliably to check your YAML files during the development phase.
Because this command is interactive, you can choose to skip any of the following stages of the check. However, running all stages by pressing Enter is recommended.
- Schema Validation: Validates the schema of individual resources according to defined rules and constraints.
- Load Input & Output Resources: Loads input and output resources, either from local snapshot files or from a management cluster.
- Detect Changes to Input Resources: Indicates which resources have changed from the given input snapshot.
- Translate Input Resources: Translates the input resources into output resources, and displays any errors or warnings.
- Output Resource Diff: Shows the difference between the output Istio and Envoy resources and the latest output snapshot.
- Input Resource Diff: Shows the difference between the input Gloo resources and the latest input snapshot.
- Envoy Config Verification: Validates resulting Envoy config, and displays any errors or warnings.
The meshctl x validate resources command can be run in online mode, in which you apply these checks to a resource against the context of your Gloo management cluster, or offline mode, in which you locally apply these checks to resource files.
For more information, see the CLI reference.
Online mode
In online mode, the command not only checks the validity of a provided CR, but also shows the anticipated changes to the input and output snapshots of translated resources in your cluster. This can help you determine the impact of resources on the rest of your Gloo environment before you apply them.
For example, say that you have the following RetryTimeoutPolicy resources in a file named retry-policy.yaml, and you want to check both the policies’ validity and the expected changes to your environment after you apply the policies.
apiVersion: resilience.policy.gloo.solo.io/v2
kind: RetryTimeoutPolicy
metadata:
name: test1
namespace: default
spec:
applyToRoutes:
- route:
labels:
app: test
config:
retries:
attempts: 1
---
apiVersion: resilience.policy.gloo.solo.io/v2
kind: RetryTimeoutPolicy
metadata:
name: test2
namespace: default
spec:
applyToRoutes:
- route:
labels:
app: test
config:
retries:
attempts: -1
To do so, you can run the following command, which uses the context of your management cluster.
meshctl x validate resources retry-policy.yaml --kubecontext $MGMT_CONTEXT
In the following example output, a validity error is reported for the attempts: -1 value.
# Starting validation phases...
ERROR Schema Validation
Fail: 1 error occurred:
* spec.config.retries.attempts: Invalid value: -1: spec.config.retries.attempts in body should be greater than or equal to 0
Saved logging output to /var/folders/b5/9rwm6sld7w3bw7q0wnnd0ym40000gn/T/validate-logs/resource-validate.log
ERROR 1 error occurred:
* spec.config.retries.attempts: Invalid value: -1: spec.config.retries.attempts in body should be greater than or equal to 0
After you fix the error, such as by changing -1 to 1, you run the validation command again:
meshctl x validate resources retry-policy.yaml --kubecontext $MGMT_CONTEXT
In this example output, the retry policies are valid, so no errors are reported for those resources. Additionally, you can review summaries of the anticipated changes to the input resources, the translated output resources, and the errors and warnings for the other existing resources in your cluster. For example, note that there are warnings reported for a Service and a VirtualDestination, and an error reported for an ExtAuthPolicy. You can check these by opening the /var/folders/b5/9rwm6sld7w3bw7q0wnnd0ym40000gn/T/validate-logs/input-resource-errors.json file.
# Starting validation phases...
SUCCESS Schema Validation
SUCCESS Load Input & Output Resources
loading snapshots from management server
WARNING Version Check
meshctl version: 2.6.0
gloo mesh version: 2.6.0
SUCCESS Translate Input Resources
SUCCESS Input Resource Diff
Diff file saved to /var/folders/b5/9rwm6sld7w3bw7q0wnnd0ym40000gn/T/validate-logs/input-snap.diff
SUCCESS Output Resource Diff
Diff file saved to /var/folders/b5/9rwm6sld7w3bw7q0wnnd0ym40000gn/T/validate-logs/output-snap.diff
SUCCESS Detect Changes to Input & Output Resources
# Summaries
Changes to Input Resources
┌─────────────────────────────────────────────┐
| Cluster | Kind | Added | Modified |
| ---------------------------------------------------- |
| mgmt-cluster | RetryTimeoutPolicy | 2 | 0 |
| ---------------------------------------------------- |
└─────────────────────────────────────────────┘
Saved input resource diffs to /var/folders/b5/9rwm6sld7w3bw7q0wnnd0ym40000gn/T/validate-logs/input-snap.diff
Changes to Output Resources
┌──────────────────────────────────────────────┐
| Cluster | Kind | Added | Modified | Removed |
| ----------------------------------------------------- |
| cluster-1 | ServiceEntry | 75 | 0 | 0 |
| ----------------------------------------------------- |
| cluster-1 | Gateway | 1 | 0 | 0 |
| ----------------------------------------------------- |
| cluster-1 | EnvoyFilter | 75 | 0 | 0 |
| ----------------------------------------------------- |
└──────────────────────────────────────────────┘
Saved output resource diffs to /var/folders/b5/9rwm6sld7w3bw7q0wnnd0ym40000gn/T/validate-logs/output-snap.diff
Resource Errors & Warnings
┌─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
| Cluster | Kind | Ns/Name | Errors | Warnings |
| --------------------------------------------------------------------------------------------------------------------------------------------- |
| cluster-1 | Service | workload-00019-cluster-1/workload-00019 | 0 | 25 |
| --------------------------------------------------------------------------------------------------------------------------------------------- |
| cluster-1 | VirtualDestination | workload-00014-cluster-1/workload-00014-portal-cluster-1-692abccf4324f8554e3a4dc66077734 | 0 | 25 |
| --------------------------------------------------------------------------------------------------------------------------------------------- |
| cluster-1 | ExtAuthPolicy | gloo-mesh/workload-00014-portal-rt-delegate-api | 1 | 0 |
| --------------------------------------------------------------------------------------------------------------------------------------------- |
└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
Saved to /var/folders/b5/9rwm6sld7w3bw7q0wnnd0ym40000gn/T/validate-logs/input-resource-errors.json
Saved logging output to /var/folders/b5/9rwm6sld7w3bw7q0wnnd0ym40000gn/T/validate-logs/resource-validate.log
Need to only check the existing, applied Gloo CRs in your cluster, without checking any new resources? Run the command without passing in a file, such as meshctl x validate resources --context $MGMT_CONTEXT.
Offline mode
In offline mode, you provide downloaded snapshots of the input Gloo resources and output Istio resources from your cluster. The command checks the validity of a CR locally, and shows the anticipated changes to these snapshots. Offline mode can be helpful when you want to locally simulate changes to your cluster environment.
For example, you get the current input and output snapshots from your cluster by running the following commands:
kubectl -n gloo-mesh port-forward deploy/gloo-mesh-mgmt-server 9091
curl "http://localhost:9091/snapshots/input" > input.json
curl "http://localhost:9091/snapshots/output" > output.json
Then, provide the downloaded snapshots in the --input-snapshot and --output-snapshot flags of the meshctl x validate resources command to run the command in offline mode as shown in the following example. The command provides summaries of the anticipated changes to the input resources, the translated output resources, and the errors and warnings for the other existing resources in your snapshot.
meshctl x validate resources retry-policy.yaml --input-snapshot input.json --output-snapshot output.json