Customization options

Review the built-in and overlay customization options available for gateway proxies in kgateway.

Choose between the following options to customize the configuration of gateway proxies and shared extensions in Solo Enterprise for kgateway.

Option	Description
Built-in configuration (recommended)	Use the built-in fields in the EnterpriseKgatewayParameters resource. These fields are validated when you apply the resource.
Overlays	Use the strategic merge patch options in the EnterpriseKgatewayParameters resource to modify the raw Kubernetes resources that the control plane generates, or to create additional resources such as Horizontal Pod Autoscalers, Vertical Pod Autoscalers, and Pod Disruption Budgets. Use this option for advanced customizations that are not covered by the built-in fields.

Built-in customization (recommended)

The EnterpriseKgatewayParameters resource comes with built-in customization options that are automatically validated when you apply the resource.

Use the built-in fields where possible. Built-in fields are validated at apply time, are considered stable, and are updated automatically when you upgrade your gateway proxies and shared extensions.

Gateway proxy

Review the built-in configurations for the gateway proxy in the spec.kube section of the EnterpriseKgatewayParameters resource. For a step-by-step guide, see Change proxy config.

Field	Description
`deployment`	Set the number of replicas and the update strategy for the proxy Deployment. If you use an HPA, do not set `replicas` here.
`envoyContainer`	Configure the Envoy container, including the image, bootstrap log level, component log levels, environment variables, resource requests and limits, and security context.
`sdsContainer`	Configure the Secret Discovery Service (SDS) sidecar container.
`podTemplate`	Configure pod-level settings, including image pull secrets, labels, annotations, node selector, affinity, tolerations, topology spread constraints, and the pod security context.
`service`	Configure the Kubernetes Service that exposes the proxy, including the type, ports, labels, annotations, and external traffic policy.
`serviceAccount`	Configure the ServiceAccount for the proxy pods.
`istio`	Configure the Istio integration for the proxy.
`stats`	Configure the stats server that exposes Prometheus metrics.

Shared extensions

Review the built-in configurations for shared extensions in the spec.kube.sharedExtensions section of the EnterpriseKgatewayParameters resource. Each extension (extauth, ratelimiter, extCache) supports the following fields unless otherwise noted. For a step-by-step guide, see Change shared extensions.

Field	Description
`enabled`	Enable or disable the extension.
`replicas`	Set the number of replicas for the extension Deployment. Do not set this field when using an HPA.
`resources`	Set CPU and memory requests and limits for the extension container.
`container`	Configure the extension container, including the image and security context.
`pod`	Configure pod-level settings, including tolerations, node affinity, topology spread constraints, and the termination grace period.
`strategy`	Configure the update strategy for the extension Deployment. Available for the `ratelimiter` extension only.

Overlays

For advanced customization that is not covered by the built-in fields, the EnterpriseKgatewayParameters resource supports overlays. Overlays use Kubernetes strategic merge patch (SMP) semantics to modify the raw Kubernetes resources after the control plane renders them. You can also use overlays to create HorizontalPodAutoscaler (HPA), VerticalPodAutoscaler (VPA), and PodDisruptionBudget (PDB) resources.

The same overlay fields apply to both the gateway proxy and shared extensions, but are configured in different sections of the EnterpriseKgatewayParameters resource:

Gateway proxy: Configure overlay fields in spec.kube. For a step-by-step guide, see Change proxy config.
Shared extensions: Configure overlay fields in spec.kube.sharedExtensions.<extension>, where <extension> is extauth, ratelimiter, or extCache. For a step-by-step guide, see Change shared extensions.

Field	Resource type	Description
`deploymentOverlay`	Deployment	Modify the Deployment after it is rendered. Common use cases include adding init containers or sidecars, configuring node scheduling settings, adding custom labels and annotations, and removing default security contexts.
`serviceOverlay`	Service	Modify the Service. A common use case is adding cloud provider-specific annotations.
`serviceAccountOverlay`	ServiceAccount	Modify the ServiceAccount.
`horizontalPodAutoscaler`	HorizontalPodAutoscaler	Create an HPA targeting the Deployment. The HPA is created only when this field is present.
`verticalPodAutoscaler`	VerticalPodAutoscaler	Create a VPA targeting the Deployment. The VPA is created only when this field is present. Requires the VPA controller. Available for the gateway proxy and the `ratelimiter` extension only.
`podDisruptionBudget`	PodDisruptionBudget	Create a PDB targeting the Deployment. The PDB is created only when this field is present.

Overlays are not validated by the Solo Enterprise for kgateway control plane at apply time. Configuration errors surface only when Kubernetes processes the resulting resource. The overlay schema reflects the underlying Kubernetes resource schema and is not stable between Kubernetes versions. Test overlay configurations after each cluster upgrade.

How overlays work

Overlays are applied after the control plane renders the base Kubernetes resources. The control plane runs through the following steps:

The control plane reads built-in configuration from the EnterpriseKgatewayParameters resource, such as kube.deployment, kube.service, and kube.podTemplate.
The control plane generates the base resources for the gateway proxy, including the Deployment, Service, and ServiceAccount.
The control plane applies any overlays that you specified in the EnterpriseKgatewayParameters resource.
The control plane creates or updates the resources in the cluster.

Remove or replace config

You can use overlays to remove configuration from the generated resources, such as the pod security context when working in OpenShift environments. The strategic merge patch supports the following methods:

Set field value to null

Set a field to a null value to remove it. You must use kubectl apply --server-side to apply the change. Without --server-side, the null value is silently dropped.

The following example removes the container-level securityContext:


spec:
  kube:
    deploymentOverlay:
      spec:
        template:
          spec:
            containers:
              - name: kgateway
                # Removes the container-level securityContext
                securityContext: null

Remove an entire field

To remove an entire field, use $patch: delete instead.

The following example removes the pod-level securityContext:


spec:
  kube:
    deploymentOverlay:
      spec:
        template:
          spec:
            # Removes the pod-level securityContext
            securityContext:
              $patch: delete

To replace a list rather than merging with it, add $patch: replace as a separate list item before your actual items:


spec:
  kube:
    deploymentOverlay:
      spec:
        template:
          spec:
            volumes:
              - $patch: replace
              - name: custom-config
                configMap:
                  name: my-custom-config

Configuration priority and precedence

You can attach a EnterpriseKgatewayParameters resource to a GatewayClass that is shared by all Gateways that use that class or to an individual Gateway. When resources are attached at both levels, they are processed in the following order:

Built-in configuration on the GatewayClass is applied first – Built-in fields such as kube.deployment, kube.service, and kube.podTemplate from the GatewayClass EnterpriseKgatewayParameters are applied first.
Built-in configuration on the Gateway overrides the GatewayClass – If the same built-in field is set on both the Gateway and the GatewayClass, the Gateway value takes precedence.
Overlay configuration on the GatewayClass is applied – After all built-in configuration is processed, the GatewayClass overlay fields are applied to the rendered resources.
Overlay configuration on the Gateway overrides the GatewayClass – If conflicting overlay configuration is specified on the Gateway, the configuration in the GatewayClass is overridden by using strategic merge patch semantics. Consider the following examples:
- For scalar values, such as replicas, the Gateway configuration takes precedence.
- For maps, such as labels, the label keys are merged. If both the Gateway and GatewayClass specify the same label key, the label key on the Gateway takes precedence.

Example

Consider the following GatewayClass configuration:


spec:
  kube:
    deploymentOverlay:
      metadata:
        labels:
          level: gc
          gc-only-label: from-gc

Consider the following Gateway configuration:


spec:
  kube:
    deploymentOverlay:
      metadata:
        labels:
          level: gw
          gw-only-label: from-gw

The resulting configuration merges both configurations as follows:


metadata:
  labels:
    level: gw          # Gateway wins on conflicting key
    gc-only-label: from-gc   # GatewayClass key preserved
    gw-only-label: from-gw   # Gateway key added

Change proxy config

Was this page helpful?