Install in air-gapped environments

Install Solo Enterprise for kgateway in an air-gapped environment.

You can install Solo Enterprise for kgateway in an air-gapped environment, such as an on-premises datacenter, a cluster that runs on an intranet or private network only, or other disconnected environments, by mirroring container images and Helm charts to a private registry.

Before you begin

Your environment must have the following components for an air-gapped installation.
- Connected device: A machine that can reach the public internet and pull images and Helm charts from public registries.
- Disconnected device: A machine in a private network or on-premises datacenter that can access the Kubernetes cluster where you install Solo Enterprise for kgateway.
- Private registry: A registry that both devices can reach, such as Sonatype Nexus Repository or JFrog Artifactory. When setting up your private registry, consider the following:
  - Registry credentials in the cluster: You might need to configure image pull secrets in Kubernetes, such as a global image pull secret in OpenShift.
  - Allowed access to public registries: If your registry synchronizes content from upstream sources, allow access to the following common domains:
    - us-docker.pkg.dev/solo-public/enterprise-kgateway
    - gcr.io/gloo-mesh
    - docker.io
The following CLIs must be available for this workflow. If your environment blocks public downloads, follow each CLI’s documentation for an air-gapped installation method.
- On the disconnected device, install kubectl, the Kubernetes command line tool. Download the kubectl version that is within one minor version of the Kubernetes cluster you plan to use.
- On the disconnected device, install helm, the Kubernetes package manager, to customize multiple settings in the Solo Enterprise for kgateway installation.
- On the connected device, install either crane or skopeo to copy multi-architecture images from the public registries to your private registry.

Step 1: Mirror images from a connected device to your private registry

On a connected device that can reach both the public registries and your private registry, mirror the necessary images for Solo Enterprise for kgateway directly between registries. This workflow preserves multi-architecture image indexes for supported images. The following images are supported for version 2.1.5:

Component	Registry	Repository	Tag
Controller	us-docker.pkg.dev/solo-public/enterprise-kgateway	enterprise-kgateway-controller	2.1.5
ExtAuth	gcr.io/gloo-mesh	ext-auth-service	0.72.2
ExtCache	docker.io	redis	7.2.13-alpine
RateLimiter	gcr.io/gloo-mesh	rate-limiter	0.17.2

Set the versions that you want to install.

export KGATEWAY_VERSION=2.1.5
export EXTAUTH_VERSION=0.72.2
export EXTCACHE_VERSION=7.2.13-alpine
export RATELIMITER_VERSION=0.17.2

Set the image locations.

export KGATEWAY_CONTROLLER_IMG=us-docker.pkg.dev/solo-public/enterprise-kgateway/enterprise-kgateway-controller:$KGATEWAY_VERSION
export ENVOY_WRAPPER_IMG=us-docker.pkg.dev/solo-public/enterprise-kgateway/envoy-wrapper:$KGATEWAY_VERSION
export EXTAUTH_IMG=gcr.io/gloo-mesh/ext-auth-service:$EXTAUTH_VERSION
export EXTCACHE_IMG=docker.io/redis:$EXTCACHE_VERSION
export RATELIMITER_IMG=gcr.io/gloo-mesh/rate-limiter:$RATELIMITER_VERSION

Set your private registry details.

export PRIVATE_REGISTRY=<registry.example.com>
export PRIVATE_REPOSITORY=<enterprise-kgateway>

Set the image locations in your private registry.

export KGATEWAY_CONTROLLER_PRIVATE=$PRIVATE_REGISTRY/$PRIVATE_REPOSITORY/enterprise-kgateway-controller:$KGATEWAY_VERSION
export ENVOY_WRAPPER_PRIVATE=$PRIVATE_REGISTRY/$PRIVATE_REPOSITORY/envoy-wrapper:$KGATEWAY_VERSION
export EXTAUTH_PRIVATE=$PRIVATE_REGISTRY/$PRIVATE_REPOSITORY/ext-auth-service:$EXTAUTH_VERSION
export EXTCACHE_PRIVATE=$PRIVATE_REGISTRY/$PRIVATE_REPOSITORY/redis:$EXTCACHE_VERSION
export RATELIMITER_PRIVATE=$PRIVATE_REGISTRY/$PRIVATE_REPOSITORY/rate-limiter:$RATELIMITER_VERSION

Copy the images to your registry. If you do not want to use external auth, rate limiting, or Redis cache, you can omit those images.

skopeo commands:

skopeo copy --all docker://$KGATEWAY_CONTROLLER_IMG docker://$KGATEWAY_CONTROLLER_PRIVATE
skopeo copy --all docker://$ENVOY_WRAPPER_IMG docker://$ENVOY_WRAPPER_PRIVATE
skopeo copy --all docker://$EXTAUTH_IMG docker://$EXTAUTH_PRIVATE
skopeo copy --all docker://$EXTCACHE_IMG docker://$EXTCACHE_PRIVATE
skopeo copy --all docker://$RATELIMITER_IMG docker://$RATELIMITER_PRIVATE

crane commands:

crane copy $KGATEWAY_CONTROLLER_IMG $KGATEWAY_CONTROLLER_PRIVATE
crane copy $ENVOY_WRAPPER_IMG $ENVOY_WRAPPER_PRIVATE
crane copy $EXTAUTH_IMG $EXTAUTH_PRIVATE
crane copy $EXTCACHE_IMG $EXTCACHE_PRIVATE
crane copy $RATELIMITER_IMG $RATELIMITER_PRIVATE

Step 2: Configure the disconnected cluster

If you have not already, create the kgateway-system namespace in your Kubernetes cluster.
```
kubectl create namespace kgateway-system
```

If your private registry requires authentication, create an image pull secret.

kubectl create secret docker-registry private-registry-credentials \
  --docker-server=$PRIVATE_REGISTRY \
  --docker-username=<username> \
  --docker-password=<password> \
  --docker-email=<email> \
  -n kgateway-system

Create an EnterpriseKgatewayParameters resource that configures the container images to be used and points to the images in the private registry. You can optionally update the resource limits and requests for the shared extensions as shown here.

kubectl apply -f- <<EOF
apiVersion: enterprisekgateway.solo.io/v1alpha1
kind: EnterpriseKgatewayParameters
metadata:
  name: shared-ent-kgateway-parameters
  namespace: kgateway-system
spec:
  kube:
    envoyContainer:
      image:
        pullPolicy: IfNotPresent
        registry: $PRIVATE_REGISTRY
        repository: $PRIVATE_REPOSITORY/envoy-wrapper
        tag: $KGATEWAY_VERSION
    sharedExtensions:
      extCache:
        enabled: true
        container:
          image:
            pullPolicy: IfNotPresent
            registry: $PRIVATE_REGISTRY
            repository: $PRIVATE_REPOSITORY/redis
            tag: $EXTCACHE_VERSION
      extauth:
        enabled: true
        container:
          image:
            pullPolicy: IfNotPresent
            registry: $PRIVATE_REGISTRY
            repository: $PRIVATE_REPOSITORY/ext-auth-service
            tag: $EXTAUTH_VERSION
      ratelimiter:
        enabled: true
        container:
          image:
            pullPolicy: IfNotPresent
            registry: $PRIVATE_REGISTRY
            repository: $PRIVATE_REPOSITORY/rate-limiter
            tag: $RATELIMITER_VERSION
EOF

Optional: Update resource limits and requests

The default shared extension deployments include reasonable resource allocations, as well as Prometheus metrics endpoints and health checks as described in the following table. If you notice performance issues with the external auth or rate limiter services, check the resource usage and adjust as necessary.

Service	CPU/Memory Requests	CPU/Memory Limits	Default Metrics Port
ext-auth-service	100m CPU, 128Mi memory	500m CPU, 256Mi memory	9091
rate-limiter	100m CPU, 128Mi memory	500m CPU, 256Mi memory	9091
ext-cache	256m CPU, 512Mi memory	Not specified, as the resources depend on your usage	N/A

Update the resource limits and requests as follows:

kubectl apply -f- <<EOF
apiVersion: enterprisekgateway.solo.io/v1alpha1
kind: EnterpriseKgatewayParameters
metadata:
  name: shared-ent-kgateway-parameters
  namespace: kgateway-system
spec:
  kube:
    envoyContainer:
      image:
        pullPolicy: IfNotPresent
        registry: $PRIVATE_REGISTRY
        repository: $PRIVATE_REPOSITORY/envoy-wrapper
        tag: $KGATEWAY_VERSION
    sharedExtensions:
      extCache:
        enabled: true
        container:
          image:
            pullPolicy: IfNotPresent
            registry: $PRIVATE_REGISTRY
            repository: $PRIVATE_REPOSITORY/redis
            tag: $EXTCACHE_VERSION
        resources:
          requests:
            cpu: 256m
            memory: 512Mi
          limits:
            cpu: 1000m
            memory: 2048Mi
      extauth:
        enabled: true
        container:
          image:
            pullPolicy: IfNotPresent
            registry: $PRIVATE_REGISTRY
            repository: $PRIVATE_REPOSITORY/ext-auth-service
            tag: $EXTAUTH_VERSION
        resources:
          requests:
            cpu: 200m
            memory: 256Mi
          limits:
            cpu: 750m
            memory: 512Mi
      ratelimiter:
        enabled: true
        container:
          image:
            pullPolicy: IfNotPresent
            registry: $PRIVATE_REGISTRY
            repository: $PRIVATE_REPOSITORY/rate-limiter
            tag: $RATELIMITER_VERSION
        resources:
          requests:
            cpu: 200m
            memory: 256Mi
          limits:
            cpu: 750m
            memory: 512Mi
EOF

Step 3: Install Solo Enterprise for kgateway

Create a Helm values file that references the private registry and the EnterpriseKgatewayParameters resource.

cat > enterprise-kgateway/values.yaml <<EOF
image:
  registry: $PRIVATE_REGISTRY/$PRIVATE_REPOSITORY
  tag: "$KGATEWAY_VERSION"
  pullPolicy: IfNotPresent
gatewayClassParametersRefs:
  enterprise-kgateway:
    group: enterprisekgateway.solo.io
    kind: EnterpriseKgatewayParameters
    name: shared-ent-kgateway-parameters
    namespace: kgateway-system
EOF

Follow the Helm installation guide and include the values file that you created. If your disconnected device cannot access public Helm registries, follow the steps in the next section.

Optional: Mirror Helm charts to a private registry

If your disconnected device cannot access public Helm registries, first mirror the charts from the connected device before installing with Helm.

Pull the main and CRD Helm charts on the connected device.

helm pull oci://us-docker.pkg.dev/solo-public/enterprise-kgateway/charts/enterprise-kgateway --version $KGATEWAY_VERSION
helm pull oci://us-docker.pkg.dev/solo-public/enterprise-kgateway/charts/enterprise-kgateway-crds --version $KGATEWAY_VERSION

Push the charts to your private registry.

helm push enterprise-kgateway-$KGATEWAY_VERSION.tgz oci://$PRIVATE_REGISTRY/$PRIVATE_REPOSITORY/helm
helm push enterprise-kgateway-crds-$KGATEWAY_VERSION.tgz oci://$PRIVATE_REGISTRY/$PRIVATE_REPOSITORY/helm

Install the charts from the private registry.

helm upgrade -i enterprise-kgateway-crds oci://$PRIVATE_REGISTRY/$PRIVATE_REPOSITORY/helm/enterprise-kgateway-crds \
  --version $KGATEWAY_VERSION \
  -n kgateway-system

helm upgrade -i enterprise-kgateway oci://$PRIVATE_REGISTRY/$PRIVATE_REPOSITORY/helm/enterprise-kgateway \
  --version $KGATEWAY_VERSION \
  -n kgateway-system \
  -f values.yaml

Step 4: Verify images

After you complete the Helm installation, confirm that workloads pull the expected images and that no image pull errors occurred.

kubectl get deploy -n kgateway-system -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{range .spec.template.spec.containers[*]}{.image}{" "}{end}{"\n"}{end}'

kubectl get pods -n kgateway-system -o wide
kubectl get events -n kgateway-system --field-selector reason=Failed

Expected output:

enterprise-kgateway-controller  registry.example.com/enterprise-kgateway/enterprise-kgateway-controller:2.1.5
http  registry.example.com/enterprise-kgateway/envoy-wrapper:2.1.5
ext-auth-service-enterprise-kgateway  registry.example.com/enterprise-kgateway/ext-auth-service:0.72.2
ext-cache-enterprise-kgateway  registry.example.com/enterprise-kgateway/redis:7.2.13-alpine
rate-limiter-enterprise-kgateway  registry.example.com/enterprise-kgateway/rate-limiter:0.17.2

The data plane pods, including the `http` gateway proxy, `ext-auth-service`, `rate-limiter`, and `ext-cache` are created only after you deploy a Gateway resource in your environment. For more information, see the [Get started](../../quickstart) guide.

If you change features or image variants, be sure to mirror those images before you upgrade Solo Enterprise for kgateway.

Advanced settings