Skip to content
You are viewing the latest documentation for Solo Enterprise for kgateway. To access the documentation for older versions, use the version switcher.

Install in air-gapped environments

Page as Markdown
Install Solo Enterprise for kgateway in an air-gapped environment.
You can install Solo Enterprise for kgateway in an air-gapped environment, such as an on-premises datacenter, a cluster that runs on an intranet or private network only, or other disconnected environments, by mirroring container images and Helm charts to a private registry.
Before you begin

  1. Your environment must have the following components for an air-gapped installation.
    • Connected device: A machine that can reach the public internet and pull images and Helm charts from public registries.
    • Disconnected device: A machine in a private network or on-premises datacenter that can access the Kubernetes cluster where you install Solo Enterprise for kgateway.
    • Private registry: A registry that both devices can reach, such as Sonatype Nexus Repository or JFrog Artifactory. When setting up your private registry, consider the following:
      • Registry credentials in the cluster: You might need to configure image pull secrets in Kubernetes, such as a global image pull secret in OpenShift.
      • Allowed access to public registries: If your registry synchronizes content from upstream sources, allow access to the following common domains:
        • us-docker.pkg.dev/solo-public/enterprise-kgateway
        • gcr.io/gloo-mesh
        • docker.io
  2. The following CLIs must be available for this workflow. If your environment blocks public downloads, follow each CLI’s documentation for an air-gapped installation method.
    • On the disconnected device, install kubectl, the Kubernetes command line tool. Download the kubectl version that is within one minor version of the Kubernetes cluster you plan to use.
    • On the disconnected device, install helm, the Kubernetes package manager, to customize multiple settings in the Solo Enterprise for kgateway installation.
    • On the connected device, install either crane or skopeo to copy multi-architecture images from the public registries to your private registry.
Step 1: Mirror images from a connected device to your private registry

On a connected device that can reach both the public registries and your private registry, mirror the necessary images for Solo Enterprise for kgateway directly between registries. This workflow preserves multi-architecture image indexes for supported images. The following images are supported for version 2.2.0:
ComponentRegistryRepositoryTag
Controllerus-docker.pkg.dev/solo-public/enterprise-kgatewayenterprise-kgateway-controller2.2.0
Envoy wrapper (data plane)us-docker.pkg.dev/solo-public/enterprise-kgatewayenvoy-wrapper2.2.0
ExtAuthgcr.io/gloo-meshext-auth-service0.81.1
ExtCachedocker.iohredis7.2.13-alpine
RateLimitergcr.io/gloo-meshrate-limiter0.18.6
WAFus-docker.pkg.dev/solo-public/enterprise-kgatewaywaf-server2.2.0
Portal controllerus-docker.pkg.dev/solo-public/enterprise-kgatewayportal-controller2.2.0
Portal web serverus-docker.pkg.dev/solo-public/enterprise-kgatewayportal-web-server2.2.0
  1. Set the versions that you want to install.
    export KGATEWAY_VERSION=2.2.0
export EXTAUTH_VERSION=0.81.1
export EXTCACHE_VERSION=7.2.13-alpine
export RATELIMITER_VERSION=0.18.6
  2. Set the image locations.
    export KGATEWAY_CONTROLLER_IMG=us-docker.pkg.dev/solo-public/enterprise-kgateway/enterprise-kgateway-controller:$KGATEWAY_VERSION
export ENVOY_WRAPPER_IMG=us-docker.pkg.dev/solo-public/enterprise-kgateway/envoy-wrapper:$KGATEWAY_VERSION
export EXTAUTH_IMG=gcr.io/gloo-mesh/ext-auth-service:$EXTAUTH_VERSION
export EXTCACHE_IMG=docker.io/redis:$EXTCACHE_VERSION
export RATELIMITER_IMG=gcr.io/gloo-mesh/rate-limiter:$RATELIMITER_VERSION
   export WAF_IMG=us-docker.pkg.dev/solo-public/enterprise-kgateway/waf-server:$KGATEWAY_VERSION
   export PORTAL_CONTROLLER_IMG=us-docker.pkg.dev/solo-public/enterprise-kgateway/portal-controller:$KGATEWAY_VERSION
   export PORTAL_WEBSERVER_IMG=us-docker.pkg.dev/solo-public/enterprise-kgateway/portal-web-server:$KGATEWAY_VERSION
  3. Set your private registry details.
    export PRIVATE_REGISTRY=<registry.example.com>
export PRIVATE_REPOSITORY=<enterprise-kgateway>
  4. Set the image locations in your private registry.
    export KGATEWAY_CONTROLLER_PRIVATE=$PRIVATE_REGISTRY/$PRIVATE_REPOSITORY/enterprise-kgateway-controller:$KGATEWAY_VERSION
export ENVOY_WRAPPER_PRIVATE=$PRIVATE_REGISTRY/$PRIVATE_REPOSITORY/envoy-wrapper:$KGATEWAY_VERSION
export EXTAUTH_PRIVATE=$PRIVATE_REGISTRY/$PRIVATE_REPOSITORY/ext-auth-service:$EXTAUTH_VERSION
export EXTCACHE_PRIVATE=$PRIVATE_REGISTRY/$PRIVATE_REPOSITORY/redis:$EXTCACHE_VERSION
export RATELIMITER_PRIVATE=$PRIVATE_REGISTRY/$PRIVATE_REPOSITORY/rate-limiter:$RATELIMITER_VERSION
   export WAF_PRIVATE=$PRIVATE_REGISTRY/$PRIVATE_REPOSITORY/waf-server:$KGATEWAY_VERSION
   export PORTAL_CONTROLLER_PRIVATE=$PRIVATE_REGISTRY/$PRIVATE_REPOSITORY/portal-controller:$KGATEWAY_VERSION
   export PORTAL_WEBSERVER_PRIVATE=$PRIVATE_REGISTRY/$PRIVATE_REPOSITORY/portal-web-server:$KGATEWAY_VERSION
  5. Copy the images to your registry. If you do not want to use external auth, rate limiting, or Redis cache, the WAF server, or Portal, you can omit those images.
    skopeo commands:
    skopeo copy --all docker://$KGATEWAY_CONTROLLER_IMG docker://$KGATEWAY_CONTROLLER_PRIVATE
skopeo copy --all docker://$ENVOY_WRAPPER_IMG docker://$ENVOY_WRAPPER_PRIVATE
skopeo copy --all docker://$EXTAUTH_IMG docker://$EXTAUTH_PRIVATE
skopeo copy --all docker://$EXTCACHE_IMG docker://$EXTCACHE_PRIVATE
skopeo copy --all docker://$RATELIMITER_IMG docker://$RATELIMITER_PRIVATE
   skopeo copy --all docker://$WAF_IMG docker://$WAF_PRIVATE
   skopeo copy --all docker://$PORTAL_CONTROLLER_IMG docker://$PORTAL_CONTROLLER_PRIVATE
   skopeo copy --all docker://$PORTAL_WEBSERVER_IMG docker://$PORTAL_WEBSERVER_PRIVATE
    crane commands:
    crane copy $KGATEWAY_CONTROLLER_IMG $KGATEWAY_CONTROLLER_PRIVATE
crane copy $ENVOY_WRAPPER_IMG $ENVOY_WRAPPER_PRIVATE
crane copy $EXTAUTH_IMG $EXTAUTH_PRIVATE
crane copy $EXTCACHE_IMG $EXTCACHE_PRIVATE
crane copy $RATELIMITER_IMG $RATELIMITER_PRIVATE
   crane copy $WAF_IMG $WAF_PRIVATE
   crane copy $PORTAL_CONTROLLER_IMG $PORTAL_CONTROLLER_PRIVATE
   crane copy $PORTAL_WEBSERVER_IMG $PORTAL_WEBSERVER_PRIVATE
Step 2: Configure the disconnected cluster

  1. If you have not already, create the kgateway-system namespace in your Kubernetes cluster.
    kubectl create namespace kgateway-system
  2. If your private registry requires authentication, create an image pull secret.
    kubectl create secret docker-registry private-registry-credentials \
  --docker-server=$PRIVATE_REGISTRY \
  --docker-username=<username> \
  --docker-password=<password> \
  --docker-email=<email> \
  -n kgateway-system
  3. Create an EnterpriseKgatewayParameters resource that configures the container images to be used and points to the images in the private registry. You can optionally update the resource limits and requests for the shared extensions as shown here.
    kubectl apply -f- <<EOF
apiVersion: enterprisekgateway.solo.io/v1alpha1
kind: EnterpriseKgatewayParameters
metadata:
  name: shared-ent-kgateway-parameters
  namespace: kgateway-system
spec:
  kube:
    envoyContainer:
      image:
        pullPolicy: IfNotPresent
        registry: $PRIVATE_REGISTRY
        repository: $PRIVATE_REPOSITORY/envoy-wrapper
        tag: $KGATEWAY_VERSION
    sharedExtensions:
      extCache:
        enabled: true
        container:
          image:
            pullPolicy: IfNotPresent
            registry: $PRIVATE_REGISTRY
            repository: $PRIVATE_REPOSITORY/redis
            tag: $EXTCACHE_VERSION
      extauth:
        enabled: true
        container:
          image:
            pullPolicy: IfNotPresent
            registry: $PRIVATE_REGISTRY
            repository: $PRIVATE_REPOSITORY/ext-auth-service
            tag: $EXTAUTH_VERSION
      ratelimiter:
        enabled: true
        container:
          image:
            pullPolicy: IfNotPresent
            registry: $PRIVATE_REGISTRY
            repository: $PRIVATE_REPOSITORY/rate-limiter
            tag: $RATELIMITER_VERSION
         waf:
           enabled: true
           container:
             image:
               pullPolicy: IfNotPresent
               registry: $PRIVATE_REGISTRY
               repository: $PRIVATE_REPOSITORY/waf-server
               tag: $KGATEWAY_VERSION
EOF
Optional: Update resource limits and requests

The default shared extension deployments include reasonable resource allocations, as well as Prometheus metrics endpoints and health checks as described in the following table. If you notice performance issues with the external auth or rate limiter services, check the resource usage and adjust as necessary.
ServiceCPU/Memory RequestsCPU/Memory LimitsDefault Metrics Port
ext-auth-service100m CPU, 128Mi memory500m CPU, 256Mi memory9091
rate-limiter100m CPU, 128Mi memory500m CPU, 256Mi memory9091
ext-cache256m CPU, 512Mi memoryNot specified, as the resources depend on your usageN/A
Update the resource limits and requests as follows:
kubectl apply -f- <<EOF
apiVersion: enterprisekgateway.solo.io/v1alpha1
kind: EnterpriseKgatewayParameters
metadata:
  name: shared-ent-kgateway-parameters
  namespace: kgateway-system
spec:
  kube:
    envoyContainer:
      image:
        pullPolicy: IfNotPresent
        registry: $PRIVATE_REGISTRY
        repository: $PRIVATE_REPOSITORY/envoy-wrapper
        tag: $KGATEWAY_VERSION
    sharedExtensions:
      extCache:
        enabled: true
        container:
          image:
            pullPolicy: IfNotPresent
            registry: $PRIVATE_REGISTRY
            repository: $PRIVATE_REPOSITORY/redis
            tag: $EXTCACHE_VERSION
        resources:
          requests:
            cpu: 256m
            memory: 512Mi
          limits:
            cpu: 1000m
            memory: 2048Mi
      extauth:
        enabled: true
        container:
          image:
            pullPolicy: IfNotPresent
            registry: $PRIVATE_REGISTRY
            repository: $PRIVATE_REPOSITORY/ext-auth-service
            tag: $EXTAUTH_VERSION
        resources:
          requests:
            cpu: 200m
            memory: 256Mi
          limits:
            cpu: 750m
            memory: 512Mi
      ratelimiter:
        enabled: true
        container:
          image:
            pullPolicy: IfNotPresent
            registry: $PRIVATE_REGISTRY
            repository: $PRIVATE_REPOSITORY/rate-limiter
            tag: $RATELIMITER_VERSION
        resources:
          requests:
            cpu: 200m
            memory: 256Mi
          limits:
            cpu: 750m
            memory: 512Mi
      waf:
        enabled: true
        container:
          image:
            pullPolicy: IfNotPresent
            registry: $PRIVATE_REGISTRY
            repository: $PRIVATE_REPOSITORY/waf-server
            tag: $KGATEWAY_VERSION
        resources:
          requests:
            cpu: 200m
            memory: 256Mi
          limits:
            cpu: 750m
            memory: 512Mi
EOF
Step 3: Install Solo Enterprise for kgateway

  1. Create a Helm values file that references the private registry and the EnterpriseKgatewayParameters resource.
    cat > enterprise-kgateway/values.yaml <<EOF
image:
  registry: $PRIVATE_REGISTRY/$PRIVATE_REPOSITORY
  tag: "$KGATEWAY_VERSION"
  pullPolicy: IfNotPresent
gatewayClassParametersRefs:
  enterprise-kgateway:
    group: enterprisekgateway.solo.io
    kind: EnterpriseKgatewayParameters
    name: shared-ent-kgateway-parameters
    namespace: kgateway-system
EOF
  2. Follow the Helm installation guide and include the values file that you created. If your disconnected device cannot access public Helm registries, follow the steps in the next section.
Optional: Mirror Helm charts to a private registry

If your disconnected device cannot access public Helm registries, first mirror the charts from the connected device before installing with Helm.
  1. Pull the main and CRD Helm charts on the connected device.
    helm pull oci://us-docker.pkg.dev/solo-public/enterprise-kgateway/charts/enterprise-kgateway --version $KGATEWAY_VERSION
helm pull oci://us-docker.pkg.dev/solo-public/enterprise-kgateway/charts/enterprise-kgateway-crds --version $KGATEWAY_VERSION
  2. Push the charts to your private registry.
    helm push enterprise-kgateway-$KGATEWAY_VERSION.tgz oci://$PRIVATE_REGISTRY/$PRIVATE_REPOSITORY/helm
helm push enterprise-kgateway-crds-$KGATEWAY_VERSION.tgz oci://$PRIVATE_REGISTRY/$PRIVATE_REPOSITORY/helm
  3. Install the charts from the private registry.
    helm upgrade -i enterprise-kgateway-crds oci://$PRIVATE_REGISTRY/$PRIVATE_REPOSITORY/helm/enterprise-kgateway-crds \
  --version $KGATEWAY_VERSION \
  -n kgateway-system

helm upgrade -i enterprise-kgateway oci://$PRIVATE_REGISTRY/$PRIVATE_REPOSITORY/helm/enterprise-kgateway \
  --version $KGATEWAY_VERSION \
  -n kgateway-system \
  -f values.yaml
Optional: Install Portal from the private registry

If you use Solo Enterprise for kgateway Portal, mirror the Portal images and point the Portal installation at your private registry. Portal includes two images, both tagged with the Solo Enterprise for kgateway version ($KGATEWAY_VERSION): the portal-controller that runs in the control plane, and the portal-web-server that backs each portal. For more information about installing Portal, see the Portal setup guide.
ComponentRegistryRepositoryTag
Portal controllerus-docker.pkg.dev/solo-public/enterprise-kgatewayportal-controller2.2.0
Portal web serverus-docker.pkg.dev/solo-public/enterprise-kgatewayportal-web-server2.2.0
  1. On the connected device, set the Portal image locations. The Portal images use the same $KGATEWAY_VERSION tag that you set earlier.
    export PORTAL_CONTROLLER_IMG=us-docker.pkg.dev/solo-public/enterprise-kgateway/portal-controller:$KGATEWAY_VERSION
export PORTAL_WEB_SERVER_IMG=us-docker.pkg.dev/solo-public/enterprise-kgateway/portal-web-server:$KGATEWAY_VERSION
export PORTAL_CONTROLLER_PRIVATE=$PRIVATE_REGISTRY/$PRIVATE_REPOSITORY/portal-controller:$KGATEWAY_VERSION
export PORTAL_WEB_SERVER_PRIVATE=$PRIVATE_REGISTRY/$PRIVATE_REPOSITORY/portal-web-server:$KGATEWAY_VERSION
  2. Copy the Portal images to your private registry.
    skopeo commands:
    skopeo copy --all docker://$PORTAL_CONTROLLER_IMG docker://$PORTAL_CONTROLLER_PRIVATE
skopeo copy --all docker://$PORTAL_WEB_SERVER_IMG docker://$PORTAL_WEB_SERVER_PRIVATE
    crane commands:
    crane copy $PORTAL_CONTROLLER_IMG $PORTAL_CONTROLLER_PRIVATE
crane copy $PORTAL_WEB_SERVER_IMG $PORTAL_WEB_SERVER_PRIVATE
  3. If your disconnected device cannot reach public Helm registries, mirror the Portal Helm charts to your private registry, following the same steps as the main charts.
    helm pull oci://us-docker.pkg.dev/solo-public/enterprise-kgateway/charts/portal-crds --version $KGATEWAY_VERSION
helm pull oci://us-docker.pkg.dev/solo-public/enterprise-kgateway/charts/portal --version $KGATEWAY_VERSION
helm push portal-crds-$KGATEWAY_VERSION.tgz oci://$PRIVATE_REGISTRY/$PRIVATE_REPOSITORY/helm
helm push portal-$KGATEWAY_VERSION.tgz oci://$PRIVATE_REGISTRY/$PRIVATE_REPOSITORY/helm
  4. On the disconnected device, set your Portal license key and install the Portal controller from your private registry. The image.registry and image.tag values point the portal-controller image at your private registry.
    export PORTAL_LICENSE_KEY=<portal-license-key>

helm upgrade -i portal-crds oci://$PRIVATE_REGISTRY/$PRIVATE_REPOSITORY/helm/portal-crds \
  --version $KGATEWAY_VERSION \
  --namespace portal-system \
  --create-namespace

helm upgrade -i portal oci://$PRIVATE_REGISTRY/$PRIVATE_REPOSITORY/helm/portal \
  --version $KGATEWAY_VERSION \
  --namespace portal-system \
  --set licensing.licenseKey=$PORTAL_LICENSE_KEY \
  --set image.registry=$PRIVATE_REGISTRY/$PRIVATE_REPOSITORY \
  --set image.tag=$KGATEWAY_VERSION \
  --set imagePullSecrets[0].name=private-registry-credentials
  5. When you create a PortalParameters resource, override the portal web server image in the spec.webServer.container.image field so that each portal web server pulls the portal-web-server image from your private registry. Create the resource in the namespace where you create your portals, such as default.
    1. Copy the image pull secret for your private registry to the namespace of your portal web server, such as default.
      kubectl create secret docker-registry private-registry-credentials \
  --docker-server=$PRIVATE_REGISTRY \
  --docker-username=<username> \
  --docker-password=<password> \
  --docker-email=<email> \
  -n default
    2. Create the portal web server with a deploymentOverlay setting for the image server, which applies a strategic merge patch to the generated web server Deployment.
      kubectl apply -f- <<EOF
apiVersion: portal.solo.io/v1alpha1
kind: PortalParameters
metadata:
  name: portal-params
  namespace: default
spec:
  webServer:
    container:
      image:
        pullPolicy: IfNotPresent
        registry: $PRIVATE_REGISTRY
        repository: $PRIVATE_REPOSITORY/portal-web-server
        tag: $KGATEWAY_VERSION
    deploymentOverlay:
      spec:
        template:
          spec:
            imagePullSecrets:
            - name: private-registry-credentials
  store:
    memory: {}
EOF
Step 4: Verify images

After you complete the Helm installation, confirm that workloads pull the expected images and that no image pull errors occurred.
kubectl get deploy -n kgateway-system -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{range .spec.template.spec.containers[*]}{.image}{" "}{end}{"\n"}{end}'

kubectl get pods -n kgateway-system -o wide
kubectl get events -n kgateway-system --field-selector reason=Failed
Expected output:
enterprise-kgateway-controller  registry.example.com/enterprise-kgateway/enterprise-kgateway-controller:2.2.0
http  registry.example.com/enterprise-kgateway/envoy-wrapper:2.2.0
ext-auth-service-enterprise-kgateway  registry.example.com/enterprise-kgateway/ext-auth-service:0.81.1
ext-cache-enterprise-kgateway  registry.example.com/enterprise-kgateway/redis:7.2.13-alpine
rate-limiter-enterprise-kgateway  registry.example.com/enterprise-kgateway/rate-limiter:0.18.6
waf-server-enterprise-kgateway  registry.example.com/enterprise-kgateway/waf-server:2.2.0
Advanced settingsMigrate from kgateway OSS