Seamlessly migrate your kgateway proxies to the Gloo Gateway control plane without downtime so that you can start using enterprise features, such as JWT authentication, staged transformations, advanced RBAC for MCP and LLMs, retries, and timeouts.

Migration approach

Gloo Gateway builds on the open-source kgateway project to provide enterprise functionality for API gateway and AI use cases. To migrate your kgateway proxies to the Gloo Gateway control plane, the following steps are required:

  1. Canary deployment for the control plane: You deploy the Gloo Gateway control plane alongside the kgateway control plane. Both control planes are running at the same time while you perform the migration of the gateway proxies. This approach ensures clear separation of responsibilities between control planes so that your gateway proxies can be migrated without disruption.
  2. In-place upgrades for gateway proxies: Update all existing kgateway proxies to use the new Gloo Gateway GatewayClass (gloo-gateway-v2 or agentgateway-enterprise). The Gloo Gateway control plane automatically takes over management of these proxies and gracefully shuts down the old proxy while deploying the new one. Traffic continues to be served by the old proxy until the new proxy becomes available. When the new proxy is available, traffic is automatically routed through the new proxy. This approach keeps the existing configuration for your Gateways, routes, and policies and ensures minimal resource overhead during the migration.

Browse through the following tabs to learn more about each step during the migration:

In the initial state, you have an existing kgateway installation in your cluster and one or more kgateway proxies that serve the traffic for your apps. The proxy is managed by the kgateway control plane as shown in the following image:

In the next step, you install the Gloo Gateway control plane alongside kgateway. The Gloo Gateway CRDs are a superset of the kgateway CRDs. Because of that, you must install the Gloo Gateway CRDs in the same namespace and with the same Helm installation name as your kgateway CRDs to avoid Helm CRD conflicts. However, you can install the Gloo Gateway control plane in the namespace of your choice.

When you are ready to migrate to Gloo Gateway, you simply switch out the GatewayClass in your kgateway proxies to point to Gloo Gateway. For example, let’s say you have a gateway proxy that currently uses the kgateway GatewayClass. To migrate this proxy to Gloo Gateway, you change the GatewayClass to the equivalent class gloo-gateway-v2. Similarly, the agentgateway GatewayClass is changed to agentgateway-enterprise.

After switching the GatewayClass, the Gloo Gateway control plane takes over the management of that gateway proxy and rolls out a new version of the proxy while gracefully shutting down the old one. This way, requests are always served by one proxy ensuring zero downtime for your apps.

When all proxies are migrated and served by the Gloo Gateway control plane, you can remove the kgateway control plane from your cluster.

Migrate to Gloo Gateway

Perform a sample migration of kgateway or agentgateway proxies to Gloo Gateway.

Step 1: Initial state

To demonstrate the zero downtime migration from kgateway, you start by installing kgateway in your cluster.

  1. Install kgateway.
  2. Deploy the sample httpbin app.

Step 2: Install Gloo Gateway

  1. Set your Gloo Gateway license key as an environment variable. If you do not have one, contact an account representative. If you want to use the capabilities of Solo Enterprise for agentgateway, you need an additional Solo Enterprise for agentgateway license.

      export GLOO_GATEWAY_LICENSE_KEY=<gloo-gateway-license-key>
export AGENTGATEWAY_LICENSE_KEY=<agentgateway-license-key>

  2. Install the Gloo Gateway CRDs. The following command uses the latest stable release, 2.0.2. For active development, update the version to 2.0.0-main. Because the CRDs are a superset of the kgateway CRDs, you must install them in the kgateway-system namespace and name the Helm installation kgateway-crds. This way, you avoid CRD conflicts during the installation.

      helm upgrade -i kgateway-crds oci://us-docker.pkg.dev/solo-public/gloo-gateway/charts/gloo-gateway-crds \
--create-namespace \
--namespace kgateway-system \
--version 2.0.2

  3. Install Gloo Gateway by using Helm. For the gateways in your data plane, choose between the Envoy-based kgateway proxy or the AI-first agentgateway proxy. You can also enable both gateway proxy types. Note that you need a separate license for each gateway proxy type that you want to enable. For more information, see Gateway proxies.

      helm upgrade -i gloo-gateway oci://us-docker.pkg.dev/solo-public/gloo-gateway/charts/gloo-gateway \
  -n gloo-system \
  --version 2.0.2 \
  --set licensing.glooGatewayLicenseKey=$GLOO_GATEWAY_LICENSE_KEY
  
      helm upgrade -i gloo-gateway oci://us-docker.pkg.dev/solo-public/gloo-gateway/charts/gloo-gateway \
  -n gloo-system \
  --version 2.0.2 \
  --set agentgateway.enabled=true \
  --set licensing.agentgatewayLicenseKey=$AGENTGATEWAY_LICENSE_KEY 
  
      helm upgrade -i gloo-gateway oci://us-docker.pkg.dev/solo-public/gloo-gateway/charts/gloo-gateway \
  -n gloo-system \
  --version 2.0.2 \
  --set agentgateway.enabled=true \
  --set licensing.glooGatewayLicenseKey=$GLOO_GATEWAY_LICENSE_KEY \
  --set licensing.agentgatewayLicenseKey=$AGENTGATEWAY_LICENSE_KEY

  4. Make sure that the gloo-gateway control plane is running.

      kubectl get pods -n gloo-system

    Example output:

      NAME                            READY   STATUS    RESTARTS   AGE
gloo-gateway-5495d98459-46dpk   1/1     Running   0          19s

Step 3: Migrate proxies

  1. Create a curl pod that continuously sends requests to the httpbin app. If a request fails and does not return a 200 HTTP response code, the message ALERT: dropped request, got HTTP $status is logged. You use this curl pod to verify that the migration of your gateway proxies was finished without any disruption for your app.

      kubectl run curl-test -it --rm --image=curlimages/curl --restart=Never -- \
  sh -c '
  while true; do
    status=$(curl -s -o /dev/null -w "%{http_code}" http://http.kgateway-system.svc.cluster.local:8080/headers -H "host: www.example.com")
    if [ "$status" != "200" ]; then
      echo "ALERT: dropped request, got HTTP $status"
    fi
    sleep 0.25
  done
'

    Example output:

      All commands and output from this session will be recorded in container logs, including credentials and sensitive information passed through the command prompt.
If you don't see a command prompt, try pressing enter.

  2. Update the GatewayClass for the http Gateway to point to the Gloo Gateway control plane. The following mapping applies:

    • Envoy proxies: The kgateway GatewayClass is changed to gloo-gateway-v2.
    • Agentgateway proxies: The agentgateway GatewayClass is changed to agentgateway-enterprise.

    Example command to update the http Gateway with the gloo-gateway-v2 GatewayClass:

      kubectl patch gateway http -n kgateway-system --type=merge -p '{"spec":{"gatewayClassName":"gloo-gateway-v2"}}'

    Example command to update all Gateways with the gloo-gateway-v2 GatewayClass:

      kubectl get gateways --all-namespaces -o json \
  | jq -r '.items[] | select(.spec.gatewayClassName=="kgateway") | [.metadata.namespace, .metadata.name] | @tsv' \
  | while IFS=$'\t' read -r ns name; do
      kubectl patch gateway "$name" -n "$ns" --type=merge -p '{"spec":{"gatewayClassName":"gloo-gateway-v2"}}'
    done

  3. Monitor the rollout of the new gateway proxy pods. Verify that a new gateway proxy pod is deployed while the old one is being terminated.

      kubectl get pods -n kgateway-system

    Example output:

      NAME                        READY   STATUS        RESTARTS   AGE
http-65c76c754c-vvbbd       1/1     Running       0          24s
http-78f5744788-7z58j       1/1     Terminating   0          78m
kgateway-6c959579f6-d757w   1/1     Running       0          98m

  4. Verify that no requests were dropped during the rollout of the new proxy pod. If a request was dropped, you see an ALERT: dropped request, got HTTP 503 message in the CLI output of the curl pod that you deployed earlier.

Step 4: Uninstall kgateway

Now that you finished the migration of your gateway proxies, you can uninstall the kgateway control plane.

  helm uninstall kgateway

Congratulations, you successfully migrated your gateway proxies to Gloo Gateway.

Next: Enable enterprise features

As you explore enterprise features, such as JWT authentication and authorization, keep the following considerations in mind:

  • Most enterprise features are enabled by using a GlooTrafficPolicy resource.
  • The GlooTrafficPolicy is a superset of kgateway’s TrafficPolicy and comes with additional fields for the enterprise capabilities.
  • As you add enterprise features to your routes or Gateways, consider moving all policies that apply to the same route or Gateway from a TrafficPolicy to a GlooTrafficPolicy.

For example, let’s assume you currently apply the following TrafficPolicy to your http Gateway to enable local rate limiting:

  apiVersion: gateway.kgateway.dev/v1alpha1
kind: TrafficPolicy
metadata:
  name: local-ratelimit
  namespace: kgateway-system
spec:
  targetRefs: 
  - group: gateway.networking.k8s.io
    kind: Gateway
    name: http
  rateLimit:
    local:
      tokenBucket:
        maxTokens: 1
        tokensPerFill: 1
        fillInterval: 100s

To enable enterprise features, such as JWT authentication and authorization, on this gateway, create a GlooTrafficPolicy and add both your local rate limiting and JWT policies in the same policy as shown in the following example:

  apiVersion: gloo.solo.io/v1alpha1
kind: GlooTrafficPolicy
metadata:
  name: http-policy
  namespace: httpbin
spec:
  rateLimit:
    local:
      tokenBucket:
        maxTokens: 1
        tokensPerFill: 1
        fillInterval: 100s
  glooJWT:
    beforeExtAuth:
      providers:
        selfminted:
          issuer: solo.io
          jwks:
            local:
              key: '{"keys":[{"kty":"RSA","kid":"solo-public-key-001","use":"sig","alg":"RS256","n":"AOfIaJMUm7564sWWNHaXt_hS8H0O1Ew59-nRqruMQosfQqa7tWne5lL3m9sMAkfa3Twx0LMN_7QqRDoztvV3Wa_JwbMzb9afWE-IfKIuDqkvog6s-xGIFNhtDGBTuL8YAQYtwCF7l49SMv-GqyLe-nO9yJW-6wIGoOqImZrCxjxXFzF6mTMOBpIODFj0LUZ54QQuDcD1Nue2LMLsUvGa7V1ZHsYuGvUqzvXFBXMmMS2OzGir9ckpUhrUeHDCGFpEM4IQnu-9U8TbAJxKE5Zp8Nikefr2ISIG2Hk1K2rBAc_HwoPeWAcAWUAR5tWHAxx-UXClSZQ9TMFK850gQGenUp8","e":"AQAB"}]}'

After you apply the GlooTrafficPolicy resource, you can remove the old TrafficPolicy resource.