Cluster Registration

Gloo Gateway Federation monitors and automatically discovers instances of Gloo Gateway on clusters that are registered with glooctl. After the registration process is complete, you can use Gloo Gateway Federation to create and apply federated configuration resources across registered clusters.

In this guide, you register a Kubernetes cluster with Gloo Gateway Federation.

Figure of federated architecture

Prerequisites

To successfully follow this guide, you need to have Gloo Gateway Federation deployed on an admin cluster and a cluster to use for registration. The cluster can either be the admin cluster or a remote cluster. For instructions, follow the Gloo Gateway Federation installation guide.

Register a cluster

Gloo Gateway Federation does not automatically register the Kubernetes cluster that it runs on. You must register both the local cluster and any remote clusters. The registration process creates a service account, cluster role, and cluster role binding on the target cluster, and stores the access credentials in a Kubernetes secret resource in the admin cluster.

Registration with glooctl

To register a cluster with glooctl, run the following command. This example registers the admin cluster.

glooctl cluster register --cluster-name local --remote-context gloo-fed
If you are running the registration command for a kind cluster on MacOS or Linux, append the `local-cluster-domain-override` flag to the command:
# MacOS
glooctl cluster register --cluster-name local --remote-context kind-local \
  --local-cluster-domain-override host.docker.internal[:6443]
# Linux
# Get the IP address of the local cluster control plane
LOCAL_IP=$(docker exec local-control-plane ip addr show dev eth0 | sed -nE 's|\s*inet\s+([0-9.]+).*|\1|p')
glooctl cluster register --cluster-name local --remote-context kind-local \
  --local-cluster-domain-override $LOCAL_IP:6443

Cluster registration creates a KubernetesCluster CR that contains information about the cluster that was just registered, including its credentials.

Credentials for the remote cluster are stored in a secret in the gloo-system namespace. The secret name is the same as the cluster-name that you specified during registration.

kubectl get secret -n gloo-system local
NAME    TYPE                 DATA   AGE
local   solo.io/kubeconfig   1      37s

In the registered cluster, Gloo Gateway Federation has created a Kubernetes service account, cluster role, and role binding. You can review these resources by running the following commands:

kubectl --cluster kind-local get serviceaccount local -n gloo-system
kubectl --cluster kind-local get clusterrole gloo-federation-controller
kubectl --cluster kind-local get clusterrolebinding local-gloo-federation-controller-clusterrole-binding

After a cluster is registered, Gloo Gateway Federation automatically discovers all instances of Gloo Gateway within the cluster. The discovered instances are stored in a Custom Resource of type GlooInstance in the gloo-system namespace. You can view the discovered instances by running the following command:

kubectl get glooinstances -n gloo-system
NAME                      AGE
local-gloo-system         95m

You have now successfully added a local or remote cluster to Gloo Gateway Federation. You can repeat the same process for any other clusters you want to include in Gloo Gateway Federation.

Manual registration

When you register a cluster with glooctl, the following Kubernetes resources are created in your cluster.

On the remote cluster:

On the local cluster that runs Gloo Federation:

If you cannot use the glooctl CLI, you can manually create these resources with the following steps.

If the cluster to register is running with KinD, empty the ca-cert section of your `~/kube/config` file, and set `insecure-skip-tls-verify: true`.
  1. Set the cluster name that you want to register as an environment variable that is used in the rest of this guide.

    export CLUSTER_NAME=target-cluster
    
  2. Optional: If Gloo Gateway is not already running on the remote cluster, install Gloo Gateway, such as with the following steps.

    # install the Gloo Federation CRDs on the target cluster:
    helm fetch glooe/gloo-ee --version ${GLOO_VERSION} --devel --untar --untardir /tmp/glooee-${GLOO_VERSION}
    kubectl --context ${CLUSTER_NAME} apply -f /tmp/glooee-${GLOO_VERSION}/gloo-ee/charts/gloo-fed/crds/
      
    # install the Gloo Gateway CRDs on the admin cluster:
    kubectl apply -f /tmp/glooee-${GLOO_VERSION}/gloo-ee/charts/gloo/crds/
    
  3. On the remote cluster, create the following Kubernetes resources.

    kubectl --context ${CLUSTER_NAME} create ns gloo-system
    kubectl --context ${CLUSTER_NAME} -n gloo-system create sa ${CLUSTER_NAME}
    secret=$(kubectl --context ${CLUSTER_NAME} -n gloo-system get sa ${CLUSTER_NAME} -o jsonpath="{.secrets[0].name}")
    token=$(kubectl --context ${CLUSTER_NAME} -n gloo-system get secret $secret -o jsonpath="{.data.token}" | base64 -d)
       
    kubectl --context ${CLUSTER_NAME} -n gloo-system apply -f - <<EOF
    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRole
    metadata:
      name: gloo-federation-controller
    rules:
    - apiGroups:
      - gloo.solo.io
      - gateway.solo.io
      - enterprise.gloo.solo.io
      - ratelimit.solo.io
      - graphql.gloo.solo.io
      resources:
      - '*'
      verbs:
      - '*'
    - apiGroups:
      - apps
      resources:
      - deployments
      - daemonsets
      verbs:
      - get
      - list
      - watch
    - apiGroups:
      - ""
      resources:
      - pods
      - nodes
      - services
      verbs:
      - get
      - list
      - watch
    ---
    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRoleBinding
    metadata:
      name: ${CLUSTER_NAME}-gloo-federation-controller-clusterrole-binding
    roleRef:
      apiGroup: rbac.authorization.k8s.io
      kind: ClusterRole
      name: gloo-federation-controller
    subjects:
    - kind: ServiceAccount
      name: ${CLUSTER_NAME}
      namespace: gloo-system
    EOF
    
  4. Prepare a Kubernetes config file. Note that the server field must match the address for Gloo Federation to connect to that remote cluster’s API server.

    cat > kc.yaml <<EOF
    apiVersion: v1
    clusters:
    - cluster:
        certificate-authority-data: "" # DO NOT empty the value here unless you are using KinD
        server: https://host.docker.internal:65404 # enter the api-server address of the target cluster
        insecure-skip-tls-verify: true # DO NOT use this unless you know what you are doing
      name: ${CLUSTER_NAME}
    contexts:
    - context:
        cluster: ${CLUSTER_NAME}
        user: ${CLUSTER_NAME}
      name: ${CLUSTER_NAME}
    current-context: ${CLUSTER_NAME}
    kind: Config
    preferences: {}
    users:
    - name: ${CLUSTER_NAME}
      user:
        token: $token
    EOF
       
    kc=$(cat kc.yaml | base64 -w0)
    
  5. On the cluster that runs gloo-fed, create the following Kubernetes resources.

    kubectl --context gloo-fed-cluster apply -f - <<EOF
    ---
    apiVersion: multicluster.solo.io/v1alpha1
    kind: KubernetesCluster
    metadata:
      name: ${CLUSTER_NAME}
      namespace: gloo-system
    spec:
      clusterDomain: cluster.local
      secretName: ${CLUSTER_NAME}
    ---
    apiVersion: v1
    kind: Secret
    metadata:
      name: ${CLUSTER_NAME}
      namespace: gloo-system
    type: solo.io/kubeconfig
    data:
      kubeconfig: $kc
    EOF
    

Now that the custom resources are created, Gloo Federation automatically discovers the Gloo Gateway deployments in the target cluster.

Next Steps

With a registered cluster in Gloo Gateway Federation, now might be a good time to read a bit more about the concepts behind Gloo Gateway Federation, or you can try out the Federated Configuration feature.