Single cluster

Use Gloo Network for Cilium to provide connectivity, security, and observability for containerized workloads with a Cilium-based container network interface (CNI) plug-in that leverages the Linux kernel technology eBPF. Gloo Network additionally provides insights into your Cilium environment through a custom dashboard.

You can follow this guide to quickly get started with Gloo Network. To learn more about the benefits and architecture, see About. To customize your installation with Helm instead, see the advanced installation guide.

Before you begin link

1. Install the following command-line (CLI) tools.

   • kubectl, the Kubernetes command line tool. Download the kubectl version that is within one minor version of the Kubernetes clusters you plan to use.
   • meshctl, the Solo command line tool.

       curl -sL https://run.solo.io/meshctl/install | GLOO_MESH_VERSION=v2.5.11 sh -
     export PATH=$HOME/.gloo-mesh/bin:$PATH
       

   • helm, the Kubernetes package manager.
   • cilium, the Cilium command line tool.

2. Create environment variables for the following details.

   • GLOO_NETWORK_LICENSE_KEY: Set your Gloo Network for Cilium license key as an environment variable. If you do not have one, contact an account representative.
   • GLOO_VERSION: Set the Gloo Network version. This example uses the latest version. Append -fips for a FIPS-compliant image. Do not include v before the version number.
   • SOLO_CILIUM_REPO: A repo key for the Solo distribution of Cilium that you can get by logging in to the Support Center and reviewing the Cilium images built by Solo.io support article.
   • CILIUM_VERSION: The Cilium version that you want to install. This example uses the latest version.

     export GLOO_NETWORK_LICENSE_KEY=<license_key>
   export GLOO_VERSION=2.5.11
   export SOLO_CILIUM_REPO=<cilium_repo_key>
   export CILIUM_VERSION=1.14.2
     

3. Optional: If you plan to run Istio with sidecar injection and the Cilium CNI in tunneling mode (VXLAN or GENEVE) on an Amazon EKS cluster, see Considerations for running Cilium and Istio on EKS.

Install the Solo distribution of the Cilium CNI link

1. Create or use Kubernetes clusters that meet the Cilium requirements. For example, to try out the Cilium CNI in Google Kubernetes Engine (GKE) clusters, your clusters must be created with specific node taints.

   1. Open the Cilium documentation and find the cloud provider that you want to use to create your clusters.

   2. Follow the steps of your cloud provider to create one or more clusters that meet the Cilium requirements.

      • The instructions in the Cilium documentation might create a cluster with insufficient CPU and memory resources for Gloo Network. Make sure that you use a machine type with at least 2vCPU and 8GB of memory.
      • The cluster name must be alphanumeric with no special characters except a hyphen (-), lowercase, and begin with a letter (not a number).
      • Multicluster setups only: For a multicluster setup, you need at least two clusters. One cluster is set up as the Gloo management plane where the management components are installed. The other cluster is registered as your data plane and runs your Kubernetes workloads and Istio service mesh. You can optionally add more workload clusters to your setup. The instructions in this guide assume one management cluster and two workload clusters.

      Example to create a cluster in GKE:

        export NAME="$(whoami)-$RANDOM"                                                        
      gcloud container clusters create "${NAME}" \
      --node-taints node.cilium.io/agent-not-ready=true:NoExecute \
      --zone us-west2-a \
      --machine-type e2-standard-2
      gcloud container clusters get-credentials "${NAME}" --zone us-west2-a
        

2. Add and update the Cilium Helm repo.

     helm repo add cilium https://helm.cilium.io/
   helm repo update
     

3. Install the CNI by using a Solo distribution of Cilium in your cluster. Be sure to include the following settings for compatability with Gloo Network, and the necessary settings for your infrastructure provider.

   warning

   The steps to install the CNI vary depending on the way you create your cluster. For example, installing the CNI in a kind cluster is different from installing the CNI in a GKE cluster. Depending on the cloud provider you use, you must update this command to add additional Helm values as suggested in the Cilium documentation.

   Generic example (add cloud provider-specific values in --set flags below):

     helm install cilium cilium/cilium \
   --namespace kube-system \
   --version ${CILIUM_VERSION} \
   --set customCalls.enabled=true \
   --set hubble.enabled=true \
   --set hubble.metrics.enabled="{dns:destinationContext=pod;sourceContext=pod,drop:destinationContext=pod;sourceContext=pod,tcp:destinationContext=pod;sourceContext=pod,flow:destinationContext=pod;sourceContext=pod,port-distribution:destinationContext=pod;sourceContext=pod}" \
   --set image.repository=${SOLO_CILIUM_REPO}/cilium \
   --set image.useDigest=false \
   --set operator.image.repository=${SOLO_CILIUM_REPO}/operator \
   --set operator.image.useDigest=false \
   --set operator.prometheus.enabled=true \
   --set prometheus.enabled=true \
   # ADD CLOUD PROVIDER-SPECIFIC HELM VALUES HERE
     

   Example for GKE:

     NATIVE_CIDR="$(gcloud container clusters describe "${NAME}" --zone "${ZONE}" --format 'value(clusterIpv4Cidr)')"
   echo $NATIVE_CIDR
   
   helm install cilium cilium/cilium \
   --namespace kube-system \
   --version ${CILIUM_VERSION} \
   --set customCalls.enabled=true \
   --set hubble.enabled=true \
   --set hubble.metrics.enabled="{dns:destinationContext=pod;sourceContext=pod,drop:destinationContext=pod;sourceContext=pod,tcp:destinationContext=pod;sourceContext=pod,flow:destinationContext=pod;sourceContext=pod,port-distribution:destinationContext=pod;sourceContext=pod}" \
   --set image.repository=${SOLO_CILIUM_REPO}/cilium \
   --set image.useDigest=false \
   --set operator.image.repository=${SOLO_CILIUM_REPO}/operator \
   --set operator.image.useDigest=false \
   --set operator.prometheus.enabled=true \
   --set prometheus.enabled=true \
   --set nodeinit.enabled=true \
   --set nodeinit.reconfigureKubelet=true \
   --set nodeinit.removeCbrBridge=true \
   --set cni.binPath=/home/kubernetes/bin \
   --set gke.enabled=true \
   --set ipam.mode=kubernetes \
   --set ipv4NativeRoutingCIDR=${NATIVE_CIDR}
     

   Generic example (add any cloud provider-specific values):

     helm upgrade -i -n kube-system cilium cilium/cilium --version ${CILIUM_VERSION} -f - <<EOF
   customCalls:
     enabled: true
   hubble:
     enabled: true
     metrics:
       enabled:
       - dns:destinationContext=pod;sourceContext=pod
       - drop:destinationContext=pod;sourceContext=pod
       - tcp:destinationContext=pod;sourceContext=pod
       - flow:destinationContext=pod;sourceContext=pod
       - port-distribution:destinationContext=pod;sourceContext=pod
   image:
     repository: ${SOLO_CILIUM_REPO}/cilium
     useDigest: false
   operator:
     replicas: 1
     prometheus:
       enabled: true
     image:
       repository: ${SOLO_CILIUM_REPO}/operator
       useDigest: false
   prometheus:
     enabled: true
   # ADD CLOUD PROVIDER-SPECIFIC HELM VALUES
   EOF
     

   Example for GKE:

     NATIVE_CIDR="$(gcloud container clusters describe "${NAME}" --zone "${ZONE}" --format 'value(clusterIpv4Cidr)')"
   echo $NATIVE_CIDR
   
   helm upgrade -i -n kube-system cilium cilium/cilium --version ${CILIUM_VERSION} -f - <<EOF
   customCalls:
     enabled: true
   hubble:
     enabled: true
     metrics:
       enabled:
       - dns:destinationContext=pod;sourceContext=pod
       - drop:destinationContext=pod;sourceContext=pod
       - tcp:destinationContext=pod;sourceContext=pod
       - flow:destinationContext=pod;sourceContext=pod
       - port-distribution:destinationContext=pod;sourceContext=pod
   image:
     repository: ${SOLO_CILIUM_REPO}/cilium
     useDigest: false
   operator:
     replicas: 1
     prometheus:
       enabled: true
     image:
       repository: ${SOLO_CILIUM_REPO}/operator
       useDigest: false
   prometheus:
     enabled: true
   nodeinit:
     enabled: true
     reconfigureKubelet: true
     removeCbrBridge: true
   cni:
     binPath: /home/kubernetes/bin
   gke:
     enabled: true
   ipam:
     mode: kubernetes
   ipv4NativeRoutingCIDR: ${NATIVE_CIDR}
   EOF
     

   Example output:

     NAME: cilium
   LAST DEPLOYED: Fri Sep 16 10:31:52 2022
   NAMESPACE: kube-system
   STATUS: deployed
   REVISION: 1
   TEST SUITE: None
   NOTES:
   You have successfully installed Cilium with Hubble.
   
   Your release version is 1.14.2.
   
   For any further help, visit https://docs.cilium.io/en/v1.12/gettinghelp
     

4. Verify that the Cilium CNI is successfully installed. Because the Cilium agent is deployed as a daemon set, the number of Cilium and Cilium node init pods equals the number of nodes in your cluster.

     kubectl get pods -n kube-system | grep cilium
     

   Example output:

     cilium-gbqgq                                                  1/1     Running             0          48s
   cilium-j9n5x                                                  1/1     Running             0          48s
   cilium-node-init-c7rxb                                        1/1     Running             0          48s
   cilium-node-init-pnblb                                        1/1     Running             0          48s
   cilium-node-init-wdtjm                                        1/1     Running             0          48s
   cilium-operator-69dd4567b5-2gjgg                              1/1     Running             0          47s
   cilium-operator-69dd4567b5-ww6wp                              1/1     Running             0          47s
   cilium-smp9c                                                  1/1     Running             0          48s
     

5. Check the status of the Cilium installation.

     cilium status --wait
     

   Example output:

     
   ____/¯¯\
    /¯¯\__/¯¯\    Cilium:             OK
    \__/¯¯\__/    Operator:           OK
    /¯¯\__/¯¯\    Envoy DaemonSet:    disabled (using embedded mode)
    \__/¯¯\__/    Hubble Relay:       disabled
       \__/       ClusterMesh:        disabled
   
   Deployment             cilium-operator    Desired: 2, Ready: 2/2, Available: 2/2
   DaemonSet              cilium             Desired: 4, Ready: 4/4, Available: 4/4
   Containers:            cilium-operator    Running: 2
                          cilium             Running: 4
   Cluster Pods:          3/3 managed by Cilium
   Helm chart version:    1.14.2
   Image versions         cilium             ${SOLO_CILIUM_REPO}/cilium:v1.14.2: 4
                          cilium-operator    ${SOLO_CILIUM_REPO}/operator-generic:v1.14.2: 2
     

6. Multicluster setups only: Repeat steps 3 - 5 to install the CNI in each cluster that you want to use in your Gloo Network environment.

Install Gloo Network for Cilium link

Install the Gloo Network components in your cluster and verify that Gloo Network can discover the Cilium CNI.

1. Save the name of your cluster as an environment variable.

     export CLUSTER_NAME=<cluster_name>
     

2. Install Gloo Network in your cluster. This command uses basic profiles to create a gloo-mesh namespace and install the Gloo control and data plane components. For more information, check out the CLI install profiles. Additionally, this command enables Cilium metrics collection so that Gloo Network can generate insights for your Cilium setup’s health and security posture.

     meshctl install --profiles gloo-core-single-cluster,cilium-flows \
     --set common.cluster=$CLUSTER_NAME \
     --set licensing.glooNetworkLicenseKey=$GLOO_NETWORK_LICENSE_KEY \
     --set telemetryCollectorCustomization.pipelines.metrics/cilium.enabled=true
     

3. Verify that your Gloo Network setup is correctly installed. If not, try debugging the relay connection. Note that this check might take a few seconds to verify that:

   • Your Gloo product license is valid and current.
   • The Gloo CRDs are installed at the correct version.
   • The management plane pods in the management cluster are running and healthy.
   • The Gloo agent is running and connected to the management server.

     meshctl check
     

   Example output:

     🟢 License status
   
   INFO  gloo-network enterprise license expiration is 25 Aug 24 10:38 CDT
   INFO  No GraphQL license module found for any product
   
   🟢 CRD version check
   
   🟢 Gloo deployment status
   
   Namespace | Name                           | Ready | Status
   gloo-mesh | gloo-mesh-mgmt-server          | 1/1   | Healthy
   gloo-mesh | gloo-mesh-redis                | 1/1   | Healthy
   gloo-mesh | gloo-mesh-ui                   | 1/1   | Healthy
   gloo-mesh | gloo-telemetry-collector-agent | 3/3   | Healthy
   gloo-mesh | prometheus-server              | 1/1   | Healthy
   
   🟢 Mgmt server connectivity to workload agents
   
   Cluster | Registered | Connected Pod                                   
   test    | true       | gloo-mesh/gloo-mesh-mgmt-server-558cddbbd7-rf2hv
   
   Connected Pod                                    | Clusters
   gloo-mesh/gloo-mesh-mgmt-server-558cddbbd7-rf2hv | 1  
     

Explore the UI link

Visualize and analyze your Cilium and Gloo Network setup by deploying sample apps and launching the Gloo UI.

Launch the dashboard link

1. Open the Gloo UI. The Gloo UI is served from the gloo-mesh-ui service on port 8090. You can connect by using the meshctl or kubectl CLIs.

   • meshctl: For more information, see the CLI documentation.

       meshctl dashboard
       

   • kubectl:
     1. Port-forward the gloo-mesh-ui service on 8090.

          kubectl port-forward -n gloo-mesh svc/gloo-mesh-ui 8090:8090
          

     2. Open your browser and connect to http://localhost:8090.

2. Review your Dashboard for an at-a-glance overview of your Gloo Network environment. Environment insights, health, status, inventories, and more are summarized in the following cards:

   • Analysis and Insights: Gloo Network recommendations for how to improve your Cilium setup.
   • Cilium and Gloo health: A status check of the Gloo Network and Cilium installations in your cluster.
   • Cluster Services: Inventory of services in your Gloo Network setup, and whether those services are in a service mesh or not.

     

3. In the Cilium tab in the Cilium and Gloo health card, verify that the Solo distribution of Cilium version was discovered.

   

Check insights link

Review the insights for your environment. Gloo Network comes with an insights engine that automatically analyzes your Cilium setup for health issues. Then, Gloo shares these issues along with recommendations to harden your Cilium setup. The insights give you a checklist to address issues that might otherwise be hard to detect across your environment.

1. On the Analysis and Insights card of the dashboard, you can quickly see a summary of the insights for your environment, including how many insights are available at each severity level, and the type of insight.

   

2. View the list of insights by clicking the Details button, or go to the Insights page.

3. On the Insights page, you can view recommendations to harden your Cilium setup, and steps to implement them in your environment. Gloo Network analyzes your setup, and returns individual insights that contain information about errors and warnings in your environment, best practices you can use to improve your configuration and security, and more.
   

   

4. On an insight that you want to resolve, click Details. The details modal shows more data about the insight, such as the time when it was last observed in your environment, and if applicable, the extended settings or configuration that the insight applies to.

5. Click the Target YAML tab to see the resource file that the insight references, and click the View Resolution Steps tab to see guidance such as steps for fixing warnings and errors in your resource configuration or recommendations for improving your security and setup.

Apply a Cilium network policy link

Deploy a demo app and try out a Cilium network policy to secure and control traffic flows between app microservices.

1. Deploy the Cilium Star Wars demo app in your cluster.

   1. Create a namespace for the demo app.

        kubectl create ns starwars
        

   2. Deploy the demo app, which includes tiefighter, xwing, and deathstar pods, and a deathstar service. The tiefighter and deathstar pods have the org=empire label, and the xwing pod has the org=alliance label.

        kubectl -n starwars apply -f https://raw.githubusercontent.com/cilium/cilium/$CILIUM_VERSION/examples/minikube/http-sw-app.yaml
        

   3. Verify that the demo pods and service are running.

        kubectl get pods,svc -n starwars
        

      Example output:

        NAME                             READY   STATUS    RESTARTS   AGE
      pod/deathstar-6fb5694d48-5hmds   1/1     Running   0          107s
      pod/deathstar-6fb5694d48-fhf65   1/1     Running   0          107s
      pod/tiefighter                   1/1     Running   0          107s
      pod/xwing                        1/1     Running   0          107s
      
      NAME                 TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)   AGE
      service/deathstar    ClusterIP   10.96.110.8   <none>        80/TCP    107s
        

2. Generate some network traffic by sending requests from the xwing and tiefighter pods to the deathstar service.

     kubectl exec xwing -n starwars -- curl -s -XPOST deathstar.starwars.svc.cluster.local/v1/request-landing
   kubectl exec tiefighter -n starwars -- curl -s -XPOST deathstar.starwars.svc.cluster.local/v1/request-landing
     

   Example output for both commands:

     Ship landed
   Ship landed
     

3. Create a Cilium network policy that allows only apps that have the org=empire label to access the deathstar app. After you create this access policy, only the tiefighter pod can access the deathstar app.

     kubectl apply -f - << EOF
   apiVersion: "cilium.io/v2"
   kind: CiliumNetworkPolicy
   metadata:
     name: "rule1"
     namespace: starwars
   spec:
     description: "L3-L4 policy to restrict deathstar access to empire ships only"
     endpointSelector:
       matchLabels:
         org: empire
         class: deathstar
     ingress:
     - fromEndpoints:
       - matchLabels:
           org: empire
       toPorts:
       - ports:
         - port: "80"
           protocol: TCP
   EOF
     

4. Send another request from the tiefighter pod to the deathstar service.

     kubectl exec tiefighter -n starwars -- curl -s -XPOST deathstar.starwars.svc.cluster.local/v1/request-landing
     

   The request succeeds, because requests from apps with the org=empire label are permitted. Example output:

     Ship landed
     

5. Send another request from the xwing pod to the deathstar service.

     kubectl exec xwing -n starwars -- curl -s -XPOST deathstar.starwars.svc.cluster.local/v1/request-landing
     

   This request hangs, because requests from apps without the org=empire label are not permitted. No Layer 7 HTTP response code is returned, because the request is dropped on Layer 3. You can enter control+c to stop the curl request, or wait for it to time out.

6. You can also check the metrics to verify that the policy allowed or blocked requests.

   1. Open the Prometheus expression browser.
      • meshctl: For more information, see the CLI documentation.

          meshctl proxy prometheus
          

      • kubectl:
        1. Port-forward the prometheus-server deployment on 9091.

             kubectl -n gloo-mesh port-forward deploy/prometheus-server 9091
             

        2. Open your browser and connect to localhost:9091/.
   2. In the Expression search bar, paste the following query and click Execute.

        rate(hubble_drop_total{destination_workload_id=~"deathstar.+"}[5m])
        

   3. Verify that you can see requests from the xwing pod to the deathstar service that were dropped because of the network policy.

Next steps link

Now that you have Gloo Network for Cilium up and running, check out some of the following resources to learn more about Gloo Network and expand your Cilium capabilities.

Cilium:

• Find out more about hardened Cilium n-4 version support built into Solo distributions of Cilium images.
• Enable additional flow logs to monitor network traffic in your cluster.
• Import the Cilium Grafana dashboard to monitor the health of your Cilium CNI.
• Apply more Cilium network policies by following the Cilium docs.

Gloo Network:

• Customize your Gloo Network installation with a Helm-based setup.
• Explore insights to review and improve your setup’s health and security posture.
• When it’s time to upgrade Gloo Network, see the upgrade guide.

Istio: To also deploy Istio in your environment, check out Gloo Mesh Core or Gloo Mesh Enterprise to quickly install and manage your service mesh for you with service mesh lifecycle management.

Help and support:

• Talk to an expert to get advice or build out a proof of concept.
• Join the #gloo-mesh channel in the Solo.io community slack.
• Try out one of the Gloo workshops.

Cleanup link

1. Remove the demo app resources and namespace.

     kubectl delete -f https://raw.githubusercontent.com/cilium/cilium/$CILIUM_VERSION/examples/minikube/http-sw-app.yaml -n starwars
   kubectl delete cnp rule1 -n starwars
   kubectl delete ns starwars
     

2. If you no longer need this quick-start Gloo Network environment, you can uninstall it by following the steps in the uninstall guide.