Install eBPF-based sidecar acceleration
Use eBPF sidecar acceleration to reduce network latency in your service mesh.
About sidecar acceleration
Gloo Mesh Enterprise ships with an optional component that enables eBPF-based sidecar acceleration.
Instead of using iptables rules to redirect ingress and egress traffic from an app container to the injected Istio sidecar proxy and vice versa, eBPF is used to intercept the traffic and shorten the data path in a service mesh. With eBPF, packets to and from apps can be directly forwarded from one socket to the other. This setup reduces network latency and the necessary packet processing in the kernel.
The Istio architecture is not changed. When you remove the Gloo Mesh add-on, Istio's iptables rules are again used to redirect network traffic to and from the sidecars, just as they were before you applied the add-on.
This is an experimental feature. If you are interested in using this feature in production, contact the Solo team so that we can understand your environment and needs.
Version support
Because this Gloo Mesh add-on uses eBPF, all nodes in the cluster must run the Linux kernel version 5.7 or higher.
The following public Kubernetes distributions have been tested:
- 1.22.8-gke.202 (default) with Container-Optimized OS with containerd (
cos_containerd
)
Before you begin
- Create or use existing Kubernetes clusters to set up your Gloo management and data plane. Make sure that each node in your cluster runs a supported Linux kernel version. If you want to view eBPF debug logs, you must be able to access a shell on the nodes directly, for example by using SSH or your public cloud provider's tooling.
- Install the Gloo management server, register workload clusters, and install Istio in workload clusters.
- Deploy a sample app that you can use to verify that traffic is redirected and accelerated with eBPF. For example, you can follow this guide to deploy the Bookinfo app.
- Set the context for one of your workload clusters as
$REMOTE_CONTEXT
.export REMOTE_CONTEXT=<workload-cluster-context>
Installation
The sidecar acceleration component can be deployed in one or all of your workload clusters that have Istio installed.
-
Install the
sidecar-accel
add-on by upgrading your Helm release in each workload cluster. Note that thesidecarAccel.debug
value is set to true to enable detailed eBPF logs. These logs are later used to verify that the sidecar acceleration works as expected. Be sure to change the$REMOTE_CONTEXT
each time you repeat this command.helm upgrade --install gloo-platform gloo-platform/gloo-platform \ --kube-context $REMOTE_CONTEXT \ --namespace gloo-mesh \ --version $GLOO_VERSION \ --values agent.yaml \ --set common.cluster=$REMOTE_CLUSTER \ --set licensing.glooMeshLicenseKey=$GLOO_MESH_LICENSE_KEY \ --set sidecarAccel.enabled=true \ --set sidecarAccel.debug=true
Make sure to include your Helm values when you upgrade either as a configuration file in the
–values
flag or with–set
flags. Otherwise, any previous custom values that you set might be overwritten. In single cluster setups, this might mean that your Gloo agent and ingress gateways are removed. To get your current values, such as for a release namedgloo-platform
, you can runhelm get values gloo-platform -n gloo-mesh > gloo-gateway-single.yaml
. For more information, see Get your Helm chart values in the upgrade guide. -
Verify that the
sidecar-accel
pods run successfully. Because the add-on is deployed as a daemon set, the number ofsidecar-accel
pods equals the number of nodes in your cluster.kubectl get pod -n istio-system --kube-context=$REMOTE_CONTEXT
Example output:
NAME READY STATUS RESTARTS AGE istio-ingressgateway-778f44479-sxssj 1/1 Running 0 12m istiod-6d67d84bc7-jq2wl 1/1 Running 0 12m sidecar-accel-hd6cg 1/1 Running 0 9m28s sidecar-accel-hz7cv 1/1 Running 0 9m28s sidecar-accel-mzspz 1/1 Running 0 9m28s
-
Generate traffic for your app. For example, if you deployed the Bookinfo app, you can log in to the product page pod and then curl the product page to generate network traffic.
kubectl exec $(kubectl get pod -l app=productpage -A --context ${REMOTE_CONTEXT} -o jsonpath='{.items[0].metadata.name}') -n bookinfo -c curl --context ${REMOTE_CONTEXT} -- curl -v http://productpage:9080
-
Verify that the sidecar acceleration works correctly by checking the eBPF debug logs.
-
Log in to a cluster node. For example, you can use
ssh
to log in the node directly or use the options that your cloud provider offers, such as opening SSH sessions to a node from the web console.ssh root@<node-IP-address>
-
Review the eBPF kernel logs. The sidecar acceleration works correctly if connections and data are successfully redirected by eBPF.
sudo cat /sys/kernel/debug/tracing/trace_pipe
Example output:
curl-57979 [000] d... 11100.185999: bpf_trace_printk: [debug] outbound connect from app container: cookie: 16445, ip: 0x380fc40a, port: 9080 curl-57979 [000] d... 11100.186128: bpf_trace_printk: [debug] redirect 88 bytes with eBPF successfully wrk:worker_1-28489 [000] d... 11100.363604: bpf_trace_printk: [debug] redirect 227 bytes with eBPF successfully
-
Limitations
Aside from the kernel version requirements, certain Istio functionality, such as the outbound exclusion of IP addresses and port ranges from the sidecar proxy, does not work as expected when the sidecar acceleration add-on is enabled. Contact the Solo team if you plan to use this Istio feature with sidecar acceleration.