Envoy WASM filters with Gloo Edge
Support for Envoy WASM filters has been added to Gloo Edge Enterprise as of version 1.6.0+. This guide is specifically for Gloo Edge 1.6.0. Note, there may have been some changes to the configuration API since prior experimental versions.
This feature is considered to be in a tech preview state of stability. While wasm functionality has
been merged to upstream envoy, wasm filters are not yet recommended for production use. This tech preview
is meant to show off the potential of WASM filters, and how they will integrate with Gloo Edge going forward.
Envoy Wasm filters are a Gloo Edge Enterprise feature.
Configuration
Getting started with WASM is simple, first we install Gloo Edge Enterprise.
Gloo Edge Enterprise can be installed using either glooctl or helm 3 as follows:
glooctl install gateway enterprise --license-key YOUR_LICENSE_KEY
helm repo add gloo https://storage.googleapis.com/solo-public-helm
1. Get a Wasm image. For more information on building your own Wasm image, see the [WebAssembly Developer's Guide](https://webassembly.org/getting-started/developers-guide/).
2. Prepare your Wasm image for use with Gloo Edge Enterprise. Review the following options.
* Store in an OCI-compliant image repository. This guide uses an example Wasm image from Solo's public Google Container Registry.
* Load the Wasm file directly into the filter. If your filter is not hosted in an image repository, you can refer to the filepath directly, such as `<directory>/<filter-name>.wasm`.
* Use an init container. In some circumstances, you might not be able to use an image repository due to enterprise networking restrictions. Instead, you can use an `initContainer` on the Gloo Edge `gatewayProxy` deployment to load a `.wasm` file into a shared `volume`.
## Configure Gloo Edge to use a Wasm filter {#configuration}
Now that Gloo Edge Enterprise is installed and you have your Wasm image, you are ready to configure Gloo Edge to use the Wasm filter. You add the filter to your gateway proxy configuration. For more information, check out the <a href=/gloo-edge/v1.13.x/reference/api/github.com/solo-io/gloo/projects/gloo/api/v1/options/wasm/wasm.proto.sk/#pluginsource>API docs</a>
.
<div id="tabset-installationadvanced_configurationwasm-1">
<ul>
<li><a href="#tabset-installationadvanced_configurationwasm-1-0">From an image registry</a></li>
<li><a href="#tabset-installationadvanced_configurationwasm-1-1">From filepath</a></li>
<li><a href="#tabset-installationadvanced_configurationwasm-1-2">From an init container</a></li>
</ul>
<div id="tabset-installationadvanced_configurationwasm-1-0">
<ol>
<li>Get the configuration for your <code>gateway-proxy</code> gateway.
<div class="highlight"><pre class="chroma"><code class="language-shell" data-lang="shell">kubectl get -n gloo-system gateways.gateway.solo.io gateway-proxy -o yaml > gateway-proxy.yaml
open gateway-proxy.yaml
</code></pre></div></li>
<li>Add the reference to your Wasm filter in the <code>httpGateway</code> section as follows.
<div class="highlight"><pre class="chroma"><code class="language-yaml" data-lang="yaml"><span class="w"> </span><span class="nt">httpGateway</span><span class="p">:</span><span class="w">
</span><span class="w"> </span><span class="nt">options</span><span class="p">:</span><span class="w">
</span><span class="w"> </span><span class="nt">wasm</span><span class="p">:</span><span class="w">
</span><span class="w"> </span><span class="nt">filters</span><span class="p">:</span><span class="w">
</span><span class="w"> </span>- <span class="nt">config</span><span class="p">:</span><span class="w">
</span><span class="w"> </span><span class="nt">'@type'</span><span class="p">:</span><span class="w"> </span><span class="l">type.googleapis.com/google.protobuf.StringValue</span><span class="w">
</span><span class="w"> </span><span class="nt">value</span><span class="p">:</span><span class="w"> </span><span class="s2">"world"</span><span class="w">
</span><span class="w"> </span><span class="nt">image</span><span class="p">:</span><span class="w"> </span><span class="l">gcr.io/solo-public/docs/assemblyscript-test:istio-1.8</span><span class="w">
</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">add-header</span><span class="w">
</span><span class="w"> </span><span class="nt">rootId</span><span class="p">:</span><span class="w"> </span><span class="l">add_header</span><span class="w">
</span></code></pre></div></li>
<li>Update the <code>gateway-proxy</code> gateway.
<div class="highlight"><pre class="chroma"><code class="language-sh" data-lang="sh">kubectl apply -n gloo-system -f gateway-proxy.yaml
</code></pre></div></li>
</ol>
</div>
<div id="tabset-installationadvanced_configurationwasm-1-1">
<ol>
<li>Get the configuration for your <code>gateway-proxy</code> gateway.
<div class="highlight"><pre class="chroma"><code class="language-shell" data-lang="shell">kubectl get -n gloo-system gateways.gateway.solo.io gateway-proxy -o yaml > gateway-proxy.yaml
</code></pre></div></li>
<li>Add the filepath reference to your <code>.wasm</code> file in the <code>httpGateway</code> section as follows.
<div class="highlight"><pre class="chroma"><code class="language-yaml" data-lang="yaml"><span class="w"> </span><span class="nt">httpGateway</span><span class="p">:</span><span class="w">
</span><span class="w"> </span><span class="nt">options</span><span class="p">:</span><span class="w">
</span><span class="w"> </span><span class="nt">wasm</span><span class="p">:</span><span class="w">
</span><span class="w"> </span><span class="nt">filters</span><span class="p">:</span><span class="w">
</span><span class="w"> </span>- <span class="nt">config</span><span class="p">:</span><span class="w">
</span><span class="w"> </span><span class="nt">'@type'</span><span class="p">:</span><span class="w"> </span><span class="l">type.googleapis.com/google.protobuf.StringValue</span><span class="w">
</span><span class="w"> </span><span class="nt">value</span><span class="p">:</span><span class="w"> </span><span class="s2">"world"</span><span class="w">
</span><span class="w"> </span><span class="nt">filePath</span><span class="p">:</span><span class="w"> </span><span class="l">filters-dir/my-filter.wasm</span><span class="w">
</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">add-header</span><span class="w">
</span><span class="w"> </span><span class="nt">rootId</span><span class="p">:</span><span class="w"> </span><span class="l">add_header</span><span class="w">
</span></code></pre></div></li>
<li>Update the <code>gateway-proxy</code> gateway.
<div class="highlight"><pre class="chroma"><code class="language-sh" data-lang="sh">kubectl apply -n gloo-system -f gateway-proxy.yaml
</code></pre></div></li>
</ol>
</div>
<div id="tabset-installationadvanced_configurationwasm-1-2">
<p>Build a Docker image that has the Wasm filter image you previously created and use this image in an init container that runs alongside the gateway proxy.</p>
<div class="notices note" >Note: If you use a C++ Wasm filter, make sure to upgrade to <code>proxy-wasm-cpp-sdk-b2e6b0759d34d760e527dadca413a285614f9e99</code>.</div>
<ol>
<li>Create a Dockerfile in the same location as your Wasm filter. The Dockerfile makes an image that has your Wasm filter, and copies the filter to the <code>/wasm-filters/</code> directory when the image runs. Later, you mount this directory in a shared volume. <em>Note: In the previous section, your Wasm file is called <code>filter.wasm</code> and is located at <code>.wasmstore/<uniqueId>/filter.wasm</code>. If built your filter with a different tool than <code>wasme</code> (such as <code>bazel</code>), your filter location might differ.</em>
<pre><code>FROM alpine
<p>COPY filter.wasm filter.wasm</p>
<p>CMD ["cp", "filter.wasm", "/wasm-filters/"]
</code></pre></li></p>
<li>Build and tag a Docker image from this Dockerfile. Replace the example values with your repository URL and preferred image name in the following example command.
<div class="highlight"><pre class="chroma"><code class="language-sh" data-lang="sh">docker build . -t localhost:8888/myorg/my-wasm-getter:1.0.0
</code></pre></div></li>
<li>Push the Docker image to an image repository that your enterprise network can access.
<pre><code>docker push localhost:8888/myorg/my-wasm-getter:1.0.0
</code></pre></li>
<li>Edit your <code>gateway-proxy</code> deployment to add an init container and mount a shared volume. For a full example, see this <a href="https://github.com/solo-io/gloo-edge-use-cases/blob/main/docs/gateway-proxy-wasm.yaml"><code>gateway-proxy-wasm.yaml</code> file</a>.
<ol>
<li>Get the configuration for the <code>gateway-proxy</code> deployment.
<div class="highlight"><pre class="chroma"><code class="language-sh" data-lang="sh">kubectl get -n gloo-system deployment gateway-proxy -o yaml > gateway-proxy-wasm.yaml
</code></pre></div></li>
<li>In the <code>spec.template.spec.volumes</code> section, add a volume named <code>wasm-filters</code> that all the containers in the template can access.
<div class="highlight"><pre class="chroma"><code class="language-yaml" data-lang="yaml"><span class="w"> </span><span class="nt">volumes</span><span class="p">:</span><span class="w">
</span><span class="w"> </span>- <span class="nt">configMap</span><span class="p">:</span><span class="w">
</span><span class="w"> </span><span class="nt">defaultMode</span><span class="p">:</span><span class="w"> </span><span class="m">420</span><span class="w">
</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">gateway-proxy-envoy-config</span><span class="w">
</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">envoy-config</span><span class="w">
</span><span class="w"> </span>- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">wasm-filters</span><span class="w">
</span></code></pre></div></li>
<li>In the <code>spec.template.spec.containers</code> section, add a mount path to the <code>wasm-filters</code> volume that you just configured.
<div class="highlight"><pre class="chroma"><code class="language-yaml" data-lang="yaml"><span class="w"> </span><span class="nt">containers</span><span class="p">:</span><span class="w">
</span><span class="w"> </span><span class="nt">volumeMounts</span><span class="p">:</span><span class="w">
</span><span class="w"> </span>- <span class="nt">mountPath</span><span class="p">:</span><span class="w"> </span><span class="l">/etc/envoy</span><span class="w">
</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">envoy-config</span><span class="w">
</span><span class="w"> </span>- <span class="nt">mountPath</span><span class="p">:</span><span class="w"> </span><span class="l">/wasm-filters</span><span class="w">
</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">wasm-filters</span><span class="w">
</span></code></pre></div></li>
<li>In the <code>spec.template.spec</code> section, add the following init container stanza, which refers to the Wasm image that you just built and mounts the volume.
<div class="highlight"><pre class="chroma"><code class="language-yaml" data-lang="yaml"><span class="w"> </span><span class="nt">initContainers</span><span class="p">:</span><span class="w">
</span><span class="w"> </span>- <span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">wasm-image</span><span class="w">
</span><span class="w"> </span><span class="nt">image</span><span class="p">:</span><span class="w"> </span><span class="l">localhost:8888/myorg/my-wasm-getter:1.0.0</span><span class="w">
</span><span class="w"> </span><span class="nt">imagePullPolicy</span><span class="p">:</span><span class="w"> </span><span class="l">IfNotPresent</span><span class="w">
</span><span class="w"> </span><span class="nt">volumeMounts</span><span class="p">:</span><span class="w">
</span><span class="w"> </span>- <span class="nt">mountPath</span><span class="p">:</span><span class="w"> </span><span class="l">/wasm-filters</span><span class="w">
</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">wasm-filters</span><span class="w">
</span></code></pre></div></li>
<li>Apply the updated <code>gateway-proxy</code> deployment.
<div class="highlight"><pre class="chroma"><code class="language-sh" data-lang="sh">kubectl apply -n gloo-system -f gateway-proxy-wasm.yaml
</code></pre></div></li>
</ol>
</li>
<li>Now that the Wasm filter is in a shared mount filepath accessible by Envoy, get the configuration for your <code>gateway-proxy</code> gateway.
<div class="highlight"><pre class="chroma"><code class="language-shell" data-lang="shell">kubectl get -n gloo-system gateways.gateway.solo.io gateway-proxy -o yaml > gateway-proxy.yaml
</code></pre></div></li>
<li>Add the filepath reference to your <code>.wasm</code> file in the <code>httpGateway</code> section as follows.
<div class="highlight"><pre class="chroma"><code class="language-yaml" data-lang="yaml"><span class="w"> </span><span class="nt">httpGateway</span><span class="p">:</span><span class="w">
</span><span class="w"> </span><span class="nt">options</span><span class="p">:</span><span class="w">
</span><span class="w"> </span><span class="nt">wasm</span><span class="p">:</span><span class="w">
</span><span class="w"> </span><span class="nt">filters</span><span class="p">:</span><span class="w">
</span><span class="w"> </span>- <span class="nt">config</span><span class="p">:</span><span class="w">
</span><span class="w"> </span><span class="nt">'@type'</span><span class="p">:</span><span class="w"> </span><span class="l">type.googleapis.com/google.protobuf.StringValue</span><span class="w">
</span><span class="w"> </span><span class="nt">value</span><span class="p">:</span><span class="w"> </span><span class="s2">"my test config"</span><span class="w">
</span><span class="w"> </span><span class="nt">filePath</span><span class="p">:</span><span class="w"> </span><span class="l">/wasm-filters/filter.wasm</span><span class="w">
</span><span class="w"> </span><span class="nt">name</span><span class="p">:</span><span class="w"> </span><span class="l">add-header</span><span class="w">
</span><span class="w"> </span><span class="nt">rootId</span><span class="p">:</span><span class="w"> </span><span class="l">add_header</span><span class="w">
</span></code></pre></div></li>
<li>Update the <code>gateway-proxy</code> gateway.
<div class="highlight"><pre class="chroma"><code class="language-sh" data-lang="sh">kubectl apply -n gloo-system -f gateway-proxy.yaml
</code></pre></div></li>
</ol>
</div>
</div>
<script>$(function(){$("#tabset-installationadvanced_configurationwasm-1").tabs();});</script>
Once this process has been completed, gloo should be up and running in the `gloo-system` namespace.
To check run the following:
```shell script
kubectl get pods -n gloo-system
```
```shell script
NAME READY STATUS RESTARTS AGE
api-server-5b57f68dc-dqfjk 3/3 Running 0 77s
discovery-5ffdfb9898-js9j2 1/1 Running 0 77s
extauth-6888f56db4-kgz2n 1/1 Running 0 77s
gateway-569488695f-zpqnj 1/1 Running 0 77s
gateway-proxy-9c954dc8-wt46j 1/1 Running 0 77s
gloo-5984f6f655-ct97s 1/1 Running 0 77s
glooe-grafana-78c6f96db-qwnd2 1/1 Running 0 77s
glooe-prometheus-kube-state-metrics-7f8fd8dd8-cfmqp 1/1 Running 0 77s
glooe-prometheus-server-6cc865559b-gl8wq 2/2 Running 0 76s
observability-6dd56c8468-xvwqc 1/1 Running 0 77s
rate-limit-c4fb9fc5b-6gm4s 1/1 Running 0 76s
redis-55d6dbb6b7-fg7wm 1/1 Running 0 77s
```
Once all of the pods are up and running you are all ready to configure your first WASM filter. The API to configure the filter can be found <a href=/gloo-edge/v1.13.x/reference/api/github.com/solo-io/gloo/projects/gloo/api/v1/options/wasm/wasm.proto.sk/#pluginsource>here</a>
.
* Example output in the `filter_chains` section:
```json
...
{
"name": "envoy.filters.http.wasm",
"typed_config": {
"@type": "type.googleapis.com/envoy.extensions.filters.http.wasm.v3.Wasm",
"config": {
"name": "myfilter",
"root_id": "add_header",
"vm_config": {
"vm_id": "gloo-vm-id",
"runtime": "envoy.wasm.runtime.v8",
"code": {
"remote": {
"http_uri": {
"uri": "http://gloo/images/ 8b3b05719379af3996d51bf6d5baed1103059fb908baec547f2136ed48aebd77" ,
"cluster": "wasm-cache",
"timeout": "5s"
},
"sha256": "8b3b05719379af3996d51bf6d5baed1103059fb908baec547f2136ed48aebd77"
}
},
"nack_on_code_cache_miss": true
},
"configuration": {
"@type": "type.googleapis.com/google.protobuf.StringValue",
"value": "my test config"
}
}
}
},
...
```
3. To check that the Wasm filter is applied, send a curl request to one of your endpoints.
```
curl -v $(glooctl proxy url)/all-pets
```
Example output: Notice the header that your Wasm filter adds.
<div class="highlight"><pre class="chroma"><code class="language-yaml" data-lang="yaml"><span class="w"> </span>*<span class="w"> </span><span class="l">TCP_NODELAY set</span><span class="w">
</span><span class="w"> </span>*<span class="w"> </span><span class="l">Connected to 34.30.251.229 (34.30.251.229) port 80 (#0)</span><span class="w">
</span><span class="w"> </span><span class="l">> GET /all-pets HTTP/1.1</span><span class="w">
</span><span class="w"> </span><span class="nt">> Host</span><span class="p">:</span><span class="w"> </span><span class="m">34.30.251.229</span><span class="w">
</span><span class="w"> </span><span class="nt">> User-Agent</span><span class="p">:</span><span class="w"> </span><span class="l">curl/7.64.1</span><span class="w">
</span><span class="w"> </span><span class="nt">> Accept</span><span class="p">:</span><span class="w"> </span><span class="cp">*/*</span><span class="w">
</span><span class="w"> </span><span class="l">> </span><span class="w">
</span><span class="w"> </span><span class="l">< HTTP/1.1 200 OK</span><span class="w">
</span><span class="w"> </span><span class="nt">< content-type</span><span class="p">:</span><span class="w"> </span><span class="l">text/xml</span><span class="w">
</span><span class="w"> </span><span class="nt">< date</span><span class="p">:</span><span class="w"> </span><span class="l">Thu, 02 Mar 2023 16:46:24 GMT</span><span class="w">
</span><span class="w"> </span><span class="nt">< content-length</span><span class="p">:</span><span class="w"> </span><span class="m">86</span><span class="w">
</span><span class="w"> </span><span class="nt">< x-envoy-upstream-service-time</span><span class="p">:</span><span class="w"> </span><span class="m">2</span><span class="w">
</span><span class="hl"><span class="w"> </span><span class="nt">< hello</span><span class="p">:</span><span class="w"> </span><span class="l">world!</span><span class="w">
</span></span><span class="w"> </span><span class="nt">< server</span><span class="p">:</span><span class="w"> </span><span class="l">envoy</span><span class="w">
</span><span class="w"> </span><span class="l">< </span><span class="w">
</span><span class="w"> </span><span class="p">[</span>{<span class="s2">"id"</span><span class="p">:</span><span class="m">1</span><span class="p">,</span><span class="s2">"name"</span><span class="p">:</span><span class="s2">"Dog"</span><span class="p">,</span><span class="s2">"status"</span><span class="p">:</span><span class="s2">"available"</span>}<span class="p">,</span>{<span class="s2">"id"</span><span class="p">:</span><span class="m">2</span><span class="p">,</span><span class="s2">"name"</span><span class="p">:</span><span class="s2">"Cat"</span><span class="p">,</span><span class="s2">"status"</span><span class="p">:</span><span class="s2">"pending"</span>}<span class="p">]</span><span class="w">
</span><span class="w"> </span></code></pre></div>