# Trying out WebAssembly on Azure Kubernetes Service - URL: https://baeke.info/2021/10/18/trying-out-webassembly-on-azure-kubernetes-service/ - Author: Geert Baeke - Published: October 18, 2021 - Topics: cloud, containers, dev, golang, kubernetes, rust, go, kubernetes, wagi, wasi, wasm, webassembly - Outdated: WASI node pools on AKS (based on Krustlet) were retired. ## Introduction In October 2021, Microsoft announced the public preview of AKS support for deploying WebAssembly System Interface (WASI) workloads in Kubernetes. You can read the announcement [here](https://azure.microsoft.com/en-us/updates/public-preview-aks-support-for-webassembly-system-interface-wasi-workloads/). In short, that means we can run another type of workload on Kubernetes, besides containers! WebAssembly is maybe best known for the ability to write code with languages such as C#, Go and Rust that can run in the browser, alongside JavaScript code. One example of this is [Blazor](https://dotnet.microsoft.com/apps/aspnet/web-apps/blazor), which allows you to build client web apps with C#. Besides the browser, there are ways to run WebAssembly modules directly on the operating system. Because WebAssembly modules do not contain machine code suitable for a specific operating system and CPU architecture, you will need a runtime that can interpret the WebAssembly [byte code](https://en.wikipedia.org/wiki/Bytecode). At the same time, WebAssembly modules should be able to interface with the operating system, for instance to access files. In other words, WebAssembly code should be able to access specific parts of the operating system outside the sandbox it is running in by default. The WebAssembly System Interface (or [WASI](https://wasi.dev/)) allows WebAssembly modules to interact with the **outside world**. It allows you to declare what the module is allowed to see and access. One example of a standalone runtime that can run WebAssembly modules is [wasmtime](https://github.com/bytecodealliance/wasmtime). It supports interacting with the host environment via [WASI](https://wasi.dev/) as discussed above. For example, you can specify access to files on the host via the --dir flag and be very specific about what files and folders are allowed. ## An example with Rust In what follows, we will create Hello World-style application with [Rust](https://www.rust-lang.org/). You do not have to know anything about Rust to follow along. As a matter of fact, I do not know that much about Rust either. I just want a simple app to run on Azure Kubernetes Service later. Here’s the source code: ```text use std::env; fn main() { println!("Content-Type: text/plain\n"); println!("Hello, world!"); printenv(); } fn printenv() { for (key, value) in env::vars() { println!("{}: {}", key, value); } } ``` **Note:** Because I am a bit more comfortable with Go, I first created a demo app with Go and used TinyGo to build the WebAssembly module. That worked great with wasmtime but did not work well on AKS. There is probably a good explanation for that. I will update this post when I learn more. To continue with the Rust application, it is pretty clear what it does: it prints the Content-Type for a HTTP response, a Hello, World! message, and all environment variables. Why we set the Content-Type will become clearer later on! To build this app, we need to target **wasm32-wasi** to build a WebAssembly module that supports WASI as well. You can run the following commands to do so (requires that Rust is installed on your system): ``` rustup target add wasm32-wasi cargo build --release --target wasm32-wasi ``` The **rustup** command should only be run once. It adds wasm32-wasi as a supported target. The **cargo build** command then builds the WebAssembly module. On my system, that results in a file in the target/wasm32-wasi/release folder called **sample.wasm** (name comes from a setting in [cargo.toml)](https://doc.rust-lang.org/cargo/reference/manifest.html) . With WebAssembly support in VS Code, I can right click the file and use **Show WebAssembly**: ![](https://baeke.info/_astro/2021-10-image.C0NrTyCE_Z2r2nVn.webp) *Showing the WebAssembly Module in VS Code (WebAssembly Toolkit for VS Code extension)* We can run this module with **cargo run** but that runs the app directly on the operating system. In my case that’s Ubuntu in Windows 11’s WSL2. To run the WebAssembly module , you can use **wasmtime**: ``` wasmtime sample.wasm ``` The module will not read the environment variables from the host. Instead, you pass environment variables from the **wasmtime** cli like so (command and result shown below): ``` wasmtime --env test=hello sample.wasm Content-Type: text/plain Hello, world! test: hello ``` ## Publishing to Azure Container Registry A WebAssembly can be published to Azure Container Registry with **wasm-to-oci** (see [GitHub repo](https://github.com/engineerd/wasm-to-oci)). The command below publishes our module: ``` wasm-to-oci push sample.wasm .azurecr.io/sample:1.0.0 ``` Make sure you are logged in to ACR with **az acr login -n <ACRNAME>**. I also enabled [anonymous pull](https://docs.microsoft.com/en-us/azure/container-registry/anonymous-pull-access) on ACR to not run into issues with pulls from WASI-enabled AKS pools later. Indeed, AKS will be able to pull these artefacts to run them on a WASI node. Here is the artefact as shown in ACR: ![](https://baeke.info/_astro/2021-10-image-1.DJzyVFga_Z1m3gs5.webp) *WASM module in ACR with mediaType = application/vnd.wasm.content.layer.v1+wasm* ## Running the module on AKS To run WebAssembly modules on AKS nodes, you need to enable the preview as described [here](https://docs.microsoft.com/en-us/azure/aks/use-wasi-node-pools). After enabling the preview, I deployed a basic Kubernetes cluster with one node. It uses kubenet by default. That’s good because Azure CNI is not supported by WASI node pools. ```bash az aks create -n wademo -g rg-aks --node-count 1 ``` After finishing the deployment, I added a WASI nodepool: ```bash az aks nodepool add \ --resource-group rg-aks \ --cluster-name wademo \ --name wasipool \ --node-count 1 \ --workload-runtime wasmwasi ``` The **aks-preview** extension (install or update it!!!) for the Azure CLI supports the **--workload-runtime** flag. It can be set to **wasmwasi** to deploy nodes that can execute WebAssembly modules. The piece of technology that enables this is the **krustlet** project as described here: [https://krustlet.dev](https://krustlet.dev/). Krustlet is basically a WebAssembly kubelet. It stands for Kubernetes Rust Kubelet. After running the above commands, the command **kubectl get nodes -o wide** will look like below: ``` NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME aks-nodepool1-23291395-vmss000000 Ready agent 3h6m v1.20.9 10.240.0.4 Ubuntu 18.04.6 LTS 5.4.0-1059-azure containerd://1.4.9+azure aks-wasipool-23291395-vmss000000 Ready agent 3h2m 1.0.0-alpha.1 10.240.0.5 mvp ``` As you can see it’s early days here! 😉 But we do have a node that can run WebAssembly! Let’s try to run our module by deploying a pod via the manifest below: ```yaml apiVersion: v1 kind: Pod metadata: name: sample annotations: alpha.wagi.krustlet.dev/default-host: "0.0.0.0:3001" alpha.wagi.krustlet.dev/modules: | { "sample": {"route": "/"} } spec: hostNetwork: true containers: - name: sample image: .azurecr.io/sample:1.0.0 imagePullPolicy: Always nodeSelector: kubernetes.io/arch: wasm32-wagi tolerations: - key: "node.kubernetes.io/network-unavailable" operator: "Exists" effect: "NoSchedule" - key: "kubernetes.io/arch" operator: "Equal" value: "wasm32-wagi" effect: "NoExecute" - key: "kubernetes.io/arch" operator: "Equal" value: "wasm32-wagi" effect: "NoSchedule" ``` Wait a moment! There is a new acronym here: **WAGI**! WASI has no network primitives such as sockets so you should not expect to build a full webserver with it. WAGI, which stands for **WebAssembly Gateway Interface**, allows you to run WASI modules as HTTP handlers. It is heavily based on [CGI](https://en.wikipedia.org/wiki/Common_Gateway_Interface), the **Common Gateway Interface** that allows mapping HTTP requests to executables (e.g. a Windows or Linux executable) via something like IIS or Apache. We will need a way to map a route such as / to a module and the response to a requests should be HTTP responses. That is why we set the Content-Type in the example by simply printing it to stdout. WAGI will also set several environment variables with information about the incoming request. That is the reason we print all the environment variables. This feels a bit like the early 90’s to me when CGI was the hottest web tech in town! 😂 The mapping of routes to modules is done via annotations, as shown in the YAML. This is similar to the [modules.toml](https://github.com/deislabs/wagi/blob/main/examples/modules.toml) file used to start a Wagi server manually. Because the WASI nodes are tainted, tolerations are used to allow the pod to be scheduled on such nodes. With the **nodeSelector**, the pod needs to be scheduled on such a node. To run the WebAssembly module, apply the manifest above to the cluster as usual (assuming the manifest is in pod.yaml: ```bash kubectl apply -f pod.yaml ``` Now run **kubectl get pods**. If the status is **Registered** vs **Running**, this is expected. The pod will not be ready either: ``` NAME READY STATUS RESTARTS AGE sample 0/1 Registered 0 108m ``` In order to reach the workload from the Internet, you need to install nginx with a value.yaml file that contains the internal IP address of the WASI node as documented [here](https://docs.microsoft.com/en-us/azure/aks/use-wasi-node-pools#running-wasmwasi-workload). After doing that, I can curl the public IP address of the nginx service of type LoadBalancer: ``` ~ curl IP Hello, world! HTTP_ACCEPT: */* QUERY_STRING: SERVER_PROTOCOL: HTTP/1.0 GATEWAY_INTERFACE: CGI/1.1 REQUEST_METHOD: GET SERVER_PORT: 3001 REMOTE_ADDR: 10.240.0.4 X_FULL_URL: http://10.240.0.5:3001/ X_RAW_PATH_INFO: CONTENT_TYPE: SERVER_NAME: 10.240.0.5 SCRIPT_NAME: / AUTH_TYPE: PATH_TRANSLATED: PATH_INFO: CONTENT_LENGTH: 0 X_MATCHED_ROUTE: / REMOTE_HOST: 10.240.0.4 REMOTE_USER: SERVER_SOFTWARE: WAGI/1 HTTP_HOST: 10.240.0.5:3001 HTTP_USER_AGENT: curl/7.58.0 ``` As you can see, WAGI has set environment variables that allows your handler to know more about the incoming request such as the HTTP User Agent. ## Conclusion Although WebAssembly is gaining in popularity to build browser-based applications, it is still early days for running these workloads on Kubernetes. WebAssembly will not replace containers anytime soon. In fact, that is not the actual goal. It just provides an additional choice that might make sense for some applications in the future. And as always, the future will arrive sooner than expected!