NetWasm 0.10: making C# on WebAssembly faster than Mono AOT

Where it started

NetWasm compiles C# ahead of time to WebAssembly: the compiler reads the IL Roslyn produces and emits Wasm, with its own runtime and a CoreLib ported from .NET. Mono's ahead-of-time WASI build is the obvious yardstick, because it is .NET on the same engine. 0.10.0 opened the release with a new garbage collector, which has its own post, and that put NetWasm ahead of Mono AOT on our allocation workloads. On plain loops it still lost: a quicksort of 100,000 integers took 77.2 ms against Mono AOT's 58.0.

Then DoomSharp, David Fowler's C# software renderer for Doom, which ships a NetWasm build, was reported to run on 0.10.1 at half the speed of .NET's NativeAOT. Our own measurement on 0.10.2 agreed in direction: 120 renders took 3.7 s at O3, Mono AOT 2.9 s, desktop .NET about 1.3 s.

This post is about the compiler work through the rest of 0.10, from 0.10.1 to 0.10.8, that changed those numbers. Every change below is a general transformation of IL or of NetWasm's own output, measured on the workload that exposed it and on a broad benchmark table before it shipped. None of them looks for a particular program.

Measuring on Wasmtime

Everything here runs on Wasmtime 49.0.3 on an Apple M1 Max, with each module compiled ahead of time so compilation is not timed. The broad table is dotnet/performance's runtime micro-benchmarks plus its vector and span library benchmarks, and a few workloads of our own: 2,213 cases, each compiled from the same C# by NetWasm, by Mono AOT on .NET 11 RC3 with Wasm SIMD on, and by desktop .NET 11 RC3, whose JIT runs fully optimized from the start.

Comparing two NetWasm builds turned out to need care. Identical loops at different code addresses differed by up to 15% on the M1, so an unrelated change that moved a hot loop looked like a speedup or a regression. Our A/B runs now align every function on a 4 KiB boundary (-C cranelift-log2_min_function_alignment=12; 128 bytes still left a byte-identical loop 1.37 times slower, because branch prediction indexes higher address bits) and alternate the two builds over several rounds. A result that does not survive that is not counted.

Profiling needed the same discipline. Wasmtime's guest profiler samples a module's functions, but a release module has no names, and function indices change between builds. We profile the timed module and take the names from a twin built with names kept, after checking that the two are the same code once the names are stripped, then cut each hot path into a small standalone kernel that runs on all three runtimes in seconds. Most fixes started as a kernel where NetWasm was clearly behind desktop .NET.

Exceptions that never happen

The first large win was in code that never throws. Wasmtime saves every callee-saved register, floating point included, around any function that contains a throw or a try_table, because it treats the calls inside as clobbering them. A C# method that can throw through a null check, an array bound or checked arithmetic held its throw inline, and every method with a root frame wrapped its body in a handler that only released the frame on the way out. So most methods paid for exception handling on every call.

In 0.10.1 a method that can throw calls a shared throw function instead, and frames are released by the handler that catches the exception, the way a native stack pointer snaps back. On our allocation workloads that took the full transaction from 43.9 to 34.9 ms, the retained object graph from 8.8 to 4.6 ms and UI churn from 17.6 to 7.9 ms.

Array checks inside loops

The quicksort was the opposite problem: a tight loop of array reads, swaps and comparisons, where every access paid a null check and a bounds check, and every swap was a call. 0.10.2 inlined small straight-line methods called inside loops, such as a tuple swap. An inlined call now reads its arguments where the caller holds them instead of copying them, so the array checks see the caller's array and can use what is already known about it. An access skips its checks when an earlier access to the same array and index made them, and nested loops stop reloading each array's length and data. The quicksort went from 77.2 to 54.0 ms, ahead of Mono AOT's 58.0.

DoomSharp

DoomSharp renders Doom in C# with no graphics API: a span of pixels, filled column by column. Our harness renders the opening view of E1M1, the scene DoomSharp's own browser benchmark uses, 120 times, from a WAD reduced to the lumps that render reads. Profiling it found problems no micro-benchmark had shown.

• Struct copies per pixel. The rasterizer's edge function, called twice per pixel, took two 40-byte structs by value and read them through property getters. The getters kept it from being a straight-line leaf, so it stayed a call, and each call copied both structs. Reading value-type accessors as field loads made it a leaf the loop inlines, and an argument the callee only reads is now read from the caller's copy. The render went from 3,595 to 2,455 ms.
• A branching helper in the pixel loop. Shading clamps a light factor through a 49-instruction helper with branches. .NET's JIT inlines callees of up to about 128 IL bytes at a hot call site; NetWasm only inlined straight-line ones. At O3 it now also inlines a small branching callee inside a loop when it has no loop, allocation or throw of its own: 2,223 to 1,934 ms. Math.Clamp itself now inlines as it does in .NET, with its range check calling a throw helper.
• Library code the renderer leans on. Lump lookup uppercases every name it compares, and sprite names are built with string interpolation. Invariant casing now skips unchanged ASCII four characters at a time, as .NET does, and interpolated strings format into one pooled buffer through .NET's own handler rather than through a StringBuilder and a boxed value per hole.

None of these is specific to Doom. The struct accessor and inlining changes show up across the micro-benchmark table, and the casing and formatting changes are CoreLib code every program shares.

SIMD and vectorized loops

0.10.7 turns on WebAssembly SIMD. Vector128.IsHardwareAccelerated and Vector.IsHardwareAccelerated are true, and Vector128<T>, Vector<T> and the PackedSimd intrinsics lower to Wasm's 128-bit instructions. That alone does little if vectors live in memory, so a vector value now has its own v128 local and a slot on the compiler's evaluation stack, and touches memory only where C# puts it there. CoreLib's span, search, ASCII, UTF-8, Base64 and numerics code is .NET 11's own, so it takes the vectorized paths .NET wrote for it.

The compiler also vectorizes ordinary loops. An element-wise loop over arrays, like c[i] = a[i] * k + b[i], runs four or more elements per iteration behind a guard that every access is in bounds and that the arrays cannot overlap, and finishes with the original loop. A loop that sums an array, or takes its minimum or maximum, becomes one call to a vectorized fold. A pure Math call of values the loop never changes, such as Math.Pow(nu * dt / dx, 2.0) in a Burgers' equation solver, is computed once before the loop, which is what let that loop vectorize. The Burgers solver's first test went from 497 ms in 0.10.6 to 71 ms; Mono AOT takes 186 and desktop .NET 93.

The matrix types came last. Transposing, inverting or multiplying a Matrix4x4 shuffles lanes between rows, and those shuffles went through Vector64 halves in software, so the matrix benchmarks took 1.55 times Mono AOT's time. Each shuffle now compiles to one Wasm i8x16.shuffle, and the family takes 0.80 of Mono AOT's time.

Divisions and their checks

The last compiler change of the release came from an issue report: a search for the largest prime below 10,000,000 by trial division ran slower than Mono AOT. Its loop computed value / divisor and value % divisor, and NetWasm divided twice, each time behind a check for a zero divisor and one for int.MinValue / -1. In 0.10.8 a remainder of the values a division just divided takes that division's quotient, value - quotient * divisor, with no checks of its own, since the division made them.

The compiler also follows the ranges that integer locals and arguments can hold, through arithmetic, comparisons and loops, and leaves out the checks those ranges rule out. In the search, the divisor starts at 3 and only grows, so its division needs no zero check, and the method has already returned for values below 2, so the dividend cannot be int.MinValue. The same analysis drops overflow checks from checked arithmetic whose result always fits and from checked conversions whose value always fits.

That is the issue's reproducer, medians of seven runs. With both operators NetWasm is now ahead of Mono AOT and desktop .NET; with one division per step desktop .NET is still a little ahead.

Where it stands

Across the 2,213 cases that every runtime runs, NetWasm 0.10.8 at O3 with NetWasmGC takes 0.41 of Mono AOT's time, as a geometric mean that leaves out five cases whose work Mono AOT's LLVM deletes (0.43 with them). On the 464 runtime benchmarks that 0.10.6 also runs, it takes 0.40 of 0.10.6's time: 0.10.6 took 1.20 times Mono AOT's time there and 2.15 times desktop .NET's, and 0.10.8 takes 0.48 and 0.87.

Lower is faster. Some single cases, in nanoseconds or milliseconds per operation:

DoomSharp, timed the same way as before, now renders its 120 frames in 1,623 ms with NetWasmGC and 1,781 ms with Compact, against 3,638 ms with NetWasm 0.10.6 and 2,371 ms with Mono AOT. Desktop .NET 11 takes 1,176 ms with NativeAOT and 1,053 ms with its JIT. That is the median of seven fresh processes per build, each timing a second batch of 120 renders after a first.

Every case runs once per runtime, compiled from the same C#: NetWasm at O3 with NetWasmGC, Mono AOT on .NET 11 RC3 with Wasm SIMD, both on Wasmtime 49.0.3, and desktop .NET 11 RC3's JIT without tiering, so it runs optimized code from the start. The three ran interleaved, in chunks of 50 cases, keeping each case's fastest of two rounds, and NetWasm 0.10.6 was timed on the same machine five days earlier. The performance page has whole-program workloads with ranges and the command that reproduces them.

What is still slower

Span searches were the clearest gap until the end of the round, when .NET's own MemoryExtensions and SpanHelpers replaced NetWasm's scalar loops: the family went from 1.62 times Mono AOT's time to 0.59, and to 0.53 once 0.10.8 brought .NET 11's comparisons: IndexOf with OrdinalIgnoreCase or InvariantCulture now takes 0.6 to 1.1 times Mono AOT's time. What remains is a handful of cases: block copies through a local's address, with CopyBlock64 at 3.3 times; Matrix4x4's CreateWorld and CreateLookAt at 1.85 and 1.41; SeekUnroll at up to 1.5; and a few strength-reduction loops that sum small integers over spans, at up to 1.2.

Desktop .NET runs the vector benchmarks about 3.1 times as fast as NetWasm, and 8.0 times as fast as Mono AOT. It compiles straight to the host's SIMD instructions, where Wasm code has fixed 128-bit operations and an engine between it and the CPU. Math takes 0.81 of Mono AOT's time: Cbrt and SinCos take less time than Mono AOT's, though Math.Pow still takes 1.4 times as long.

A few benchmarks run slower than in 0.10.6 although NetWasm's output for them is better. In NewtR, a Newton-Raphson loop, the value being refined used to live in linear memory and is now a Wasm local. Wasmtime's register allocator keeps that local in a stack slot around the loop because it is still needed after a later call, and the store and reload on each iteration make the loop take 1.9 times as long. Mono AOT's build of the same loop runs at the same speed as ours. This is the kind of result that only shows when you look at the native code, which is why every change here was measured on a table this size.