QA Wolf runs its Windows desktop tests on disposable Google Cloud VMs

End-to-end tests on a Windows desktop app need a clean machine, and a clean machine takes two to three minutes to boot. QA Wolf, a managed QA company, published an engineering write-up this week of how it runs these tests at scale: a warm pool of Google Compute Engine VMs that each live for one test and then die, with the test code itself running on Linux. The piece reached the Hacker News front page the same day with 79 points.

The key design choice is to treat each Windows VM like an Android emulator: nobody logs in, no test reuses a machine, and the pool replaces each machine as it is used up.

How the pool works

The device scheduler, which already hands out Android emulators to tests, now also hands out Windows VMs. It tracks the fleet through three Kubernetes custom resources: a WindowsVMTemplate says what a VM looks like, a WindowsVMPool says how many ready VMs to hold for a given template, and a WindowsVM is one machine. A reconciler compares the records against what is running on Google Compute Engine and closes the gap: if a pool is three short, it creates three; when a test releases its claim, that VM is destroyed and a fresh one is built in its place.

Four properties fall out of that design, and the write-up names each one. A VM is never reused, so no test can see another test's files or logins and the "passes on its own, fails in the suite" class of failure never comes up. Warm pools absorb the two-to-three-minute cold start. If a claimed VM goes quiet for 60 seconds, the controller assumes the test is gone and reclaims the machine. If one cloud zone cannot supply a VM, the pool creates it in another zone.

What the test sees

Test code never runs on Windows. It runs in a Linux pod and talks to a claimed VM over three connections: Playwright over CDP for browsers, Appium for native Windows apps, and a WebRTC screen stream for live viewing and recording. A single API hides the split, so one test can sign in through a browser, switch to a native app, and check the result without naming which protocol carried each step.

The desktop agent on the VM is written in Go with no CGo, so the whole binary cross-compiles and the Mac and Linux builds are unaffected by the Windows code. It captures the screen with FFmpeg, drives mouse, keyboard and clipboard through Windows' own user32.dll, and exposes diagnostic endpoints that report what the capture pipeline and the WebRTC link are doing. A PowerShell setup script produces the base machine image and publishes it to a GCE image family, so an upgrade happens by attrition: old VMs are used up and idle VMs on an outdated image get recycled.

The shape, if your tests have to touch Windows

QA Wolf's own Windows product is paid, so the write-up is marketing as well as reporting. Keep that in mind on the "signed URL only our API can sign" and "no stored cloud keys" lines, which are features of QA Wolf's build and not generic claims. The architecture itself is the shape most CI systems that have to touch Windows will keep arriving at: a Kubernetes-managed warm pool, single-use VMs, a Linux runner driving the Windows machine over established protocols, and screen capture that treats the remote desktop as a first-class artefact.

Source

• Primary: QA Wolf, "How We Solved Windows Desktop Automation"
• Hacker News discussion, 79 points