Claude wrote a C SQLite extension that ran 2x faster than Rust

Simon Willison asked Claude Opus 5.5 to build a SQLite extension. The extension adds a parse_number() function. Claude wrote four versions, and Willison benchmarked them on 2 million rows. He posted the results on his blog.

The optimised C version ran at 19.2 nanoseconds per row on mixed data. The two Rust versions ran at 40.5 and 45.9 nanoseconds. A plain C version, written first, ran at 67.3 nanoseconds. It was slower than both Rust builds.

What the function does

The function strips ,, $, £ and %, handles a leading -, and returns NULL for anything it cannot parse. Willison says he was out walking the dog when he thought of it. He sent the idea to Claude Opus 5.5 in a regular chat. He asked it to compare a C SQLite extension and a Rust one.

Claude wrote four versions: plain C, optimised C, a Rust version on rusqlite, and a Rust version on sqlite-loadable-rs. It also wrote the test harness and a benchmark script. The tests include 40 hand-written cases and a fuzzer that ran 2 million random inputs. Willison says his role was to steer the inquiry and read the results.

The numbers

On mixed input data, the medians across the four builds and plain SQL were:

• Optimised C: 19.2 ns per row
• sqlite-loadable-rs: 40.5 ns per row
• rusqlite: 45.9 ns per row
• Plain C: 67.3 ns per row
• Pure SQL replacement: 328.9 ns per row

Decimal-only input is where the first round was most lopsided. Plain C ran at 106 ns per row there, because it leaned on strtod(). The Rust versions ran at 47 and 55 ns. After Willison asked Claude to optimise the C version with Clinger's fast path, the same data ran at 18 ns. The eight-million-row combined test and fuzz run produced zero mismatches across all four builds.

The binaries and the code

The stripped C extension is 14 KB. The two Rust extensions are 298 KB and 321 KB. The gap comes from the Rust standard library being linked into each extension.

The code counts went the other way. The plain C version is 75 lines. The optimised C version is 104 lines. The two Rust versions are 30 and 36 lines each, with a 41-line shared parser.

One prompt moved the benchmark from Rust winning to C winning by about 2x. The task, the model and the person at the keyboard stayed the same. Willison asks whether the first pass is the measurement. His answer is no. A benchmark that stops at the first version reports how well the model guesses at an implementation. It leaves the real speed of the language untested.

The research report, the benchmarks and the four builds are at simonw/research on GitHub.

Source

Simon Willison: parse_number() as a SQLite extension: C vs Rust