← All posts

We Built a POSIX Conformance Suite in Rust. It Found Kernel Bugs on the First Run.

rustposixrtosconformancetesting

There is no open-source, Rust-native POSIX conformance test suite. LTP is Linux-specific. VSX is proprietary. If you're building an RTOS or microkernel with a POSIX compatibility layer, you have nothing to test it against.

So we built one. 362 tests, covering 52 x86-64 syscalls across memory management, signals, sockets, pipes, timers, I/O multiplexing, and process identity. The binary is 22KB, uses no libc, no standard library, and makes syscalls directly via inline assembly. It runs on Linux as a reference and on any POSIX-compatible RTOS without modification.

The same binary, two targets

The key design decision: the conformance binary is a static, #![no_std], #![no_main] ELF that uses raw syscall instructions. No libc abstraction layer, no runtime. It talks directly to whatever kernel is underneath.

Build it once. Run it on Linux — that's your reference. Run the same binary on your RTOS — the delta between the two outputs is your conformance gap. Linux is the oracle.

What it found

We developed this suite for uKernel, a Rust microkernel with a POSIX compatibility layer. On the first run against the kernel, the suite found three bugs:

  1. Kernel panic on negative file descriptors. Calling dup2(1, -1) caused an index-out-of-bounds crash. The fd table didn't validate negative values, resulting in a u64::MAX index.
  2. nanosleep blocking for 584 years. Passing a negative tv_sec to nanosleep should return EINVAL. Instead, the kernel interpreted it as an unsigned value and blocked. The negative test caught it immediately.
  3. Pipe buffer resource leak. Closed pipes weren't freeing their buffer slots. After 8 pipe operations, every subsequent pipe2() returned EMFILE, silently skipping ~400 tests.

All three were one-line fixes. Without the conformance suite, they would have surfaced as mysterious failures in real applications — musl crashing on startup, Kanidm hanging on initialization, or fd exhaustion under load.

Output

Here's what the suite looks like running on uKernel (AMD SVM, QEMU):

=== TLS: arch_prctl + FS-relative access ===
  [PASS] arch_prctl(SET_FS) returns 0
  [PASS] GET_FS returns SET_FS value
  [PASS] fs:[0] reads correct value
  [PASS] fs:[8] reads correct value
  [PASS] fs:[16] reads correct value
  [PASS] fs:[24] reads correct value
  [PASS] fs:[0] write modifies TLS block

=== Nanosleep: negative tests ===
  [PASS] nanosleep: negative tv_sec returns EINVAL
  [PASS] nanosleep: negative tv_nsec returns EINVAL
  [PASS] nanosleep: tv_nsec >= 1e9 returns EINVAL
  [PASS] nanosleep: very large tv_nsec returns EINVAL

=== Epoll: positive tests ===
  [PASS] epoll_create1: basic
  [PASS] epoll_ctl: ADD
  [PASS] epoll_wait: EPOLLIN after write
  [PASS] epoll_ctl: DEL
  [PASS] epoll_ctl: EPOLLET

=== Poll: positive tests ===
  [PASS] poll: pipe read ready after write (POLLIN)
  [PASS] poll: multiple fds returns count
  [PASS] poll: infinite timeout returns when ready

════════════════════════════════════════════════════════════
SUMMARY: 342 passed, 20 failed
════════════════════════════════════════════════════════════

Output from uKernel running on AMD SVM (QEMU). 342 of 362 tests passing. The 20 failures are known implementation gaps — not regressions.

Test methodology

Every syscall gets three categories of tests:

  • Positive: Normal usage with expected return values. Not just "did it return 0" — the tests exercise the API semantically. TLS tests write through fs:[offset]. Memory tests write patterns and read them back. Pipe tests verify data integrity end-to-end.
  • Negative: Invalid arguments with specific errno verification. Not "did it fail" but "did it fail with exactly EINVAL." A test that accepts any negative return value is a test that can't find bugs.
  • Boundary: Zero-length operations, maximum values, alignment edge cases. These are the tests that find the off-by-one errors and the unsigned/signed confusion bugs.

Standards

The suite targets the POSIX Standard Environment (PSE) profiles defined in IEEE 1003.13-2003:

  • PSE51 — Minimal Realtime: memory, signals, clocks
  • PSE52 — Realtime Controller: adds process management, IPC
  • PSE53 — Dedicated Realtime: adds networking, I/O multiplexing, filesystem

Coverage is not complete for any profile. The suite is a work in progress — contributions are welcome.

Get it

The source is available under the MIT license:

git clone https://gitlab.com/vinci-consulting/posix-conformance.git
cd posix-conformance
cargo build --release
./target/release/posix-conformance

362 tests. Zero dependencies. If your RTOS has a POSIX layer, this will tell you where the gaps are.