06 · The std::fs toolbox: handling files in Rust

Quipu-Log's entire storage engine runs on nothing but Rust's standard library — std::fs and std::io — with no special OS features and no external libraries. This chapter collects the standard APIs that actually appear throughout the Quipu-Log codebase and looks at them from the angle of "this is the full set of tools we have."

Opening files: File and OpenOptions

The most fundamental file operations are open and close. In Rust, std::fs::File is the file handle and OpenOptions controls how it's opened.

crates/quipu-core/src/storage/segment.rs — opening filesuse std::fs::{File, OpenOptions};

// write-capable open (segment write path)
let file = OpenOptions::new()
    .create(true).truncate(false).read(true).write(true)
    .open(path)?;

// read-only open (skim, snapshot reads)
let file = File::open(path)?;  // read-only shorthand

A File is automatically closed when dropped — no explicit close() needed, thanks to Rust's RAII.

The I/O traits: Read, Write, Seek

Three core traits do the actual data transfer.

crates/quipu-core/src/storage/segment.rs — writing a frameuse std::io::{BufReader, BufWriter, Read, Seek, SeekFrom, Write};

// frame header 4+4+8 = 16 bytes, then the payload
self.writer.write_all(&(payload.len() as u32).to_le_bytes())?;
self.writer.write_all(&crc.to_le_bytes())?;
self.writer.write_all(&timestamp.to_le_bytes())?;
self.writer.write_all(payload)?;

Buffered I/O: BufReader and BufWriter

Calling write() directly on a File triggers a system call every time. Frame header: 4 bytes, CRC: 4 bytes, timestamp: 8 bytes — that's three syscalls for just the header. BufWriter accumulates data in an internal buffer (256 KB) and issues a single syscall when it flushes.

The same principle applies to reads. BufReader reads a large chunk from disk at once, and read_exact() then pulls from that buffer:

crates/quipu-core/src/storage/segment.rs — skim() readslet mut reader = BufReader::with_capacity(256 * 1024, file);
let mut header = [0u8; FRAME_HEADER];
reader.read_exact(&mut header)?;   // pull 16 bytes from the buffer

File and directory operations: the full list actually used

Here are all the std::fs functions used in Quipu-Log's storage module. This is the complete toolbox:

File locking: File::try_lock (MSRV 1.89)

Quipu-Log enforces a single-writer principle. If a second process tries to open the same root directory as a store, the in-memory index becomes inconsistent and segment writes collide. An OS advisory file lock is what prevents that.

crates/quipu-core/src/store.rs — AuditStore::open()let lock = std::fs::OpenOptions::new()
    .create(true).truncate(false).write(true)
    .open(cfg.root.join("LOCK"))?;

lock.try_lock().map_err(|e| match e {
    std::fs::TryLockError::WouldBlock => Error::Locked(cfg.root.display().to_string()),
    std::fs::TryLockError::Error(e) => Error::Io(e),
})?;

File::try_lock() was stabilized in Rust 1.89. That's why Quipu-Log's MSRV (minimum supported Rust version) is 1.89. The reason is spelled out in Cargo.toml:

Cargo.toml# MSRV floor: the store guards its root with `std::fs::File::try_lock` and
# matches on `std::fs::TryLockError`, both stabilized in Rust 1.89.
rust-version = "1.89"

A note on advisory locks: they have no effect within a single process — they only prevent conflicts between processes. When the process dies, the OS releases the lock automatically (the fd is closed). There's something elegant about the lock file: it doesn't need even a single byte of content — just opening it and calling try_lock is enough.