The Quipu-Log Book

Tamper-evident audit logs
and a filesystem storage engine

Storage engines and tamper-evident audit logs, built on nothing but the filesystem — explained from the ground up — build intuition from analogies, confirm against real code, and see how database-grade guarantees are rebuilt on nothing but plain files.

Part 1 · Starting point: files, not a DB

01What an audit log is, and why “files, no DB” 02The map, and the DB↔filesystem correspondence

Part 2 · Filesystem basics

03Filesystem basics: files, directories, inodes, descriptors 04Reading & writing, and the atomicity of rename 05When data really hits disk: the page cache and fsync 06The std::fs toolbox: handling files in Rust

Part 3 · The heart of the engine: the append-only log

07Append-only log = the WAL is the database 08Segment files and rollover 09Record framing: length, CRC32, magic/version 10Serialization: turning structs into bytes

Part 4 · Re-creating what a DB gave you for free, on files

11Durability: fsync policy and group commit 12Atomicity and crash recovery: torn writes and tail detection 13Concurrency I: single-writer and the file lock 14Concurrency II: read snapshots and MVCC 15Indexing: in-memory index + on-disk tokens 16Query execution: segment scans and cursor pagination 17Deletion and retention: segment unlink vs DELETE 18Store layout and format versioning

Part 5 · Integrity: making tampering evident (Security I)

19Hash functions and SHA-256: a digital fingerprint 20The Merkle history tree: committing the log to 32 bytes 21Inclusion proofs and consistency proofs 22Checkpoints and external anchoring 23Tamper-evident vs tamper-proof, and the threat model

Part 6 · Confidentiality: searching while keeping secrets (Security II)

24Field protection in four levels: plaintext/SHA-256/HMAC/RSA 25Symmetric, asymmetric, hybrid encryption and AEAD 26Blind indexes: searching without plaintext 27Write-only deployment: a server without the private key 28Key management: keyring, versioning, rotation vs re-keying

Part 7 · The write & read paths

29The async pipeline: non-blocking emit and backpressure 30Reliability: retries, backoff, DLQ, idempotency 31The tower middleware: auto-recording HTTP requests 32Permissions (RBAC), filters, and meta-audit

Part 8 · Distribution, operations, scaling

33Single point of failure and availability: the client spool 34Horizontal scaling: sharding, consistent hashing, read replicas 35Server, client, MCP, and observability