Append-Only Log

The pattern: never modify in place; only append new entries. The log is the source of truth, and current state is derived by replaying or aggregating it. Used in databases (WAL), message systems (Kafka), distributed storage (Iceberg/Delta data files), version control (Git), and event-sourced application architectures.

The trade-off: storage cost + replay time vs. simplicity + auditability. Append-only logs grow forever, so they need compaction or retention policies; replay to current state is slow without snapshots. The win is profound: every state change is recorded, atomic writes are easy, replication is “ship the log,” and time-travel queries are trivial.

Deepens in Year 1 Phase 3 (Postgres WAL on disk) and again in Year 3 Phase 15 where Iceberg + Parquet become the modern canonical example inside basecamp’s lakehouse tier.

Related patterns

• write-ahead-logging — the durability special case of the same idea.
• snapshot-plus-delta — what you pair with an append-only log to bound replay cost.
• stream-processing — Kafka topics are the canonical streaming substrate.
• delivery-semantics — the log’s offset model is what makes at-least-once and exactly-once tractable.
• replication — most replicated systems ship the log, not the state.