Storage Engine

💾 Storage Architecture

Stackhouse does not implement its own storage engine (no custom WAL/MemTable/SSTable stack exists in this codebase). It is a Rust/Axum service that stores all data in PostgreSQL via sqlx — durability, WAL, compaction, MVCC, and caching are all delegated to Postgres itself.

Architecture Overview

Components

1. Stackhouse-Store (connection pool)

Location: stackhouse/src/platform/db.rs (StackhouseStore)

A thin wrapper around sqlx::PgPool (PgPoolOptions, default 20 max connections, 3s acquire timeout). Provides execute, query, query_simple, insert_returning_id, execute_batch, and simple insert/scan/delete helpers used throughout the API layer. Test isolation uses a fresh CREATE SCHEMA IF NOT EXISTS stackhouse_test_<timestamp>_<counter> per test run, set via search_path, not an in-memory database.

2. Schema-Later Guard (automatic schema evolution)

Location: stackhouse/src/security/guard.rs

Caches known table schemas in a DashMap, diffs incoming JSON payload keys against information_schema on cache miss, validates new keys as safe SQL identifiers (rejecting reserved keywords), and issues ALTER TABLE ... ADD COLUMN IF NOT EXISTS for new fields — capped at 1000 columns per table. See Schema Evolution for the full type-inference mapping.

3. Schema migrations (versioned, developer-authored)

Location: stackhouse/src/db/schema_migrations.rs

A separate system from the Schema-Later Guard above: tracks up_sql/down_sql migrations with SHA-256 checksums in a stackhouse_schema_migrations table, supports migrate(), rollback_one(), rollback_to(version), and checksum verification. This does not run automatically on writes — it's for developer-driven schema changes.

4. Object/blob storage

Location: stackhouse/src/storage/ (mod.rs, acl.rs, cdn.rs, lifecycle.rs, s3_compat.rs, tus.rs, versioning.rs, explorer.rs, scanning.rs)

A separate subsystem for file storage (buckets, objects, S3-compatible API, resumable uploads via the tus protocol, lifecycle policies, ACLs, CDN integration, content scanning). Metadata is tracked in Postgres tables (stackhouse_buckets, stackhouse_objects) — the backing store is the same Postgres pool used everywhere else. See Storage for the full API.

Performance

No throughput/latency numbers are published for this layer — actual performance is governed by the underlying PostgreSQL deployment (instance size, connection pool limits, indexes, network) rather than by any code in this repository.

---

Next: Schema Evolution