1 hour ago · 14 min read2763 words · Tech · hide · 0 comments

What if a DuckDB database could have a history, writable branches and read replicas on other machines, all from SQL, with nothing but an S3 bucket in between? No database server, no catalog service, no lock manager. That’s what duckdb-differential 1.0 does. A week ago I wrote about differential storage for DuckDB and the local workflows it enables: validating imports on branches, reproducing reports, testing migrations on clones. Version 1.0 is built for DuckDB 2.0 and adds what was missing to use the same databases across machines: Continuous WAL shipping (SHIP_WAL): every commit of a database reaches S3, within a second or before the commit returns. Followers (FOLLOW true): read-only replicas on other hosts that move to each new commit on their own. Failover (differential_recover_branch): another host takes over a branch as of its last shipped commit. Remote garbage collection (differential_cleanup_remote_files): S3 data no branch or snapshot needs anymore is cleaned up. Together…

No comments yet. Log in to reply on the Fediverse. Comments will appear here.