5 min read
Resumable Inside Airbnb ingestion with Airflow and S3
A verified bronze ingestion boundary: consistent DuckDB and Polars output, conditional S3 publication, manifest recovery and a mapped Airflow DAG.
data-engineering / airflow / s3
Back to home
Data engineering blog
Notes from my work.
Practical guides to browser analytics with DuckDB-WASM and Parquet, MotherDuck data applications, Next.js architecture, and self-hosted Obsidian LiveSync. Each article follows a project I have built.
- 7 min read
Comparing Polars, DuckDB, chDB and pandas on 30 million events
A correctness-checked CPU comparison across four workloads, with isolated processes, recorded timing ranges and peak memory evidence.
data-engineering / polars / duckdb
- 6 min read
From Flink streams to retry-safe Airflow batches
Synthetic TCP events through Flink and Kafka into asset-triggered Airflow batches, with durable Parquet publication, explicit offset ownership and verified replay.
data-engineering / airflow / kafka
- 13 min read
Follow the money: MotherDuck analytics over 179 million transfers
Build interactive analytics over 179 million synthetic IBM AML transfers with MotherDuck, DuckLake and DuckDB-WASM, tracing dashboard filters to SQL.
motherduck / ducklake / duckdb
- 5 min read
Building a Next.js Portfolio with DuckDB-WASM and Parquet
How I built a Next.js portfolio with DuckDB-WASM, MDX and remote Parquet on S3 and CloudFront, including browser workers and lazy query loading.
nextjs / architecture / duckdb-wasm
- 11 min read
Self-hosting Obsidian LiveSync with CouchDB on AWS Lightsail
Self-host Obsidian LiveSync with CouchDB, Docker and Caddy on AWS Lightsail. Download the pinned setup, verify CORS and permissions, and test backup recovery.
obsidian / couchdb / self-hosting
- 2 min read
Querying ATP Tennis Data with DuckDB-WASM and Parquet
Run SQL on ATP tennis data in your browser with DuckDB-WASM, remote Parquet files and CloudFront. Explore match wins and aces with live queries.
duckdb / data-engineering / webassembly