“Moving several large, crucial Postgres datasets (some of them tens of TB large) to data lake gave us a net savings of over a million dollars for 2022”
Notion went from 20 billion block rows in 2021 to more than 200 billion, spread over 480 logical Postgres shards. It fed Snowflake through 480 Fivetran connectors until 90% of its writes turned out to be updates, the workload warehouses handle worst. Kafka, Hudi, Spark and S3 cut ingestion from over a day to minutes and saved more than a million dollars in year one. The savings came out of Snowflake’s bill.