Craig @craigkerstiens.com · 21/11/2025It takes 10 times as much energy to kill good ideas as it does to create them. 020
Craig @craigkerstiens.com · 02/06/2025Five years ago I joined @crunchydata.com, shortly after I wrote about having unfinished business with Postgres. Today as part of Snowflake that journey is continuing. We've built some amazing things, but are just getting started. www.crunchydata.com/blog/crunchy...crunchydata.comCrunchy Data Joins Snowflake | Crunchy Data BlogWe are excited to announce that Crunchy Data is joining Snowflake to bring Postgres to the AI Data Cloud. 6315
Craig @craigkerstiens.com · 23/05/2025We've got food duty at the first kids travel tournament of the year, was thinking about trying to pull off good quality pour over coffee. Anyone have good mechanisms when unclear a good power source for kettle exists? 110
Craig @craigkerstiens.com · 19/05/2025Way back at Heroku when were having a company onsite... we took the entire company to the SF MOMA for a Dieter Rams exhibit. It wasn't just the designers focused on design it was everyone. 1111
Craig @craigkerstiens.com · 07/05/2025The details matter. In this case, completely revamping our newsletter signup screen ahead of an upcoming conference. I mean why wouldn't you execute SQL to sign up for a database newsletter? 181
Craig @craigkerstiens.com · 07/05/2025New release of pg_parquet including: * Amazon S3 * Azure Blob Storage * Google Cloud Storage * http(s) stores * local files Still the easiest way to simplify some of your ETL allowing Postgres and parquet to play well together. www.crunchydata.com/blog/announc...crunchydata.comAnnouncing pg_parquet v.0.4.0: Google Cloud Storage, https storage, and more | Crunchy Data Blogpg_parquet is a copy/to from for Postgres and Parquet. We're excited to announce integration with Google Cloud storage, https, and additional formats. 081
Reposted by CraigMary Branscombe @marypcbuk.bsky.social · 06/05/2025Iceberg has gone from being the thing Netflix (and then Apple) built for their own enormous data lakes to a collaborative open standard where even the competition is learning to co-operate and align: I dug into the Iceberg summit and asked @craigkerstiens.com and others why it's so usefulthestack.technologyThe Iceberg revolution: A catalyst for data transformation?"It took two commands, and it saved us $30,000 a month on our cloud bill" 193
Craig @craigkerstiens.com · 04/05/2025Updating my nulls today with: \pset null 🤖 Can't decide if it's for R2-D2, C-3PO, or BB-8, but closest I can get for May the 4th. May keep it for a few days... 000
Craig @craigkerstiens.com · 03/05/2025A few weeks in, but still every time I demo this it feels absolutely magical. Finally convergence of transactional and analytical data. www.crunchydata.com/blog/logical...crunchydata.comLogical replication from Postgres to Iceberg | Crunchy Data BlogWe've launched native logical replication from Postgres tables in any Postgres server to Iceberg tables managed by Crunchy Data Warehouse. 030
Craig @craigkerstiens.com · 28/04/2025Now live on the Crunchy Bridge dashboard the ability to seamlessly sync data between your operational database over to your data warehouse for analytics - docs.crunchybridge.com/changelog#da...docs.crunchybridge.comChangelog 030
Reposted by CraigGunnar Morling @gunnarmorling.dev · 23/04/2025My guess would be people are just using Iceberg connectors, see things seem to work, and compaction is an after-thought. That's why IMO good Iceberg support is not a connector feature but an engine feature. Like what Crunchy is doing with their DWH, or, for Kafka, Confluent with Tableflow. 262
Craig @craigkerstiens.com · 23/04/2025A shocking take-away for me from a few weeks ago at Iceberg Summit and similarly at Data Council today is for all those using Iceberg yet so few doing compaction on their data lake files. To me seemed a requirement for any production Iceberg usage, otherwise you wake up shocked in a few months. 290
Craig @craigkerstiens.com · 23/04/2025Yesterday met someone at Data Council that was very familiar with @crunchydata.com team and described us as basically being team Avengers for Postgres/databases. That's a new one, but will totally take it. 160
Craig @craigkerstiens.com · 23/04/2025In talking with customers that were using CDC tools to get data from Postgres into a data warehouse, 80% of their spend on ETL was the Postgres data movement. Further many of these tools often caused production outages. We knew as soon as we launched Crunchy Data Warehouse we had to solve that. 110
Craig @craigkerstiens.com · 23/04/2025Sitting in Ryan Blue’s talk at Data Council and about to leave from Q&A to head to office hours and question comes up about CDC from databases to Iceberg… @marcoslot.com makes a hard U-turn to tune in 040
Craig @craigkerstiens.com · 22/04/2025@andypavlo.bsky.social on HN sums it up well OLAP vs. OLTP isn't right vs. wrong, they're designed for different things. Have a version of this on a slide when explaining Crunchy Data Warehouse and how it's for very different purposes than stock Postgres. 291
Reposted by CraigMarco Slot @marcoslot.com · 22/04/2025And there it is: Native logical replication from any Postgres server to Iceberg managed by Crunchy Data Warehouse. Speed up Postgres analytical queries 100x with 2 commands. 2202
Reposted by CraigCrunchy Data @crunchydata.com · 22/04/2025Today we're announcing the availability of logical replication from Postgres to Iceberg with Crunchy Data Warehouse. Now you can seamlessly move data and stream changes from your operational database into an analytical system. www.crunchydata.com/blog/logical...crunchydata.comLogical replication from Postgres to Iceberg | Crunchy Data BlogWe've launched native logical replication from Postgres tables in any Postgres server to Iceberg tables managed by Crunchy Data Warehouse. 1131
Reposted by CraigRichard Bishop @richardb.bsky.social · 22/04/2025If you're using Postgres for your app data you can stop stitching together a myriad of ETL tools and analytics data stores and use one database for everything. This is both awesome and a lot of fun to use. 092
Craig @craigkerstiens.com · 22/04/2025The way this converges operational and analytical systems is nothing short of magical. bsky.app/profile/crun... 030
Craig @craigkerstiens.com · 22/04/2025🐘 meets 🧊 End to end, under 2 minutes. - Two commands to replicate data from Postgres -> Iceberg - Synced over 10m rows under a minute - Data is continually processed and updated in Iceberg - count(*) in Postgres over 300ms down to under 20ms 030
Craig @craigkerstiens.com · 22/04/2025We now expose the physical zone directly to users on Crunchy Bridge. Most think they know what an availability zone is, but did you know by default AWS randomizes the names between accounts. You have to get to the physical zone id to colocate things - docs.crunchybridge.com/changelog#fe...docs.crunchybridge.comChangelog 010
Craig @craigkerstiens.com · 16/04/2025I've learned over the years when @louisemeta.bsky.social tells me she has an idea for a blog post I don't hesitate, just simply ask when can I see it because I know it's going to be awesome. This is no exception 072
Craig @craigkerstiens.com · 15/04/2025This was a lot of fun and covered a ton in 45 minutes. If you watch and have questions happy to setup time to dig in deeper with anyone interested. 010
Craig @craigkerstiens.com · 08/04/2025What's this? Flexible/variable long term backup retention 👀 150
Craig @craigkerstiens.com · 08/04/2025Databases are changing, things are evolving, historically your database was the storage format, storage engine, and the querying engine. 1181
Craig @craigkerstiens.com · 04/04/2025Another day another several demos of Crunchy Data Warehouse. The ah-ha moments and lightbulbs that go off for people when I demo this remind me of the early demos of Heroku. 030
Craig @craigkerstiens.com · 04/04/2025Great talk from @marcoslot.com at recent Iceberg meetup on building a Postgres Data Warehouse with Iceberg! www.youtube.com/watch?v=cEnq...youtube.comBuilding a Postgres Data Warehouse with IcebergYouTube video by Apache Iceberg™ Meetup 030
Craig @craigkerstiens.com · 04/04/2025Within Crunchy Data Warehouse we have a lot of new flexibility in how you work with your database. First, you have standard Postgres heap tables. 110
Craig @craigkerstiens.com · 02/04/2025Great post from @simpsoka.bsky.social on what she calls "papercuts" - kathykorevec.substack.com/p/papercuts-...kathykorevec.substack.comPapercuts: The little fixes that made a big difference at GitHubHow fixing the smallest annoyances helped rebuild trust, momentum, and a better developer experience at GitHub. 032
Craig @craigkerstiens.com · 02/04/2025If you want to be really good at databases, don't start with learning SQL, start with relational algebra and relational calculus. I know it sounds crazy, but once you get that the rest of SQL comes so shockingly easy. Completely changes how you think about data. 1100
Craig @craigkerstiens.com · 02/04/2025How many people use window functions in Postgres? It's one of those features that when you need them they're perfect, letting you compare between previous or future records or organize across records, but I always find it difficult to describe them to folks. How would you explain window functions? 431
Craig @craigkerstiens.com · 01/04/2025It's increasing to almost half the time @gerikerstiens.bsky.social comes down the stairs in an outfit that matches mine (to be clear I was dressed well before her) and today it's matching softball team shirts 000
Craig @craigkerstiens.com · 01/04/2025When we first launched Crunchy Data Warehouse the thing that surprised me most was how many wanted to self-manage on-premises. Took us a few months, but here it is. 131
Craig @craigkerstiens.com · 28/03/2025Have you setup a psqlrc file? It's like bashrc but for psql within Postgres. Some handy things in mine: \x auto \timing \pset null 👻 1120
Craig @craigkerstiens.com · 27/03/2025I've encountered a database without constraints that "enforces them within the app" I've encountered data quality issues. Postgres has a variety of constraints to help, not just primary/unique and foreign keys, but also not null and check constraints. 2132
Reposted by CraigMarco Slot @marcoslot.com · 26/03/2025We weren't really thinking of log management as a target use case, but Iceberg is ideal as the final destination for logs, and having transactions & built-in job scheduling & a fast query engine (& laser focus on developer experience) makes things really simple and cost-effective. 172
Craig @craigkerstiens.com · 26/03/2025Something I've always found pride in is making developer tooling more accessible. You won't find many that think Postgres isn't powerful, but navigating it can be overwhelming. Scheduled jobs in a database aren't a "new thing", pg_cron has long been around but it wasn't always "friendly". 130
Craig @craigkerstiens.com · 26/03/2025I got a number of questions on how we saved $30k a month on cloudwatch by moving logs directly to S3/Iceberg with Postgres so I wrote up how in a bit more detail - www.crunchydata.com/blog/reducin...crunchydata.comReducing Cloud Spend: Migrating Logs from CloudWatch to Iceberg with Postgres | Crunchy Data BlogHow we migrated our internal logging for our database as a service, Crunchy Bridge, from CloudWatch to S3 with Iceberg and Postgres. The result was simplified logging management, better access with SQ... 072
Reposted by CraigCrunchy Data @crunchydata.com · 26/03/2025We talk with users every week about whether Citus is a good fit for them. While Citus is a very powerful Postgres extension it has very specific use cases where it does fit, here we break down the cases where Citus is a fit as well as when it's not -buff.lyCitus: The Misunderstood Postgres Extension | Crunchy Data BlogWhat applications and use cases make the most sense for Citus. 011
Craig @craigkerstiens.com · 25/03/2025Partitioning in Postgres is great if you've got any form of time series data. It can allow higher ingest, faster queries, and easily archiving old data. Common examples could be: - logs - events - metrics 230
Craig @craigkerstiens.com · 24/03/2025I'm not throwing shade at Citus. It was and still is some amazing engineering. You just probably don't need it, in most cases just use Postgres without Citus. www.crunchydata.com/blog/citus-t...crunchydata.comCitus: The Misunderstood Postgres Extension | Crunchy Data BlogWhat applications and use cases make the most sense for Citus. 010
Craig @craigkerstiens.com · 24/03/2025Just gave a demo of Crunchy Data Warehouse, they're currently already running Iceberg running Spark and several other things for their pipeline. They asked about compaction walked them through - www.crunchydata.com/blog/automat... "So we don't have to deal with all the complexity we've built"? Yup!crunchydata.comAutomatic Iceberg Maintenance Within Postgres | Crunchy Data BlogIceberg can create orphan files during snapshot changes or transaction rollbacks. Crunchy Data Warehouse automatically cleans up the orphan files using a new autovacuum feature. 040
Craig @craigkerstiens.com · 24/03/2025When's the last time you checked in on how your queries were performing? If you can't remember now is a great time to check in with pg_stat_statements: SELECT total_exec_time, mean_exec_time as avg_ms, calls, query FROM pg_stat_statements ORDER BY mean_exec_time DESC LIMIT 10; 090
Craig @craigkerstiens.com · 24/03/2025Looks like I’m ended March with a lovely ski glass sunburn/tan 010