iiinova — where your digital self keeps evolving

Craig @craigkerstiens.com · 21/11/2025

It takes 10 times as much energy to kill good ideas as it does to create them.

Craig @craigkerstiens.com · 02/06/2025

For the past five years we've focused on production ready Postgres, which is unique from most other database providers. Nothing about that focus changes today, me and the entire team are as laser focused as every on that, but now with an expanded mission of a unified data platform.

060

Craig @craigkerstiens.com · 02/06/2025

Increasingly a first class Postgres experience isn't complete without a seamless analytics experience.

160

Craig @craigkerstiens.com · 02/06/2025

Five years ago I joined @crunchydata.com, shortly after I wrote about having unfinished business with Postgres. Today as part of Snowflake that journey is continuing. We've built some amazing things, but are just getting started. www.crunchydata.com/blog/crunchy...

crunchydata.com

Crunchy Data Joins Snowflake | Crunchy Data Blog

We are excited to announce that Crunchy Data is joining Snowflake to bring Postgres to the AI Data Cloud.

6315

Craig @craigkerstiens.com · 23/05/2025

We've got food duty at the first kids travel tournament of the year, was thinking about trying to pull off good quality pour over coffee. Anyone have good mechanisms when unclear a good power source for kettle exists?

110

Craig @craigkerstiens.com · 19/05/2025

The design of everything from reception to HR to engineering mattered, appreciating and prioritizing design at that level definitely set it apart at a time where that wasn't common.

061

Craig @craigkerstiens.com · 19/05/2025

Way back at Heroku when were having a company onsite... we took the entire company to the SF MOMA for a Dieter Rams exhibit. It wasn't just the designers focused on design it was everyone.

1111

Craig @craigkerstiens.com · 19/05/2025

Production ready Postgres. That's it.

160

Craig @craigkerstiens.com · 09/05/2025

👋 if you want to come over to near Berkeley

010

Craig @craigkerstiens.com · 07/05/2025

The details matter. In this case, completely revamping our newsletter signup screen ahead of an upcoming conference. I mean why wouldn't you execute SQL to sign up for a database newsletter?

181

Craig @craigkerstiens.com · 07/05/2025

New release of pg_parquet including: * Amazon S3 * Azure Blob Storage * Google Cloud Storage * http(s) stores * local files Still the easiest way to simplify some of your ETL allowing Postgres and parquet to play well together. www.crunchydata.com/blog/announc...

crunchydata.com

Announcing pg_parquet v.0.4.0: Google Cloud Storage, https storage, and more | Crunchy Data Blog

pg_parquet is a copy/to from for Postgres and Parquet. We're excited to announce integration with Google Cloud storage, https, and additional formats.

081

Reposted by Craig

Mary Branscombe @marypcbuk.bsky.social · 06/05/2025

Iceberg has gone from being the thing Netflix (and then Apple) built for their own enormous data lakes to a collaborative open standard where even the competition is learning to co-operate and align: I dug into the Iceberg summit and asked @craigkerstiens.com and others why it's so useful

thestack.technology

The Iceberg revolution: A catalyst for data transformation?

"It took two commands, and it saved us $30,000 a month on our cloud bill"

193

Craig @craigkerstiens.com · 05/05/2025

Happy revenge of the sith day to all who celebrate.

140

Craig @craigkerstiens.com · 04/05/2025

Updating my nulls today with: \pset null 🤖 Can't decide if it's for R2-D2, C-3PO, or BB-8, but closest I can get for May the 4th. May keep it for a few days...

000

Craig @craigkerstiens.com · 03/05/2025

A few weeks in, but still every time I demo this it feels absolutely magical. Finally convergence of transactional and analytical data. www.crunchydata.com/blog/logical...

crunchydata.com

Logical replication from Postgres to Iceberg | Crunchy Data Blog

We've launched native logical replication from Postgres tables in any Postgres server to Iceberg tables managed by Crunchy Data Warehouse.

030

Craig @craigkerstiens.com · 28/04/2025

Now live on the Crunchy Bridge dashboard the ability to seamlessly sync data between your operational database over to your data warehouse for analytics - docs.crunchybridge.com/changelog#da...

docs.crunchybridge.com

Changelog

030

Craig @craigkerstiens.com · 24/04/2025

If I had known we didn't have to fully manage the Iceberg for folks we might have shipped so much sooner 😂

020

Craig @craigkerstiens.com · 23/04/2025

Was sort of discussed in an office hours session yesterday, you can "sort of" do it with pyIceberg, but only sort of, it's not really an "easy" button for it. But still don't think that's the primary reason people aren't.

100

Reposted by Craig

Gunnar Morling @gunnarmorling.dev · 23/04/2025

My guess would be people are just using Iceberg connectors, see things seem to work, and compaction is an after-thought. That's why IMO good Iceberg support is not a connector feature but an engine feature. Like what Crunchy is doing with their DWH, or, for Kafka, Confluent with Tableflow.

262

Craig @craigkerstiens.com · 23/04/2025

Nope, didn't really get into it in that much detail lots of quick hallway conversations.

010

Craig @craigkerstiens.com · 23/04/2025

A shocking take-away for me from a few weeks ago at Iceberg Summit and similarly at Data Council today is for all those using Iceberg yet so few doing compaction on their data lake files. To me seemed a requirement for any production Iceberg usage, otherwise you wake up shocked in a few months.

290

Craig @craigkerstiens.com · 23/04/2025

Yesterday met someone at Data Council that was very familiar with @crunchydata.com team and described us as basically being team Avengers for Postgres/databases. That's a new one, but will totally take it.

160

Craig @craigkerstiens.com · 23/04/2025

With yesterdays launch of logical replication for our Data Warehouse. Now you can still use CDC for your other data pipelines, but for your primary Postgres from operational -> analytical that is solved for you–without buying yet another tool.

020

Craig @craigkerstiens.com · 23/04/2025

In talking with customers that were using CDC tools to get data from Postgres into a data warehouse, 80% of their spend on ETL was the Postgres data movement. Further many of these tools often caused production outages. We knew as soon as we launched Crunchy Data Warehouse we had to solve that.

110

Craig @craigkerstiens.com · 23/04/2025

Sitting in Ryan Blue’s talk at Data Council and about to leave from Q&A to head to office hours and question comes up about CDC from databases to Iceberg… @marcoslot.com makes a hard U-turn to tune in

040

Craig @craigkerstiens.com · 22/04/2025

@andypavlo.bsky.social on HN sums it up well OLAP vs. OLTP isn't right vs. wrong, they're designed for different things. Have a version of this on a slide when explaining Crunchy Data Warehouse and how it's for very different purposes than stock Postgres.

291

Reposted by Craig

Marco Slot @marcoslot.com · 22/04/2025

And there it is: Native logical replication from any Postgres server to Iceberg managed by Crunchy Data Warehouse. Speed up Postgres analytical queries 100x with 2 commands.

2202

Reposted by Craig

Crunchy Data @crunchydata.com · 22/04/2025

Today we're announcing the availability of logical replication from Postgres to Iceberg with Crunchy Data Warehouse. Now you can seamlessly move data and stream changes from your operational database into an analytical system. www.crunchydata.com/blog/logical...

crunchydata.com

Logical replication from Postgres to Iceberg | Crunchy Data Blog

We've launched native logical replication from Postgres tables in any Postgres server to Iceberg tables managed by Crunchy Data Warehouse.

1131

Craig @craigkerstiens.com · 22/04/2025

One of the best parts, because it builds on native Postgres logical replication you could also leverage for larger data sets... - Ingest in Postgres with partitioning - Add/remove partitions to the replication set - Retain all your data in Iceberg - Smaller recent set in Postgres

010

Reposted by Craig

Richard Bishop @richardb.bsky.social · 22/04/2025

If you're using Postgres for your app data you can stop stitching together a myriad of ETL tools and analytics data stores and use one database for everything. This is both awesome and a lot of fun to use.

092

Craig @craigkerstiens.com · 22/04/2025

The way this converges operational and analytical systems is nothing short of magical. bsky.app/profile/crun...

030

Craig @craigkerstiens.com · 22/04/2025

🐘 meets 🧊 End to end, under 2 minutes. - Two commands to replicate data from Postgres -> Iceberg - Synced over 10m rows under a minute - Data is continually processed and updated in Iceberg - count(*) in Postgres over 300ms down to under 20ms

030

Craig @craigkerstiens.com · 22/04/2025

Now playing: Eye of the tiger

010

Craig @craigkerstiens.com · 22/04/2025

We now expose the physical zone directly to users on Crunchy Bridge. Most think they know what an availability zone is, but did you know by default AWS randomizes the names between accounts. You have to get to the physical zone id to colocate things - docs.crunchybridge.com/changelog#fe...

docs.crunchybridge.com

Changelog

010

Craig @craigkerstiens.com · 16/04/2025

I've learned over the years when @louisemeta.bsky.social tells me she has an idea for a blog post I don't hesitate, just simply ask when can I see it because I know it's going to be awesome. This is no exception

072

Craig @craigkerstiens.com · 15/04/2025

This was a lot of fun and covered a ton in 45 minutes. If you watch and have questions happy to setup time to dig in deeper with anyone interested.

010

Craig @craigkerstiens.com · 14/04/2025

Fast decisions on each of the above completely changes the velocity of a team.

020

Craig @craigkerstiens.com · 14/04/2025

3. These don't align with where we're headed. Don't capture these. Be straight with customers, don't say "we'll take this into account within planning". This doesn't mean you may not change you're mind and do it in the future, but if it's important it'll come back up.

110

Craig @craigkerstiens.com · 14/04/2025

2. This isn't a bad idea, but is a bigger investment. Track this, put it in a queue to evaluate, see how many other customers want this. Put it in your effort vs. impact planning. Some of these you may never do, but still make sense to actually evaluate more deeply of investment relative to impact.

120

Craig @craigkerstiens.com · 14/04/2025

1. This makes sense and is small, you should just do it. Could be a bug or a feature. Don't put this "into planning to evaluate" just do it. The mental overhead to plan and revisit and get into the to do list makes more work than just doing the thing.

120

Craig @craigkerstiens.com · 14/04/2025

There are 3 types of customer feedback you can get:

130

Craig @craigkerstiens.com · 08/04/2025

Looking like tomorrow...

100

Craig @craigkerstiens.com · 08/04/2025

What's this? Flexible/variable long term backup retention 👀

150

Craig @craigkerstiens.com · 08/04/2025

Postgres will still be a staple for OLTP workloads. It's also a very key piece of the OLAP workloads, but it's one piece of several in a bigger puzzle.

031

Craig @craigkerstiens.com · 08/04/2025

- Postgres isn't going away, it's become a standard of databases. - Iceberg is the file format that changes things and turns immutable columnar files into a living database - S3 (or S3 compatible) is the future of storage layers

2124

Craig @craigkerstiens.com · 08/04/2025

But, when it comes to analytics even for an always on analytics engine separating these pieces has a much greater impact. In this future world I strongly believe:

120

Craig @craigkerstiens.com · 08/04/2025

Increasingly these things are being split up into pieces, especially in the case of analytics. You see some of this with some trying to separate compute from storage. This works for Postgres, and is fine, though when you're database is always on with active users you see less benefit.

130

Craig @craigkerstiens.com · 08/04/2025

Databases are changing, things are evolving, historically your database was the storage format, storage engine, and the querying engine.

1181

Craig @craigkerstiens.com · 04/04/2025

Another day another several demos of Crunchy Data Warehouse. The ah-ha moments and lightbulbs that go off for people when I demo this remind me of the early demos of Heroku.

030