iiinova — where your digital self keeps evolving

tempest98.bsky.social @tempest98.bsky.social · 18/08/2026

Giving a talk about PgCache this Wednesday. First talk, excited! - hopefully it goes well

Reposted by @tempest98.bsky.social

PgCache 🐘🦀 @pgcache.bsky.social · 05/05/2026

Stop scaling Postgres with brute force. Read replicas copy 100% of your data for a fraction of queries...and add real ops overhead. PgCache: cache the hot 20% that matters. #PostgreSQL #SRE

231

Reposted by @tempest98.bsky.social

PgCache 🐘🦀 @pgcache.bsky.social · 29/04/2026

Release notes for PgCache/0.4.9: 0.4.9 - 2026-04-24 Materialized query results Pre-PG18 search_path change tracking Default admission_threshold changed from 2 to 1 New mv_size_ratio setting (default 10)

101

Reposted by @tempest98.bsky.social

PgCache 🐘🦀 @pgcache.bsky.social · 14/04/2026

"Our query cache failed with parameterized queries because the fingerprints went wrong.” Heard that last week. It's why we're building PgCache: to support the features that matter in production, like prepared statements and partitions. Looking for 1–2 more teams to pilot (DM me)

001

tempest98.bsky.social @tempest98.bsky.social · 06/12/2025

Been a productive day and a half. Last night, I refactored and split up one of the larger files in the project. This morning, cleaned up all the clippy warnings, including a couple of hundred about index_slicing. Although, a good chunk of those were in tests, where I'm happy to allow index_slicing

100

tempest98.bsky.social @tempest98.bsky.social · 05/12/2025

I've also reached the point where I've worked on the project long enough that early decisions made for expediency now need to be revisited. I didn't know enough at the time to know what really needed to be done, so now coming back around to those decisions actually feels kind of good.

100

tempest98.bsky.social @tempest98.bsky.social · 05/12/2025

Well, the metrics have been working as intended, showing where the fallback behavior of PgCache was masking code not working as expected. This led to adding support for binary parameters in the extended query protocol implementation. The initial implementation only worked for the text format.

000

tempest98.bsky.social @tempest98.bsky.social · 13/11/2025

Made some good progress over the last week. Added some basic metrics on cache behavior - e.g. counts of cache hits and misses. Having metrics for users of the cache will be important, but the main motivation was to use these metrics in integration tests to verify internal cache behavior.

100

tempest98.bsky.social @tempest98.bsky.social · 05/11/2025

How is my last post 18 days ago, now there is more posting to catch up on.... let's summarize what I've done since my last substantive post: I did complete the sql join work, so the cache now supports simple equi-joins, and using CDC keeps the cache up to date or invalidates queries as necessary.

100

tempest98.bsky.social @tempest98.bsky.social · 18/10/2025

Oy, well, it's been a while since I last posted. I ran into some technical difficulties with my code that were disappointing and got de-motivated for a couple of weeks. I was hoping the cache would never have to invalidate anything, but that isn't going to be realistic.

100

tempest98.bsky.social @tempest98.bsky.social · 28/08/2025

I mostly have subquery support working and have moved on to supporting VALUES clauses, and learning a bunch about SQL I didn't know, or at least the dialect #postgres uses. The surprising one being that you don't have to have any columns in the SELECT target list, so "SELECT FROM ..." is valid.

100

tempest98.bsky.social @tempest98.bsky.social · 26/08/2025

I've also been working on a small blog post updating this cool article for PG 17: blog.jooq.org/10-cool-sql-... Spoiler: only two optimizations have been added since the tests done against PG 9.6, and even then one of the changes is only partial.

blog.jooq.org

10 Cool SQL Optimisations That do not Depend on the Cost Model

Cost Based Optimisation is the de-facto standard way to optimise SQL queries in most modern databases. It is the reason why it is really really hard to implement a complex, hand-written algorithm i…

100

tempest98.bsky.social @tempest98.bsky.social · 26/08/2025

The path for updating the CDC flows for handling a simple join has become twistier than expected. I'm setting off on the path to add support for subqueries to the AST structure I've been using. I knew I would need to do this eventually, but I thought eventually would be much later.

000

tempest98.bsky.social @tempest98.bsky.social · 18/08/2025

I have my first join query returning results from the cache. It's been a bunch of work updating code flows that were assuming a single table to work with and the code is not pretty. Next step is updating the CDC related flows, and after that, a bunch of cleanup.

000

tempest98.bsky.social @tempest98.bsky.social · 14/08/2025

I've started work to bring support for simple joins to the cache. This ended up entailing a fair amount of refactoring, some expected and some not so much.

100

tempest98.bsky.social @tempest98.bsky.social · 09/08/2025

I removed a hardcoded reference to the "public" schema in the code. Attempting to handle schemas properly, but I'm sure I'll have to go through and make another pass at getting it right. The hardcoded name was bothering me, so happy to have taken this step.

000

tempest98.bsky.social @tempest98.bsky.social · 09/08/2025

I got in some quick wins today - added handling for truncate logical replication messages. The only message I still have left to handle is the type definition message, which I haven't even thought about where to start for those yet.

100

tempest98.bsky.social @tempest98.bsky.social · 07/08/2025

Worked on handling relation messages from the #PostgreSQL logical replication stream. At first I thought I'd be able to compare the existing table to the table defined in the replication stream, and then realized that makes no sense.

100

tempest98.bsky.social @tempest98.bsky.social · 05/08/2025

Added support for reading settings from a TOML file. 😌

010

tempest98.bsky.social @tempest98.bsky.social · 05/08/2025

I've found the ParseResult structure returned from pg_query::parse() to be fairly tedious to navigate and use. I've now created my own structures, the top level called SqlQuery, and a function to convert a ParseResult to a SqlQuery.

100

tempest98.bsky.social @tempest98.bsky.social · 30/07/2025

I haven't posted in a while because I moved back to LA from Munich. I was able to get some work done last week, which was mainly finding a bunch of mistakes and kind of dumb things I did. So productive week because the code is now in a much better place.

100

tempest98.bsky.social @tempest98.bsky.social · 12/07/2025

I started using the iddqd crate, it is perfect for a couple of use cases I have in the code. docs.rs/iddqd/latest...

docs.rs

iddqd - Rust

Maps where keys are borrowed from values.

000

tempest98.bsky.social @tempest98.bsky.social · 12/07/2025

I got the logical replication stream handling working, the current code is now functionally back to parity with a PoC I had written first. The architecture and the code are on a much better foundation. I'm still figuring out a good design for the threads handling the cache.

100

tempest98.bsky.social @tempest98.bsky.social · 11/07/2025

The caching part of the cache is now caching data. In implementing it I've been trying to come up with a threading architecture that makes sense. I must have rewritten the same chunk of code 3 or 4 times to come up with a design that works and that I like. Although, I don't really like it yet.

100

tempest98.bsky.social @tempest98.bsky.social · 07/07/2025

I started working on the caching part of the cache. Still no caching happening though. I added in the check if the query meets the criteria for caching, in which case the query is sent to the cache handling thread, the query is run, and results are returned.

100

tempest98.bsky.social @tempest98.bsky.social · 03/07/2025

a couple of updates - the first test is committed, an integration test that runs a few queries to verify the proxy is working. Learned about the CARGO_BIN_EXE_<name> environment variable to launch the executable application to be able to test against it.

docs.rs

pgtemp - Rust

pgtemp is a Rust library and cli tool that allows you to easily create temporary PostgreSQL servers for testing without using Docker.

100

tempest98.bsky.social @tempest98.bsky.social · 01/07/2025

I started using Error Set for errors. Still not sure the best way to structure errors, so this seems like a good start. I'm using one error set per module, which might not be the intended usage. It is working for now, we'll see how it evolves. docs.rs/error_set/la...

docs.rs

error_set - Rust

Error Set

000

tempest98.bsky.social @tempest98.bsky.social · 28/06/2025

Managed to get md5 passthrough auth to work, and scram authentication came along for the ride. Excited to have it working. Next up, figuring out how to get tests in place and to work on error handling. and after that, maybe get to start working on the actual caching part of things.

000

tempest98.bsky.social @tempest98.bsky.social · 28/06/2025

tokio stream merge worked quite nicely. back to having the proxying working again (with only trust authentication), the foundation to inspect and act on the messages is now in place. The postgres startup and authentication flow is a little tricky, will start by trying to get md5 auth to work.

000

tempest98.bsky.social @tempest98.bsky.social · 27/06/2025

After getting the raw proxying working, I thought modifying it to inspect and act on the actual messages would be fairly straightforward. Spoiler: it was not. Spent a couple of days bagging my head on async, select!, and mutability; and wandering down few dead ends.

100

tempest98.bsky.social @tempest98.bsky.social · 23/06/2025

Now I'm wishing for a spawn_local_scoped(), looks like that won't be in the cards. Here's a good article about why not: without.boats/blog/the-sco... and a GitHub issue about it that has some good discussion: github.com/tokio-rs/tok...

without.boats

The Scoped Task trilemma

000

tempest98.bsky.social @tempest98.bsky.social · 23/06/2025

I have the raw proxying of bytes working, which turns out to be simple in Rust and Tokio. It took me a little bit to find my way there. I can start working on interpreting the stream of data as part of the Postgres protocol. I'll need to improve the error handling eventually as well.

100

tempest98.bsky.social @tempest98.bsky.social · 22/06/2025

Decided I should use something more structured than println! Added tracing and tracing_subscriber to the project. I was surprised how noisy the default output is, but happy it was straightforward to add a custom formatter. Now I have a nice line with the log level, thread name, span name and message

000

tempest98.bsky.social @tempest98.bsky.social · 19/06/2025

Turned my attention from the proof of concept code to starting the architecture for what I hope will be the production code. Been a bit obsessed with reading about async runtimes, so might as well follow where my attention seemed to be heading anyway.

110

tempest98.bsky.social @tempest98.bsky.social · 16/06/2025

Got pass-through authentication working with md5 passwords, probably not a good idea but wanted to get something to work as an exercise in understanding how authentication works.

100

tempest98.bsky.social @tempest98.bsky.social · 16/06/2025

I've used postgres for years and just took their authentication for granted. Now that I'm looking at it, I realize it's a bit of a frankenstein with three separate layers in play. And also not conducive to building a proxy around it, which explains why pgbouncer uses it's own auth file.

111

tempest98.bsky.social @tempest98.bsky.social · 13/06/2025

Started, or least intending to look at how proxying authentication for postgres can be handled. Instead reading about multi and current thread runtimes for tokio, and looking at glommio.

000

tempest98.bsky.social @tempest98.bsky.social · 09/06/2025

I'm planning to use this feed as a way to keep myself accountable, posting progress and thoughts as I go. Today implemented basic handling for support AND and OR logical operators in where clauses. The code now supports the equality operator and those two boolean operators.

010

tempest98.bsky.social @tempest98.bsky.social · 09/06/2025

I started work on a side project to build a cache for PostgreSQL that keeps the cache up to date using the logical replication stream. Talked with a friend who got excited about the idea and am now trying to turn it into a startup. At a minimum the journey should be exciting.

021