Sign in

Nikhil Benesch

@benesch.bsky.social
1.3K followers 606 following 218 posts

Systems engineer @turbopuffer.bsky.social. Former CTO @materialize.com.

PostsRepliesMedia
Nikhil Benesch @benesch.bsky.social · 22/07/2025
They’re calling it our most boring feature yet. boring(n): mundane; works exactly as expected; the highest praise you can give a database
050
Nikhil Benesch @benesch.bsky.social · 10/06/2025
tpuf Python client went async today! 🐡 ❤️ 🐍 Async client perf slightly edges out sync client perf under heavy query load. github.com/turbopuffer/...
060
Nikhil Benesch @benesch.bsky.social · 14/05/2025
Big day today at turbopuffer HQ.
091
Reposted by Nikhil Benesch
turbopuffer @turbopuffer.bsky.social · 07/03/2025
when we said "coming soon" we really meant it now puffin' in aws-us-east-1 and aws-eu-central-1
041
Nikhil Benesch @benesch.bsky.social · 07/03/2025
Things move fast at turbopuffer. Now puffin' in aws/us-east-1 and aws/eu-central-1 too.
130
Nikhil Benesch @benesch.bsky.social · 06/03/2025
I've been dreaming about conditional writes on S3 for years. I couldn't have asked for a better way to celebrate than getting to ship tpuf on AWS. 🐡
0131
Nikhil Benesch @benesch.bsky.social · 26/02/2025
It’s a rare day that my love of going to battle with build systems pays off like this. Kudos to GCP for a very impressive new SKU. 🐡💨
0120
Nikhil Benesch @benesch.bsky.social · 06/02/2025
Couldn't be more excited to be joining the team at @turbopuffer.bsky.social. 🐡💨
0170
Nikhil Benesch @benesch.bsky.social · 30/12/2024
Just catching up on my NULL BITMAPS and this is easily the best intuition for write skew I've ever seen described. The analogy to merge skew in a codebase is genius.
090
Nikhil Benesch @benesch.bsky.social · 19/12/2024
Well well well: www.crunchydata.com/blog/pg_incr... Incremental pipelines come to Postgres via Crunchy Data! This is like "dbt incremental", not true incremental view maintenance like @materialize.com or Snowflake's dynamic tables, but it's a neat step towards IVM.
crunchydata.com
pg_incremental: Incremental Data Processing in Postgres | Crunchy Data Blog
We are excited to release a new open source extension called pg_incremental. pg_incremental works with pg_cron to do incremental batch processing for data aggregations, data transformations, or import...
0213
Reposted by Nikhil Benesch
Phil Eaton @eatonphil.bsky.social · 12/12/2024
The 6th (and last of 2024!) NYC Systems talks are next Thursday! We've got @jaronoff.com of Omlet and @benesch.bsky.social of Materialize speaking. :) nycsystems.xyz/december-202...
0202
Reposted by Nikhil Benesch
DBA @pgdba.bsky.social · 10/12/2024
🚀 Postgres Compatibility Index (PCI): Think your shiny new Postgres derivative is "Postgres-compatible"? Test it, score it, and know the truth. 🐘 Thanks @gunnarmorling.dev @tudor.golubenco.org @benesch.bsky.social @deverts.bsky.social for the inspiration. Blog ==> tinyurl.com/mmfcbczz
tinyurl.com
PostgreSQL Compatibility Index: The Fellowship of the Database
In the mystical realm of databases, a new hero rises every few moons — a shiny, next-gen PostgreSQL derivative, boldly claiming to be…
4155
Reposted by Nikhil Benesch
Chris @chris.blue · 05/12/2024
First new post in a couple of weeks! There's been a lot of activity around regattastorage.com this week, so I decided to write about the space. tl;dr It's pretty exciting!
materializedview.io
The Quest for a Distributed POSIX-Compatible Filesystem
Distributed POSIX filesystems have proven elusive, but we're getting closer. Perhaps that's all we need.
2209
Reposted by Nikhil Benesch
Marc Brooker @marcbrooker.bsky.social · 05/12/2024
Today's third blog post on Aurora DSQL: this time looking at the write and commit path, and how conflicts are detected and handled: brooker.co.za/blog/2024/12...
brooker.co.za
DSQL Vignette: Transactions and Durability - Marc's Blog
46515
Reposted by Nikhil Benesch
Conor McCarter @conormccarter.bsky.social · 04/12/2024
For anyone keeping score in the open table format wars, AWS launching their s3 table bucket with only Iceberg support seems like a really big deal. Nothing is named in such a way that they can't add support for Delta down the line, but this feels like a huge endorsements of Iceberg (vs. others)
181
Reposted by Nikhil Benesch
Marc Brooker @marcbrooker.bsky.social · 04/12/2024
My second blog post on Aurora DSQL, with a deep dive on how reads work: brooker.co.za/blog/2024/12...
brooker.co.za
DSQL Vignette: Reads and Compute - Marc's Blog
510029
Nikhil Benesch @benesch.bsky.social · 04/12/2024
S3 (Iceberg) Tables is everything I dreamt of, and more. I blogged some long-form thoughts: meltware.com/2024/12/04/s... I think we're about to see an explosion of data tools (@materialize.com, @clickhouse.com, @duckdb.org, et al.) learn to write Iceberg tables via S3 table buckets. #databs
meltware.com
A First Look at S3 (Iceberg) Tables
AWS announced S3 Tables today, which brings native support for Apache Iceberg to S3. It’s hard to overstate how exciting this is for the data analytics ecosystem. This post is a quick rundown of my th...
1510637
Reposted by Nikhil Benesch
Marc Brooker @marcbrooker.bsky.social · 03/12/2024
Aurora DSQL uses most of the SQL layer (parser, planner, optimizer, etc) from PostgreSQL itself. The storage layer, concurrency control, replication, etc are ground-up.
1284
Nikhil Benesch @benesch.bsky.social · 03/12/2024
GCP is now the only major cloud provider without a serverless Postgres offering. As of today AWS has DSQL: aws.amazon.com/about-aws/wh... Azure partnered with Neon in August: neon.tech/blog/neon-is...
aws.amazon.com
Announcing Amazon Aurora DSQL (Preview) - AWS
Discover more about what's new at AWS with Announcing Amazon Aurora DSQL (Preview)
8173
Reposted by Nikhil Benesch
Luc van Donkersgoed @lucvandonkersgoed.com · 03/12/2024
IT'S HERE! Amazon Aurora Distributed SQL (DSQL). It's DynamoDB and Postgres merged into a truly serverless relational database. This is THE announcement of Re:Invent 2024. Everything else is irrelevant.
buff.ly
Announcing Amazon Aurora DSQL (Preview)
AWS introduces Amazon Aurora DSQL, a serverless, distributed SQL database offering unlimited scalability, high availability, and zero infrastructure management...
34110
Nikhil Benesch @benesch.bsky.social · 03/12/2024
S3 gets *native* Iceberg support. 🤯 From the AWS blog: aws.amazon.com/blogs/aws/ne... We knew something like this was coming (bsky.app/profile/bene...), but it's an even deeper integration than I expected.
aws.amazon.com
New Amazon S3 Tables: Storage optimized for analytics workloads | Amazon Web Services
Amazon S3 Tables optimize tabular data storage (like transactions and sensor readings) in Apache Iceberg, enabling high-performance, low-cost queries using Athena, EMR, and Spark.
4459
Nikhil Benesch @benesch.bsky.social · 01/12/2024
@glaubercosta.bsky.social nails an explanation of 1) why vector databases are mostly a feature not a product but 2) why nonetheless @turbopuffer.bsky.social is on to something. From a Software Huddle interview a few months back: www.youtube.com/watch?v=a4X7...
youtube.com
SQLite, Turso, and the State of Databases with Glauber Costa
YouTube video by Software Huddle
2181
Nikhil Benesch @benesch.bsky.social · 28/11/2024
@fasterthanli.me what's the performance of loona like? 👀 I'm in the market for a faster S3 client in Rust and loona is the first plausible project I surfaced. bsky.app/profile/bene...
110
Nikhil Benesch @benesch.bsky.social · 27/11/2024
Awesome work from @antiguru.bsky.social that can cut @materialize.com memory use by 50% on a common class of workload. Most importantly for our customers: Materialize is now 50% cheaper on these workloads!
160
Nikhil Benesch @benesch.bsky.social · 27/11/2024
Somehow Starlark inherited all the same Unicode problems that Python 2 had. 🙃 github.com/tidbyt/pixle... cost me more time than I would like to admit.
github.com
Add a .bytes() method to HTTP responses by benesch · Pull Request #1095 · tidbyt/pixlet
The new .bytes() method returns the body as bytes rather than a string. It is modeled after the .json() method, which parses the body as JSON. The use case is parsing HTTP requests that return bina...
251
Nikhil Benesch @benesch.bsky.social · 26/11/2024
Something interesting is brewing in Iceberg-on-S3 land. 👀 lists.apache.org/thread/v7x65... cc @eatonphil.bsky.social
lists.apache.org
3295
Nikhil Benesch @benesch.bsky.social · 26/11/2024
Annoying detail about S3's new `If-Match` implementation: these writes can fail with 409 Conflict (docs: docs.aws.amazon.com/AmazonS3/lat...) I guess you're just meant to retry, but it's not clear to me why S3 wouldn't just retry internally. The whole point is that the requests are now idempotent!
1141
Reposted by Nikhil Benesch
Sirupsen @sirupsen.bsky.social · 25/11/2024
S3 has finally added `put-if-match` for compare-and-swap! Expect AWS turbopuffer regions very soon aws.amazon.com/about-aws/wh...
aws.amazon.com
Amazon S3 adds new functionality for conditional writes - AWS
Discover more about what's new at AWS with Amazon S3 adds new functionality for conditional writes
26013
Nikhil Benesch @benesch.bsky.social · 25/11/2024
Mission accomplished! I’ll be living in luxury with eight (8!) upcoming subway arrivals displayed on my Tidbyt. Developing for the Tidbyt was a mostly pleasant experience. Needing to hand roll a protobuf parser in Starlark was a curveball though: github.com/benesch/tidb...
130
Nikhil Benesch @benesch.bsky.social · 25/11/2024
Working on the important stuff today: teaching my Tidbyt to display subway arrival times for more than two trains at a time. Here’s what the current app looks like.
080
Nikhil Benesch @benesch.bsky.social · 24/11/2024
Went to write my first app for my Tidbyt today only to discover they've decided to join Modal: tidbyt.com/blogs/tidbyt... What a bittersweet moment. I think the world of the folks at Modal, but I'm sad to see the end of Tidbyt manufacturing.
tidbyt.com
Tidbyt is joining Modal
We're really excited to announce that the Tidbyt team will be joining Modal. Don't worry, the Tidbyt platform isn't going anywhere! This change simply means that we now have the resources to continue ...
040
Reposted by Nikhil Benesch
jasnonaz @jasnonaz.bsky.social · 21/11/2024
Full house at the NYC dbt Meetup tonight
4191
Nikhil Benesch @benesch.bsky.social · 19/11/2024
Anyone know what happened to PlanetScale Boost? It seems to have disappeared from @planetscale.com's docs and blog.
250
Nikhil Benesch @benesch.bsky.social · 14/11/2024
@arjunnarayan.com 👋
270
Nikhil Benesch @benesch.bsky.social · 13/11/2024
Has anyone made an Iceberg catalog implementation that uses only S3? Should be possible now that S3 supports conditional puts. Seems like it'd be a lot easier to operate than any of the existing Iceberg catalogs.
5184