iiinova — where your digital self keeps evolving

v @avi.im · 21/04/2026

I often get emails asking for ideas (or materials) to learn about distributed systems. I always recommend getting started with Gossip Glomers. Only six challenges, some easy and some difficult, but an absolute top tier fun and learning experience fly.io/dist-sys

2223

v @avi.im · 17/04/2026

Here is an insane piece of lore inside SQLite's source code I am researching VACUUM and I was studying their code. In VACUUM, SQLite creates a temp file prefixed with `etilqs_` Here is why:

** 2006-10-31: The default prefix used to be "sqlite_". But then
** Mcafee started using SQLite in their anti-virus product and it
** started putting files with the "sqlite" name in the c:/temp folder.
** This annoyed many windows users. Those users would then do a
** Google search for "sqlite", find the telephone numbers of the
** developers and call to wake them up at night and complain.
** For this reason, the default name prefix is changed to be "sqlite"
** spelled backwards. So the temp files are still identified, but
** anybody smart enough to figure out the code is also likely smart
** enough to know that calling the developer will not help get rid
** of the file.
*/
#ifndef SQLITE_TEMP_FILE_PREFIX
# define SQLITE_TEMP_FILE_PREFIX "etilqs_"
#endif

3444106

v @avi.im · 14/04/2026

AI is going to radically change how people learn and upskill. Last week, my fren asked for resources to learn distributed systems. I asked him to try Gossip Glomers. Today he mentioned that he did them all: he prompted GPT to solve them and then verified the solutions 🔥

320

v @avi.im · 25/01/2026

redditor explains why job hunting in tech is exactly like auditioning for acting roles

I have the benefit of having an actress in my life, so I get to watch the process of auditioning.

Auditioning is a crap-shoot. You never really know what the director is looking for. They may have someone in mind already and the whole audition is a courtesy / box-check on some grant program they're operating the theater under. You do your best, you take rejection, you suit up and do it again. She does some additional work: she researches the theaters, keeps her ear to the ground, talks to other actors in the area about their experience (that's easier with actors, where the gigs are one-offs; software engineers aren't moving as fast through industry so you have to build a larger contact web to get a richer picture of who's hiring in your area).

But the biggest thing that controls whether you get a role is if you keep showing up. The role finds you; your control over getting the role (unless you're a big enough name that people recognize it) is minimal.

I know it's not the most comfor

180

v @avi.im · 23/01/2026

In the 90s, Linus Torvalds had a much superior language to write the Linux kernel. But, since he is Finnish, he couldn't Smalltalk

130

v @avi.im · 28/10/2025

AEADs provide a verification tag after encryption. For each page, we need a nonce too. Both the nonce & the tag become metadata for an encrypted page So where do you store them? We could store them separately, but it's much better & neater to store them in the page itself (1/5)

240

v @avi.im · 26/10/2025

The B Tree data structure fascinates me. Databases use B Trees to store data on disk, organizing everything into pages that typically range from 4kb to 8kb. All I/O operations happen in units of these pages. The page looks like this... (1/9)

170

v @avi.im · 22/10/2025

Pro database tip: enable `SQL_SAFE_UPDATES` in MySQL to avoid accidental UPDATE/DELETE queries without a WHERE clause. It forces you to use a key or a LIMIT, instead of wiping whole database by mistake at 2:19am.

2172

v @avi.im · 19/10/2025

Sharding. Database sharding is one of the common techniques to scale a database horizontally. You split the db into small parts called shards and distribute them across machines. Shards are typically in the few hundreds or even thousands (for extremely large databases).

130

v @avi.im · 17/10/2025

SQLite has a page where they explain why they use C. They specifically elaborate on why not Rust www.sqlite.org/whyc....

All that said, it is possible that SQLite might one day be recoded in Rust. Recoding SQLite in Go is unlikely since Go hates assert(). But Rust is a possibility. Some preconditions that must occur before SQLite is recoded in Rust include:

Rust needs to mature a little more, stop changing so fast, and move further toward being old and boring.
Rust needs to demonstrate that it can be used to create general-purpose libraries that are callable from all other programming languages.
Rust needs to demonstrate that it can produce object code that works on obscure embedded devices, including devices that lack an operating system.
Rust needs to pick up the necessary tooling that enables one to do 100% branch coverage testing of the compiled binaries.
Rust needs a mechanism to recover gracefully from OOM errors.
Rust needs to demonstrate that it can do the kinds of work that C does in SQLite without a significant speed penalty.

3251

v @avi.im · 23/09/2025

Lil trivia to remember when it comes to Snapshot Isolation jepsen.io/consistenc...

https://jepsen.io/consistency/models/snapshot-isolation

210

Reposted by v

Andy Pavlo @andypavlo.bsky.social · 18/09/2025

Next week is the start of @db.cs.cmu.edu's latest seminar series: Future Data Systems @samarchdb.bsky.social and I are hosting speakers from leading systems in the datalake / lakehouse space. Mondays @ 4:30pm ET via Zoom. Open to the public. Videos posted to YouTube: db.cs.cmu.edu/seminars/fal...

Carnegie Mellon University
Future Data Systems
Fall 2025 Seminar Series
Mondays @ 4:30pm ET

14013

v @avi.im · 18/09/2025

The correct answer is either. Transaction B gets a snapshot that may or may not include the changes from A. SI does not provide real time guarantees. If you need that, you need Strict Serializability, which guarantees that transactions are ordered in real time.

Snapshot isolation implies read committed. However, it does not impose any real-time constraints. If process A completes write w, then process B begins a read r, r is not necessarily guaranteed to observe w. Some databases provide real-time variants of snapshot isolation. Compare with strict serializability, which provides a total order and real-time guarantees.

131

v @avi.im · 16/09/2025

Database systems question Assume the database is in snapshot isolation mode. If transaction A updates, and writes x, commits, *then* transaction B starts and reads x's value, then B will see (assume single node for simplcity): 1 - Value before A's write 2 - Value written by A 3 - Either 4 - 🍿

221

v @avi.im · 13/09/2025

Published a new blog post: Setsum - order agnostic, additive, subtractive checksum post - avi.im/blag/2025/setsum code - github.com/avinassh/...

Setsum - order agnostic, additive, subtractive checksum

Say you’re building a database replication system. The primary sends logical operations to replicas, which apply them in order:

{"op": "add", "id": "apple"}
{"op": "add", "id": "apple"}
{"op": "remove", "id": "orange"}

After the replica processes these changes (add two apples, remove an orange), how do you verify both nodes ended up in the same state? One naive (rather horrible) approach is to dump both states and compare them directly. It’s expensive, impractical, and doesn’t scale!

1133

v @avi.im · 11/09/2025

The great lock in is here! For those wanting to get into systems programming and/or database internals, consider hacking on Turso DB, the SQLite rewrite in Rust. Here's why: 1. It's a database!

150

Reposted by v

Tom Ballinger @ballingt.com · 08/09/2025

Hey we're hiring for in-person engineering roles in SF. I really enjoy my job and you might too. Come hang out and build developer tools!

1104

v @avi.im · 09/09/2025

Where can I learn about how AI companies use caching, KV stores, and databases differently for LLMs, agentic workloads? Someone also mentioned to me that old/traditional services aren't suited for these, so they also build databases internally. (e.g., OpenAI acquired Rockset)

051

Reposted by v

Franck Pachot @franckpachot.bsky.social · 09/09/2025

For nearly a decade, MongoDB provided reliable persistence with one of the most robust storage engines. Yet somehow, the oldest jokes keep persisting, too, so here are some facts: dev.to/franckpachot...

dev.to

Resilience of MongoDB's WiredTiger Storage Engine to Disk Failure Compared to PostgreSQL and Oracle

There have been jokes that have contributed to persistent myths about MongoDB's durability. The...

171

Reposted by v

Joy Gao @joygao.bsky.social · 07/09/2025

Netflix had it all wrong, don’t waste engineering resources to build your own chaos monkey infrastructure, just put production on AWS us-east-1 and you get chaos monkey for free. Just kidding, just kidding…

1203

v @avi.im · 07/09/2025

This is the opening text of Transaction Processing: Concepts and Techniques by Jim Gray "Six thousand years ago, the Sumerians invented writing for transaction processing."

Six thousand years ago, the Sumerians invented writing for transaction processing

031

Reposted by v

Jared @mathbook.cafe · 06/09/2025

My ”workshop” docs.trychroma.com/docs/overvie... github.com/ollama/ollam... www.zotero.org Basically I toss documentation into Zotero, parse the snapshots to create embeddings. The query first retrieves documents via the embedding, then shoves those into the LLM prompt with the query to respond

github.com

111

v @avi.im · 06/09/2025

Published a new post: Oldest recorded transaction. This totally could have been just a tweet (skeet?), but I wanted to publish something today. avi.im/blag/2025/old...

The other day I posted a tweet with this image which I thought was funny:

This got me thinking, can I insert this date in today’s database? What is the oldest timestamp a database can support?

So I checked the top three databases: MySQL, Postgres, and SQLite:

141

v @avi.im · 06/09/2025

Which is the best local LLM to set up for asking questions about code? I have large codebases like the Linux kernel, Postgres, etc. I want to ask questions like "find methods that do XYZ" and also post large code snippets and ask for explanations.

300

Reposted by v

Lean Focused Research Organization @lean-lang.org · 05/09/2025

If you're you curious about #LeanLang and want to understand the connection between #programming and #proofs, check out this great new video by Ank Yog. The analogy between Chess and true propositions is particularly compelling! www.youtube.com/watch?v=QXQN...

0177

v @avi.im · 05/09/2025

My extreme opinion is that anything other than serializable isolation is a scam. Database people haven't figured out how to make it fast, so we have ended up with other half baked isolation levels.

030

v @avi.im · 04/09/2025

This is the oldest transaction database from 3100 BC - recording accounts of malt and barley groats. Considering this thing survived 5000 years (holy shit!) with zero downtime and has stronger durability guarantees than most databases today. I call it rock solid durability.

A cuneiform tablet about an administrative account, with entries concerning malt and barley groats, 3100–2900 BC.

130

Reposted by v

Gunnar Morling @gunnarmorling.dev · 03/09/2025

Nice one! Spoke about this kind of architecture a while ago too: speakerdeck.com/gunnarmorlin.... Back then, using Kafka Streams as an IVM engine and Infinispan as the serving layer for denormalized views, but Sqlite (or DuckDB) would work too, with better queryability, as you say m

speakerdeck.com

Keep your cache always fresh with Debezium! (Current 22)

The saying goes that there are only two hard things in Computer Science: cache invalidation, and naming things. Well, turns out the first one is solved …

161

v @avi.im · 04/09/2025

btw, the new semester of the database systems course is back! The first video is already out, with an assignment where you implement a Count-Min Sketch 15445.courses.cs.cmu.edu/fall2025/

CMU course home page - https://15445.courses.cs.cmu.edu/fall2025/

050

v @avi.im · 03/09/2025

your priorities need to be in the right places if you want to pursue a database-centric lifestyle

Q10: Is it a good idea to start a family while taking this course?

No, we strongly advise that every student who is serious about pursuing a database-centric lifestyle refrain from acquiring, obtaining, or receiving a baby during the semester. Children are a crushing obligation that requires at least 30 hours of your time per week, which could have been spent learning about and working on database systems instead.

190

Reposted by v

DuckDB @duckdb.org · 02/09/2025

We just launched the “DuckDB in Science” site, a curated collection of papers, lectures and podcasts about DuckDB in research: duckdb.org/science/ 🎡 If you would like to learn more about DuckDB in Science, consider joining our meetup in London this Thursday: www.meetup.com/duckdb/event...

24514

v @avi.im · 03/09/2025

Have you built anything serious in v lang? How was your experience? vlang.io

010

v @avi.im · 31/08/2025

Published a new blog post: Replacing a Cache Service with a Database We already use databases, why can't we use them to replace caches as well? Will we ever replace caches entirely with databases? In this post, I will share some ideas, & discuss how we are moving toward this avi.im/blag/2025/db...

Why do we even use caches?

Caches solve one important problem: providing pre-computed data at insanely low latencies, compared to databases. I am talking about typical use cases where we use a cache along with the db (cache aside pattern), where the application always talks with cache and database, tries to keep the cache up to date with the db. There are other patterns where cache itself talks with DBs, but I think this is the more common pattern where application talks to both cache and database.

4143

v @avi.im · 31/08/2025

do you use any apps to manage bookmarks? i'm looking for something with the following features: - save links with notes and tags - archive the actual content of posts - search functionality across titles, content, etc. - ai integration would be a nice bonus

460

Reposted by v

PVLDB @pvldb.bsky.social · 03/08/2025

Vol:18 No:8 → Saving Private Hash Join 👥 Authors: Laurens Kuiper, Paul Gross, Peter Boncz, Hannes Mühleisen 📄 PDF: www.vldb.org/pvldb/vol18/p2748-kuip…

0144

v @avi.im · 30/08/2025

Published a new blog post: SQLite commits are not durable under default settings avi.im/blag/2025/sq...

Previously, I claimed that transactions in SQLite with WAL are not durable under default settings. Turns out, I was only half wrong but technically correct; the issue is actually with SQLite in rollback journal mode (the default). This post is now amended with the changes.

Here’s what I mean by durability: when the database acknowledges that a transaction is committed, it’s ‘durably’ saved to disk. That is, neither an application crash nor an OS crash should make that transaction disappear. Imagine you make a new commit, the db acknowledges success, and suddenly your OS reboots. Do you expect your transaction changes to be persisted? For example, in Postgres you can expect your changes to be there. This is how most OLTP databases behave.

061

Reposted by v

Andy Pavlo @andypavlo.bsky.social · 25/08/2025

Today is the new semester for @db.cs.cmu.edu's Intro to Database Systems! We're going harder into material than before. More challenging projects but you can use LLMs to help. We also have 10min talks each Wed from leading DB companies: 15445.courses.cs.cmu.edu/fall2025

15445.courses.cs.cmu.edu

CMU 15-445/645 :: Intro to Database Systems (Fall 2025)

You want to know whether this is the premier course at Carnegie Mellon University on the design and implementation of database management systems? Well, it is. This course rips through data models (re...

16318

Reposted by v

Jake Gold @jacob.gold · 24/08/2025

Even though I've used Go + SQLite for many years, I still find myself wasting time on new projects. Trying to solve this problem with a transparent attempt at employing Cunningham’s Law to solicit feedback/corrections from anyone who knows better. Hopefully I'll end up with a nice little library!

jacob.gold

Go + SQLite Best Practices

SQLite is amazing Most developers already know SQLite is great. If you have not tried it, you are missing out. SQLite is extremely fast, featureful, reliable, and easy to use. When your use case fits,...

3394

Reposted by v

Sam Rose @samwho.dev · 23/08/2025

✨ Big O ✨ Let me take you on a visual introduction to what big O notation is in my new blog post: samwho.dev/big-o. With big O notation you can better understand how algorithms will perform in practice, finding orders of magnitude improvements often with very simple changes to your code.

29413105

v @avi.im · 23/08/2025

Weekend paper time! One more attempt to read this paper before I give up again

171

Reposted by v

sophie alpert @sophiebits.com · 23/08/2025

I keep getting into conversations with engineers where I talk about how desperately I want my database to support materialized views with incremental updates and too often I get a blank stare in response. so I wrote a new post about why you should care! sophiebits.com/2025/08/22/m...

sophiebits.com

Materialized views are obviously useful

1615420

v @avi.im · 20/08/2025

A new theorem called the L2AW just dropped. It's kind of an alternative to the CAP theorem! law-theorem.com

040

Reposted by v

Alex Miller @alexmillerdb.bsky.social · 19/08/2025

For anyone else trying to catch up on DBSP, my recommended flow of learning is: 1. Watch the talk: www.youtube.com/watch?v=omOH... (h/t @wslim.bsky.social) 2. Read the spec/book: mihaibudiu.github.io/work/dbsp-sp... (h/t @avi.im) 3. Read the VLDB paper List is ordered by assumed knowledge of reader

1263

v @avi.im · 20/08/2025

thankfully this is not an issue for databases, as most of them have checksums and handle corruptions gracefully

Report: Microsoft's latest Windows 11 24H2 update breaks SSDs/HDDs, may corrupt your data [Update]

161