Sign in

Jepsen

@jepsen.mastodon.jepsen.io.ap.brid.gy
2.1K followers 1 following 59 posts

Breaking distributed systems, one fault at a time. [bridged from mastodon.jepsen.io/@jepsen on the fediverse by fed.brid.gy ]

PostsRepliesMedia
Jepsen @jepsen.mastodon.jepsen.io.ap.brid.gy · 22/09/2026
Jepsen 0.3.14 is now available, including several new generators, a slew of bugfixes and performance improvements especially in generators, and some new features for the web interface. github.com/jepsen-io/jepsen/release…
github.com
Release v0.3.14 · jepsen-io/jepsen
This release focuses on generator performance and correctness, and introduces several new generators. It also improves the web interface and fixes some long-standing issues with time zones. Bugfixe...
051
Jepsen @jepsen.mastodon.jepsen.io.ap.brid.gy · 19/08/2026
Heads-up for Elle users--I believe there's another bug in the `rw` workload which causes `linearizable-keys?` and friends to incorrectly report cyclic version errors for versions which are not in fact cyclic. Working on that now.
010
Jepsen @jepsen.mastodon.jepsen.io.ap.brid.gy · 31/07/2026
A new #Jepsen release to support caching expensive generators and final reads for cycle-detection tests: github.com/jepsen-io/jepsen/release…
github.com
Release 0.3.13 · jepsen-io/jepsen
This is a small release. It makes expensive generators more efficient with a cache, allows rw and append tests a final chance to detect data loss at the end of tests, and lets file snapshot nemeses...
030
Jepsen @jepsen.mastodon.jepsen.io.ap.brid.gy · 20/07/2026
Delighted to announce the release of #Jepsen 0.3.12, now available on Clojars and GitHub, as usual. github.com/jepsen-io/jepsen/release…
github.com
Release 0.3.12 · jepsen-io/jepsen
A big release! This fixes several deadlocks and a safety error in Elle's inference of version orders for rw workloads which caused tests to incorrectly report that version orders were cyclic. It al...
050
Jepsen @jepsen.mastodon.jepsen.io.ap.brid.gy · 14/07/2026
Could someone please poke the AWS Marketplace support team for me? There's a bug in the add instances workflow, they no longer accept emails, and the web form they insist on using for filing tickets only accepts 1000 characters and seems to have an 80% chance of routing directly to /dev/null.
020
Jepsen @jepsen.mastodon.jepsen.io.ap.brid.gy · 29/06/2026
Missed this post on MariaDB's Snapshot Isolation implementation from Percona: www.percona.com/blog/mariadbs-snaps… In short, instead of allowing transactions to silently lose updates, MariaDB now (mostly) returns an error so the […]
mastodon.jepsen.io
Original post on mastodon.jepsen.io
141
Jepsen @jepsen.mastodon.jepsen.io.ap.brid.gy · 06/06/2026
AWS changed a few things about CloudFormation that broke the Jepsen cluster product. Version 0.3.11.2 should work now: github.com/jepsen-io/jepsen#aws
010
Jepsen @jepsen.mastodon.jepsen.io.ap.brid.gy · 26/03/2026
If you want to chat about checking predicate safety, come chat at BugBash 2026. April 22-24, DC: antithesis.com/bugbash/conference20…
antithesis.com
BugBash 2026
A conference on extracting reliable software from the slop factory
040
Jepsen @jepsen.mastodon.jepsen.io.ap.brid.gy · 17/03/2026
A new #Jepsen report, on MariaDB Galera Cluster 12.1.2. Two cases of write loss, along with P4 (Lost Update) and Stale Read in healthy clusters. jepsen.io/analyses/mariadb-galera-c…
jepsen.io
Jepsen: MariaDB Galera Cluster 12.1.2
076
Jepsen @jepsen.mastodon.jepsen.io.ap.brid.gy · 10/03/2026
Jepsen 0.3.11 is now available on Clojars. This is a fairly small release, but it does update Jepsen to work with DB nodes running Debian Trixie. github.com/jepsen-io/jepsen/release…
github.com
Release 0.3.11 · jepsen-io/jepsen
This is a fairly minor release. It updates Jepsen to work with Debian Trixie DB nodes, fixes a few minor bugs, and introduces some very small quality-of-life improvements, along with the usual slew...
041
Jepsen @jepsen.mastodon.jepsen.io.ap.brid.gy · 08/12/2025
A new #Jepsen report: we demonstrate data loss and persistent split-brain in the NATS streaming system, in response to simulated power failures/OS crashes. jepsen.io/analyses/nats-2.12.1
jepsen.io
Jepsen: NATS 2.12.1
12412
Jepsen @jepsen.mastodon.jepsen.io.ap.brid.gy · 02/12/2025
Jepsen and Antithesis worked together to write a glossary for anyone building, testing, and operating distributed systems. It covers the basics of concurrency, consistency models and phenomena, faults, and some testing approaches: antithesis.com/resources/reliabilit… […]
mastodon.jepsen.io
Original post on mastodon.jepsen.io
0165
Jepsen @jepsen.mastodon.jepsen.io.ap.brid.gy · 02/12/2025
A new #Jepsen release, 0.3.10, brings improved support for controllable random value generation, and running tests inside Antithesis. Jepsen's composable generator system has also been extracted to a minimal library, making it easier to re-use in other systems […]
mastodon.jepsen.io
Original post on mastodon.jepsen.io
040
Jepsen @jepsen.mastodon.jepsen.io.ap.brid.gy · 18/08/2025
The latest Jepsen talk, from Systems Distributed in June, goes live in 15 minutes. We'll be doing a live chat during the premier, if you want to chat about databases and testing. :-) www.youtube.com/watch?v=dpTxWePmW5Y
054
Jepsen @jepsen.mastodon.jepsen.io.ap.brid.gy · 07/08/2025
A new #Jepsen report! We tested early builds of Capela, an unreleased distributed programming environment, and found twenty-two issues, including four language problems, fourteen crashes or non-fatal panics, performance degradation, and three safety issues including lost update […]
mastodon.jepsen.io
Original post on mastodon.jepsen.io
083
Jepsen @jepsen.mastodon.jepsen.io.ap.brid.gy · 06/08/2025
An interview with Kaivalya Apte, on The GeekNarrator podcast. We talk about mapping properties to tests, type I and II errors, performance, LLMs, and more. www.youtube.com/watch?v=IvE1VbOol88
071
Jepsen @jepsen.mastodon.jepsen.io.ap.brid.gy · 05/08/2025
The video of my BugBash talk, "Jepsen 17: ACID Jazz" is out now! www.youtube.com/watch?v=v8cG2hh10SQ
061
Jepsen @jepsen.mastodon.jepsen.io.ap.brid.gy · 15/07/2025
Antithesis and Jepsen are releasing a glossary of terms useful in distributed systems testing: antithesis.com/resources/reliabilit…
antithesis.com
A distributed systems reliability glossary
A list of key concepts for building and testing reliable distributed systems, with basic definitions and deep references.
02113
Jepsen @jepsen.mastodon.jepsen.io.ap.brid.gy · 07/06/2025
Systems Distributed. June 19-20, Amsterdam. systemsdistributed.com
A parody of John Waters' "Serial Mom", except it's "Serializable Mom". I'm holding scissors (to partition the network) and trying to channel my best homage to Kathleen Turner.
3505
Jepsen @jepsen.mastodon.jepsen.io.ap.brid.gy · 06/06/2025
A new #Jepsen report! We worked with TigerBeetle to find seven crashes, elevated latencies during single-node failures, and requests which were retried forever in version 0.16.11. We found only two safety issues: missing results for queries with multiple predicates, and incorrect timestamps in a […]
mastodon.jepsen.io
Original post on mastodon.jepsen.io
14112
Jepsen @jepsen.mastodon.jepsen.io.ap.brid.gy · 07/05/2025
Jepsen 0.3.9 is now available, including a module for restarting flaky databases, a few improvements to downloading logs, more capable generators, and friendlier error messages. github.com/jepsen-io/jepsen/release…
github.com
Release 0.3.9 · jepsen-io/jepsen
A medium-sized release, this version makes several quality-of-life improvements. We (finally!) download log files only once, rather than twice at the end of a run; this should make multi-GB log dow...
091
Jepsen @jepsen.mastodon.jepsen.io.ap.brid.gy · 29/04/2025
A small issue in Amazon RDS for PostgreSQL: at the "Repeatable Read” isolation level, which in PostgreSQL normally means Snapshot Isolation, Amazon RDS for PostgreSQL clusters appear to exhibit Long Fork. We observed this behavior in healthy clusters, in versions ranging from 13.15 to 17.4 […]
mastodon.jepsen.io
Original post on mastodon.jepsen.io
11915
Jepsen @jepsen.mastodon.jepsen.io.ap.brid.gy · 30/03/2025
Added four new phenomena to Jepsen's docs: P4 (Lost Update), A5A (Read Skew), A5B (Write Skew) and Process. jepsen.io/consistency/phenomena#sql
091
Jepsen @jepsen.mastodon.jepsen.io.ap.brid.gy · 06/03/2025
Jepsen 0.3.8 is now available. It includes a new nemesis for file corruption, and improvements to clock-skew tests. github.com/jepsen-io/jepsen/release…
github.com
Release 0.3.8 · jepsen-io/jepsen
The centerpiece of 0.3.8 is a new nemesis for corrupting files: jepsen.nemesis.file. This nemesis can be scoped to specific regions of a file, is aware of chunk structure (e.g. database pages) and ...
040
Jepsen @jepsen.mastodon.jepsen.io.ap.brid.gy · 05/03/2025
What IS Strong Serializability, really? Ever want to try writing your own gossip service. Two open sessions of Jepsen's training classes are coming up: the Distributed Systems Fundamentals class, and (for the first time!) its accompanying workshop […]
mastodon.jepsen.io
Original post on mastodon.jepsen.io
1133
Jepsen @jepsen.mastodon.jepsen.io.ap.brid.gy · 05/03/2025
I'll be speaking on Jepsen at Bug Bash (DC, April 3-4), and Systems Distributed (Amsterdam, June 19-20). Come join! bugbash.antithesis.com systemsdistributed.com
bugbash.antithesis.com
BugBash 2025: software reliability conference
Join us at a tech conference organized by Antithesis to explore reliable software development. April 3-4, 2024 at Yours Truly DC hotel, Washington D.C.
0113
Jepsen @jepsen.mastodon.jepsen.io.ap.brid.gy · 21/01/2025
So, uh, the last time I had to call malloc() was a quarter century ago, and I am struggling to do basic tasks without corrupting the heap. I would love to hire one of you stripey-socked C witches for a very small contract to help finish […]
mastodon.jepsen.io
Original post on mastodon.jepsen.io
12520
Jepsen @jepsen.mastodon.jepsen.io.ap.brid.gy · 22/12/2024
Added descriptions of the SQL isolation level anomalies P0, P1, P2, and P3 to the phenomena page: jepsen.io/consistency/phenomena#sql
0214
Jepsen @jepsen.mastodon.jepsen.io.ap.brid.gy · 04/12/2024
Released version 0.2.4 of Maelstrom, Jepsen's workbench for writing toy distributed systems: github.com/jepsen-io/maelstrom
github.com
GitHub - jepsen-io/maelstrom: A workbench for writing toy implementations of distributed systems.
A workbench for writing toy implementations of distributed systems. - jepsen-io/maelstrom
080
Jepsen @jepsen.mastodon.jepsen.io.ap.brid.gy · 04/12/2024
There's a few tickets left for the distributed systems class coming up in just over a week. If you'd like to join, now's the time. :-) www.eventbrite.com/e/distributed-sy…
eventbrite.com
Distributed Systems Fundamentals
Learn the theory and practice behind distributed systems, from safety and liveness to deployment and monitoring.
094
Jepsen @jepsen.mastodon.jepsen.io.ap.brid.gy · 03/12/2024
Bin Wang put together a Jepsen test for Patroni, a PostgreSQL replication system. All sorts of good stuff in here, including that the cluster can't handle a series of single-node failures: www.binwang.me/2024-12-02-PostgreSQ…
binwang.me
Jepsen Test on Patroni: A PostgreSQL High Availability Solution | Bin Wang - My Personal Blog
0163
Jepsen @jepsen.mastodon.jepsen.io.ap.brid.gy · 27/11/2024
Antithesis, Buf, and Jepsen are running a joint webinar on December 5th. We'll discuss a Kafka protocol safety issue, talk about the challenges of distributed systems testing, and show how Jepsen and Antithesis helped identify critical safety errors in Bufstream. Come watch Antithesis pause […]
mastodon.jepsen.io
Original post on mastodon.jepsen.io
0198
Jepsen @jepsen.mastodon.jepsen.io.ap.brid.gy · 12/11/2024
"That's never been my understanding of auto-commit, that would be a crazy default wouldn't it?" I feel like there's a real need for database user research--building a quantitative and qualitative picture of how actual users understand the systems they use […]
mastodon.jepsen.io
Original post on mastodon.jepsen.io
180
Jepsen @jepsen.mastodon.jepsen.io.ap.brid.gy · 12/11/2024
A new #Jepsen report! We worked with Buf to analyze the safety of Bufstream, a Kafka-compatible streaming system. We found three safety and two liveness issues in Bufstream 0.1.0, including the loss of acknowledged writes in healthy clusters. These problems were resolved by version 0.1.3 […]
mastodon.jepsen.io
Original post on mastodon.jepsen.io
12610
Jepsen @jepsen.mastodon.jepsen.io.ap.brid.gy · 07/11/2024
Last year we reported that MySQL and MariaDB's "Repeatable Read" was badly broken. The MariaDB team has been hard at work, and they've added a new flag, `--innodb-snapshot-isolation=true`, which causes "Repeatable Read" to prevent Lost Update, Non-repeatable Read, and violations of Monotonic […]
mastodon.jepsen.io
Original post on mastodon.jepsen.io
120
Jepsen @jepsen.mastodon.jepsen.io.ap.brid.gy · 26/10/2024
If you've been waiting for a chance to take Jepsen's distributed systems class, we've got great news. We're running a special session December 16-19 that anyone can register for. www.eventbrite.com/e/distributed-sy…
0195
Jepsen @jepsen.mastodon.jepsen.io.ap.brid.gy · 24/10/2024
Just want to give a shout out to Irene Kannyo, who's edited every Jepsen analysis over the last four years. She brings her background as scientist and writer to our industry-focused reports, maintains an evolving style guide, streamlines prose, ensures diagrams match descriptions, and so much […]
mastodon.jepsen.io
Original post on mastodon.jepsen.io
071
Jepsen @jepsen.mastodon.jepsen.io.ap.brid.gy · 18/10/2024
#Jepsen 0.3.6 is now available on GitHub and Clojars. This is a sizeable release. It includes a significant correctness bugfix for a rare bug that could make operations in the history print with the wrong data. It also adds a new namespace for composing databases, nemeses, and generators when […]
mastodon.jepsen.io
Original post on mastodon.jepsen.io
021
Jepsen @jepsen.mastodon.jepsen.io.ap.brid.gy · 15/10/2024
Just a heads-up: I'll be speaking at GOTO Chicago next Tuesday, 2024-10-24, on JVM performance techniques in Jepsen: gotochgo.com/2024/schedule?date=202…
120
Jepsen @jepsen.mastodon.jepsen.io.ap.brid.gy · 29/08/2024
Jepsen 16: I Can't Believe It's Not ACID, delivered at Systems Distributed 2024. RavenDB, MariaDB/MySQL, RedPanda. www.youtube.com/watch?v=ecZp6cWhDjg…
002
Jepsen @jepsen.mastodon.jepsen.io.ap.brid.gy · 08/08/2024
A new, short #Jepsen report is out on jetcd 0.8.2. Jetcd is the official Java client library for etcd. We show that it contains an improper retry mechanism which causes lost update, circular information flow, and aborted read. I hope you enjoy it! jepsen.io/analyses/jetcd-0.8.2
120
Jepsen @jepsen.mastodon.jepsen.io.ap.brid.gy · 23/07/2024
I'll be speaking at GOTO Chicago on performance techniques in Jepsen. It's gonna be a weird collage of parallelism, pure functions, immutable data structures, and deforestation with low-level techniques like bitsets, avoiding sharing between threads, packing structures into mutable arrays […]
mastodon.jepsen.io
Original post on mastodon.jepsen.io
020
Jepsen @jepsen.mastodon.jepsen.io.ap.brid.gy · 18/07/2024
Jepsen has a blog now, with RSS and Atom feeds: jepsen.io/blog.
050
Jepsen @jepsen.mastodon.jepsen.io.ap.brid.gy · 21/05/2024
I'm going to speak about MySQL, RavenDB, and Datomic at Systems Distributed 2024, June 27 & 28th, in New York: systemsdistributed.com/#about-confe…
020
Jepsen @jepsen.mastodon.jepsen.io.ap.brid.gy · 15/05/2024
Pleased to announce a new #Jepsen report on Datomic Pro 1.0.7075. Datomic is an unusual system in many ways. Temporal! Immutable! Datalog! Strong Session Serializable*! We found that Datomic appears to offer Strong Session Serializable inter-transaction semantics, up to Strong Serializable when…
mastodon.jepsen.io
Original post on mastodon.jepsen.io
31411