iiinova — where your digital self keeps evolving

I think there is also an interesting query optimization problem to be solved. You could avoid putting the variables of the delta term at the beginning of the GJ run for a particular dQ and this would result in fewer "old" state permutations for certain derived relations. Has anybody worked on this?

100

My question essentially concerns this space / time tradeoff. Is there any prior art here ? Has anybody implemented this and what tradeoff have they chosen?

100

For maximum efficiency I now need to save "old" states for these relations in potentially many different variables orders. Saving many different variable orders seems expensive. I think there is standard space / time tradeoff to be done here.

210

Every delta query in the above (when implemented) starts with the delta term and so also the variable order of the GJ run is determined by the delta term. Let's say some of the participating relations are not base relations but derived relations (or joins, not joins, predicates and fn applications).

100

I am working on a triplestore (entity-attribute-value model) so for base relations the above is "easy" to implement as I have all permutations of the primary index EAV at hand anyway.

100

I am running GJ for every delta query using ZSets. On variable levels where the variables of a participating relation have not been exhausted, I am using the multiplicative idenity (1) and only use the "real" weight on the last level.

100

The other question concerns WCOJ in connection with IVM. I am essentially trying to combine WCOJ with DBSP. After some initial back and forth I recently realized I just ended up with the following formula from your "Distributed Evaluation of Subgraph..." paper.

100

I initially implemented this still level by level, but then realized that variable unifications for C could "leak" into the other branch, so I think there is no way around isolation. I was mainly wondering how you avoid the result set copying cost or if you do any other smart thing here.

320

Let's say you now have two or branches (two rules sharing the same head) that participate in C, D now. Are you copying the result set that you have so far for (A,B) and run both branches in isolation?

100

From my understanding of Datatoad (which is likely quite shallow) you run a full WCOJ per disjunctive branch, right? To make it concrete let's you have variable order A,B,C,D and you have a partial unified result set for (A,B).

100

Finn Völkel @fivo.bsky.social · 14/05/2026

Hey @frankmcsherry.bsky.social. I have two questions. One related to Datatoad, the other related to some other work of yours. I hope that is ok.

200

Read the word Distributolog for the first time today 😂

000

Finn Völkel @fivo.bsky.social · 28/04/2026

Blogged about Triplox, a system I'm building on top of SlateDB. finnvolkel.com/triplox-log-... #slatedb #datomic

Triplox Log 1 - Introduction · Finn Völkel

020

Finn Völkel @fivo.bsky.social · 10/04/2026

Wrote some more thoughts down on multi-way joins in DBSP. finnvolkel.com/more-thought... It's reads more like a note to myself.

Frank McSherry @frankmcsherry.bsky.social · 29/03/2026

010

Reposted by Finn Völkel

As part of an attempt to get Claude to help me communicate better, we have .. a Claude-assisted mdbook for datatoad: www.frankmcsherry.org/datatoad/int... Still a work in progress, but it came together much faster with help.

frankmcsherry.org

Datatoad

051

Reposted by Finn Völkel

clojureberlin.bsky.social @clojureberlin.bsky.social · 09/02/2026

We are having a casual Get Together this Thursday 12th of February (I know it's short notice). No talk lined up, just casual chat. clojure.berlin/events/2026-...

clojure.berlin

Get together drinks @ Trespassers

045

Reposted by Finn Völkel

Finn Völkel @fivo.bsky.social · 03/01/2026

WCOJ meets DBSP. This is the post I wanted to get at when starting the series. finnvolkel.com/wcoj-wcoj-me...

WCOJ - WCOJ meets DBSP · Finn Völkel

031

Finn Völkel @fivo.bsky.social · 03/01/2026

WCOJ meets DBSP. This is the post I wanted to get at when starting the series. finnvolkel.com/wcoj-wcoj-me...

WCOJ - WCOJ meets DBSP · Finn Völkel

031

Reposted by Finn Völkel

Finn Völkel @fivo.bsky.social · 02/01/2026

The 4th post in the WCOJ series. An intro to DBSP, Z-Sets and connections to EDN Datalog indices. This will set us up for a WCOJ meets DBSP algorithm. finnvolkel.com/wcoj-dbsp-zs...

WCOJ - DBSP, ZSets and Datalog · Finn Völkel

111

Finn Völkel @fivo.bsky.social · 02/01/2026

The 4th post in the WCOJ series. An intro to DBSP, Z-Sets and connections to EDN Datalog indices. This will set us up for a WCOJ meets DBSP algorithm. finnvolkel.com/wcoj-dbsp-zs...

WCOJ - DBSP, ZSets and Datalog · Finn Völkel

111

Finn Völkel @fivo.bsky.social · 31/12/2025

Idea from a friend of mine. People who set off fireworks should be drafted immediately.

000

In section "3.1.2 A batching optimization to reduce memory" of the paper you mentioned, you seem to restrict the size of the extensions `propose` returns, but this then means batches might be quite different in size.

100

> benefit from imposing a limit on the tuples to produce at once (currently: 100M) ^from you post. When doing the joining of `a`, you don't know how many tuples you get if all variables have been unified. I was mainly wondering how you decide to cut your batches when being at the `a` level in GC.

100

I meant (2.). If the variable order is (a,b,c), naive GJ will produce all a's before looking at b's, so I was wondering what you do in datatoad (or elsewhere) besides potential cardinality estimation.

100

Finn Völkel @fivo.bsky.social · 15/12/2025

Hey. Love that you are blogging on WCOJ again. I was wondering how you deal with the limit on the tuples produced in GenericJoin. In Leapfrog this seems quite easy as you are producing the tuples one at a time at the bottom, but in GenericJoin you need (?) to take some decisions higher up or not?

200

Reposted by Finn Völkel

The 3rd post in the WCOJ series. Implementing some logical datalog connectors in the context of GenericJoin. finnvolkel.com/wcoj-datalog...

WCOJ - Datalog and GenericJoin · Finn Völkel

Finn Völkel @fivo.bsky.social · 15/12/2025

The 3rd post in the WCOJ series. Implementing some logical datalog connectors in the context of GenericJoin. finnvolkel.com/wcoj-datalog...

WCOJ - Datalog and GenericJoin · Finn Völkel

Finn Völkel @fivo.bsky.social · 11/12/2025

Reposted by Finn Völkel

The second post in the series is on an actual implementation of a WCOJ algorithm. finnvolkel.com/wcoj-generic...

WCOJ - Generic Join · Finn Völkel

Finn Völkel @fivo.bsky.social · 11/12/2025

The second post in the series is on an actual implementation of a WCOJ algorithm. finnvolkel.com/wcoj-generic...

WCOJ - Generic Join · Finn Völkel

Finn Völkel @fivo.bsky.social · 10/12/2025

I started to write a blog series on worst-case optimal join finnvolkel.com/wcoj-graph-j...

WCOJ - Graph-Join correspondence · Finn Völkel

100

Finn Völkel @fivo.bsky.social · 24/11/2025

Wrote a blog post about continuation-passing style in @xtdb.com. finnvolkel.com/continuation...