iiinova — where your digital self keeps evolving

YouTube video by Hasgeek TV

This is the theory Feldera is built on, and the reason we can deliver sub-second data freshness on massive, ever-changing data. 🎥 Watch the full talk: www.youtube.com/watch?v=4ORe...

youtube.com

Incremental Computation

000

Any SQL? Yes, even that monstrous 50,000-line SQL program with thousands of joins, deeply nested views, aggregations, sliding windows, and recursive queries. All of it can run incrementally.

100

At Hasgeek Rootconf’s special edition on Databases, Feldera’s co-founder and Chief Scientist, Mihai Budiu opened the conference by explaining DBSP, the mathematical theory that proves *any* SQL can be turned into an incremental version that only computes on what changes.

Kris Jenkins @krisajenkins.bsky.social · 08/07/2026

“It’s not a system improvement. It’s a fundamentally different way of doing things.”

100

Reposted by Feldera

There’s a fascinating algorithm called DBSP that single-handedly managed to combine streaming and batch systems and solve a decades-old database efficiency problem. Lalith Suresh joined me to explain the core of how it works: 😊 youtu.be/CyvnH8OUCUA

youtu.be

Making Materialized Views Actually Fast with DBSP (with Lalith Suresh)

YouTube video by Developer Voices

0136

Feldera @feldera.bsky.social · 08/07/2026

Try Felderize: github.com/feldera/feld... P.S. Fred, our chief mascot, asked why the skill wasn't named after him. We reminded him he hasn't shipped anything this quarter. He reminded us he's a flying fish.

030

Feldera @feldera.bsky.social · 08/07/2026

Now you can level up your Spark batch pipelines to true Incremental View Maintenance (IVM) in a single prompt. Most Spark SQL runs on Feldera unmodified, producing row-by-row identical results. Felderize handles the rest. Going from Spark to Feldera has never been easier.

120

Feldera @feldera.bsky.social · 08/07/2026

How do customers convert 10,000+ lines of production Spark SQL into running Feldera pipelines in minutes? They hand it to their AI agents powered by our latest skill, Felderize.

130

✨ On Feldera: all 15 views maintained incrementally, and the total runtime was <1 second. That’s True IVM: the engine only computes what changed. No fallback to costly recompute, and always continuously fresh.

🗜️ Forced incremental mode: might look better, as Databricks only fell back once to full recompute. But there was no runtime improvement. Still 58 seconds.

100

🚗 Auto mode: Databricks chose the refresh strategy itself and fell back to costly recompute on 11 of the 15 of the views. Total runtime: 58 seconds.

100

We implemented the same set of queries over the same data in both systems. The workload models a classic medallion pipeline: 15 Silver and Gold views over orders, inventory, and clickstream data.

100

Once you've experienced a True Incremental View Maintenance (IVM) engine, recomputing everything from scratch sounds absolutely absurd. Too slow and too expensive to justify. We set up a little match of our own: Databricks Incrementally Materialized Views vs. Feldera.

121

Feldera will ingest the current snapshot, follow the Delta transaction log, incrementally maintain your views, and write changes back to Delta Lake within seconds. The downside? You may never wait for a batch run again. We're sorry in advance for the inconvenience. Try it at try.feldera.com

try.feldera.com

000

If you run Spark batch jobs over Delta Lake and want to improve data freshness from hours to seconds, you can now point Feldera directly at your existing tables.

100

We just improved Feldera support for deletion vectors and column mapping in Delta Lake tables. What does this mean for you? You can now experience True Incremental View Maintenance without having to move your data.

Feldera @feldera.bsky.social · 23/06/2026

The first batch covers setup, deployment, documentation search, and Spark SQL translation. Everything an agent needs to use Feldera easily, right out of the box. What skill should we build next?

100

Feldera @feldera.bsky.social · 23/06/2026

80% of enterprise Feldera pipelines are now authored by agents. So we built them skills.

100

Feldera @feldera.bsky.social · 19/06/2026

The benchmark source code is available here: github.com/feldera/feld...

github.com

000

Feldera @feldera.bsky.social · 19/06/2026

Watch what happens when we run a simple 1v1 comparison against ClickHouse. As we turn up the volume on the transaction records, things start to really cook. Just not for us. Feldera barely broke a sweat. Incremental view maintenance never degrades at scale.

100

🧠 Complex SQL: rolling aggregates, joins, nested views, etc. ⚡ Updates computed in milliseconds and reflected in the Gold layer in <1s. 🚫 No fallback to full recomputation This is a real-time medallion architecture running on your existing SQL and Delta Lake tables. Try it today: try.feldera.com

000

There is no substitute for true Incremental View Maintenance. Our forward-deployed engineer, @Anand, built a reference implementation that shows how to transform your existing medallion architecture into a real-time pipeline with:

100

No matter how much compute—or money—you throw at Spark, it still struggles to deliver truly fresh data. Microbatching and Spark's incrementally refreshed materialized views may look promising, but complex queries quickly expose their limits, forcing them to fall back to full recomputation.

100

Right now, somewhere, a data engineer is getting another message from stakeholders: “Why is the Gold layer late?” It is a familiar reminder that the traditional medallion architecture was not built for real-time freshness. It is being held back by legacy batch processing.

121

How Feldera Customers Slash Cloud Spend (10x and beyond)

A month later, the data volume grew 10x to 2TB, and the latency is still under 200ms! With Feldera, the latency and compute required is always guaranteed to be O(delta), no matter how complex your SQL pipeline. Learn more: www.feldera.com/blog/how-fel...

feldera.com

By only needing compute resources proportional to the size of the change, instead of the size of the whole dataset, businesses can dramatically slash compute spend for their analytics.

000

When they first launched this pipeline for a subset of their data (200GB), the latency for all views to update when inputs changed, was under 200ms.

100

Here's one customer's query plan from a medallion production pipeline: 61 input tables feeding 33 output views, with 217 joins, 27 aggregations, and 287 other operators, all maintained incrementally using a laptop worth of resources.

100

A true incremental compute engine's latency should be *flat* no matter how much your data grows (O(delta) vs O(size of data)). Let's see this in action.

Feldera stayed under 3 seconds even at 1 BILLION records. This is because Feldera is a true incremental view maintenance engine. We only update what changed, for any arbitrarily complex SQL, at any scale. So as your transaction volume grows, the more expensive every second of delay becomes.

000

Here’s what latency looks like as transaction volume grows: 😎 20M transactions: 4.6s - your dashboards are keeping up, for the most part 😳 200M transactions: 32s - refreshes start lagging behind real transactions 🥵 1B transactions: 226s - by the time fraud shows up, it’s too late

100

He benchmarked Feldera against Clickhouse, which is one of the fastest analytical databases in existence and perfectly suited for this type of workload. It held up fine at 20M records. But as he turned up the volume of records, this is where things got spicy!

100

To simulate a live event stream, he ingested batches of 1K+ updates, then he defined several simple views like sliding-window aggregates, enrichment joins, etc.

100

Our CTO, Leonid Ryzhyk ran a benchmark derived from user workloads we see in the wild against ClickHouse. He started with a small dataset with 20M credit card transactions.

100