iiinova — where your digital self keeps evolving

Reposted by Rohan Desai

Apurva Mehta @apurvamehta.com · 03/02/2025

This will be the foundation for being able to branch @kafkastreams.bsky.social apps to support seamless blue/green deploys. It also allows you to branch a previous version of the state to debug an issue from the past. Powerful stuff coming to the world of stream processing!

2134

Reposted by Rohan Desai

Sophie Blee-Goldman @ableegoldman.bsky.social · 11/12/2024

For any Kafka Streams fans out there who want to know what it's like scaling an EOS application up to multiple TB of state at ~10k+ records/s with minimal downtime -- make sure to catch the Q&A we're hosting tomorrow with Metronome. Join us Dec 12th at 9:30am PST! www.linkedin.com/events/72647...

linkedin.com

Q&A: Scaling Event Driven Architectures with Metronome | LinkedIn

We're hosting a live Q&A session with two engineers who have seen it all: - Casey Crites, a founding engineer at Metronome, implemented and scaled the Kafka pipeline at Metronome from zero to where i...

0114

Reposted by Rohan Desai

Apurva Mehta @apurvamehta.com · 04/12/2024

This was a fun conversation with @thegeeknarrator.bsky.social and @yingjunwu.bsky.social about the world of SQL stream processing on one side and event driven apps on the other. Thanks for having me on the show, Kaivalya!

183

Reposted by Rohan Desai

Apurva Mehta @apurvamehta.com · 26/11/2024

It's said that Silicon Valley is special because the density of smart and motivated people leads to chance encounters that don't happen elsewhere. I can attest to that. Here's how a coffee resulted in @responsive.dev building a database optimized for stream processing in 8 months. (1/n)

1236

Reposted by Rohan Desai

Apurva Mehta @apurvamehta.com · 16/11/2024

From one of @responsive.dev 's customers: "will report back .. when we start to change ttl lengths, which we're currently set up to do through launchdarkly based on the client and environment...pretty slick." They are changing the TTL of rows in their Kafka Streams state stores dynamically 🤩

072

Reposted by Rohan Desai

Apurva Mehta @apurvamehta.com · 20/11/2024

Welcome @ableegoldman.bsky.social to BlueSky! Kafka Streams extraordinaire!

1125

Reposted by Rohan Desai

Apurva Mehta @apurvamehta.com · 13/11/2024

Some problems are impossible to solve without stream processing: for instance, did you know that metronome.com leverages Kafka and Kafka Streams to deliver real time billing features like spend limits at scale? (1/3)

175

Reposted by Rohan Desai

Almog @almog.xyz · 10/11/2024

Coming from Kafka land, I’m always surprised at how “time” is an afterthought in many other databases. Hoping SlateDB gets it right: 1) have only one clock, 2) let users specify the clock, 3) enforce monotonic clocks, 4) use seq numbers (not time) for txns. Are we missing anything?

4132

Reposted by Rohan Desai

Diptanu Choudhury @diptanu.bsky.social · 09/11/2024

Looking forward to this! We just started using SlateDB in @tensorlake.bsky.social’s compute engine, looking forward to not needing dynamodb for CAS. One less moving part! @chris.blue when is the new release coming? :)

191

Reposted by Rohan Desai

Chris @chris.blue · 03/11/2024

👋 @vigneshc.bsky.social! All, Vinesh is a maintainer and committer for slatedb.io. He implemented the manifest and has been working on fizzbee.io proofs with JP. In his spare time, he’s a manager for Azure’s streaming team. 😁

0183

Reposted by Rohan Desai

Kaivalya Apte @thegeeknarrator.bsky.social · 03/11/2024

I am soon interviewing @apurvamehta.com from @responsive.dev and Yingjun Wu from @risingwave.bsky.social on all things stream processing. This is going to be an amazing episode. Feel free to ask any questions related to Stream processing and I will add the most interesting ones. Shoot!

6177

Reposted by Rohan Desai

Apurva Mehta @apurvamehta.com · 29/10/2024

Is it end of the road for RocksDB in stream processing? Disaggregated state is the clearly superior architecture, with @responsive.dev investing heavily in SlateDB.io while Flink 2.0 has forked RocksDB. Here's why we've bet on SlateDB for Kafka Streams: www.responsive.dev/blog/why-sla...

responsive.dev

If not RocksDB, then what? Why SlateDB is the right choice for Stream Processing.

Why we think SlateDB is the right foundation for a state storage service for Kafka Streams.

1185

Reposted by Rohan Desai

Chris @chris.blue · 28/10/2024

This is gonna be cool... "The writer will also support initializing a new database from an existing checkpoint. This allows users to 'fork' an instance of SlateDb, allowing it to access all the original db’s data from the checkpoint, but isolate writes to a new db instance."

github.com

RFC for supporting checkpoints/clones by rodesai · Pull Request #278 · slatedb/slatedb

A cloud native embedded storage engine built on object storage. - RFC for supporting checkpoints/clones by rodesai · Pull Request #278 · slatedb/slatedb

3153