iiinova — where your digital self keeps evolving

Micah Wylde @micahw.com · 21/01/2026

S2 is incredibly cool, and now you can run it yourself!

Reposted by Micah Wylde

Craig Dennis @craigsdennis.dev · 21/01/2026

I built a tool called LinkedOut to solve the "links in the air" problem during my talks. It’s a serverless data lake built on the @cloudflare.social Data Platform. Ingest: Pipelines Store: R2 + Apache Iceberg Security: Access Real-time analytics with zero egress fees. Full build video below.

231

Reposted by Micah Wylde

Jeremy Morrell @jeremymorrell.dev · 27/09/2025

Another brand new new feature is the R2 data catalog: blog.cloudflare.com/cloudflare-d... Build something with Pipelines and R2 SQL. I suggest receiving OpenTelemetry data and then surfacing that in a web app (logs should be fairly straightforward), but there are tons of uses for this.

blog.cloudflare.com

Announcing the Cloudflare Data Platform: ingest, store, and query your data directly on Cloudflare

The Cloudflare Data Platform, launching today, is a fully-managed suite of products for ingesting, transforming, storing, and querying analytical data, built on Apache Iceberg and R2 storage.

132

Reposted by Micah Wylde

Jeremy Morrell @jeremymorrell.dev · 25/09/2025

It's early, but I'm excited about direction that the Cloudflare Data Platform is taking. Trying to set up similar pipelines on other clouds would typically be $$$ and take tons of expertise. Managing kafka and multiple services for ingestion, compaction, etc blog.cloudflare.com/cloudflare-d...

blog.cloudflare.com

Announcing the Cloudflare Data Platform: ingest, store, and query your data directly on Cloudflare

The Cloudflare Data Platform, launching today, is a fully-managed suite of products for ingesting, transforming, storing, and querying analytical data, built on Apache Iceberg and R2 storage.

381

Micah Wylde @micahw.com · 25/09/2025

The news is finally out! Cloudflare has a Data Platform! We're starting with serverless streaming pipelines (powered by arroyo), a managed Iceberg Catalog, and a new distributed SQL engine built on top of DataFusion

0120

Reposted by Micah Wylde

Cloudflare @cloudflare.social · 04/06/2025

Cloudflare is at Snowflake Summit in San Francisco this week! Swing by our booth 2605 to chat about the new Cloudflare R2 Data Catalog and how it can make your data management and analytics easier!

082

Micah Wylde @micahw.com · 27/05/2025

Next Monday after the Snowflake Summit keynote! Hang out on our beautiful roof with other cool data folks, and hear some great speakers from LanceDB, @mooncakelabs.bsky.social, Eventual, Marimo, Bobsled, and @cloudflare-dev.bsky.social! lu.ma/dbq1hfij

lu.ma

Modern Data w/ Cloudflare + Friends · Luma

Come talk about modern data formats, streaming ingestion, query engines and how you feel about Iceberg at Cloudflare's HQ. We'll be running a series of…

010

Reposted by Micah Wylde

Chris @chris.blue · 18/04/2025

Ok, y'all. This took me several weeks and a ton of help from @frankmcsherry.bsky.social and @lalithsuresh.bsky.social. I dug into timely dataflow, differential dataflow, and DBSP to get you up to speed on IVM engines and materialized views. Enjoy!

materializedview.io

Everything You Need to Know About Incremental View Maintenance

An overview of incremental view maintenance, why it’s useful, and how you can implement it.

47618

Micah Wylde @micahw.com · 16/04/2025

I’m only a week into life at @cloudflare-dev.bsky.social but already amazed by how much of Cloudflare is built _on_ Cloudflare. I’d never have guessed you could get so far with just workers + durable objects!

000

Reposted by Micah Wylde

rmoff 🏃‍♂️🫖🥓 @rmoff.net · 10/04/2025

Couple of big announcements from @cloudflare.social today for folk in #dataBS: * Acquisition of Arroyo, launch of Pipelines for streaming ingestion: blog.cloudflare.com/cloudflare-a... * Launch of R2 Data Catalog—a managed Apache Iceberg catalog for R2 blog.cloudflare.com/r2-data-cata...

blog.cloudflare.com

Just landed: streaming ingestion on Cloudflare with Arroyo and Pipelines

We’ve just shipped our new streaming ingestion service, Pipelines — and we’ve acquired Arroyo, enabling us to bring new SQL-based, stateful transformations to Pipelines and R2.

093

Micah Wylde @micahw.com · 26/03/2025

Arroyo 0.14.0 is now available, including new lookup joins, support for nested updating aggregates, struct types, new syntax, and a bunch of improvements and fixes: www.arroyo.dev/blog/arroyo-...

arroyo.dev

Announcing Arroyo 0.14.0

Arroyo 0.14 is now available! This release introduces support for lookup joins, more powerful updating SQL, new syntax, structs in DDL, and more!

010

Micah Wylde @micahw.com · 01/03/2025

Arroyo is sitting at 3,999 stars... who's going to put us over the top github.com/ArroyoSystem...

130

Micah Wylde @micahw.com · 25/02/2025

You'd think that the key to being a fast streaming engine is like clever join algorithms, but it's mostly just being really good at JSON. Arroyo uses Arrow and the arrow-rs JSON decoder along with some streaming extensions. I think it's pretty cool, so I wrote up a long explanation of how it works

arroyo.dev

Fast columnar JSON decoding with arrow-rs

JSON is the most common serialization format used in streaming pipelines, so it pays to be able to deserialize it fast. This post covers in detail how the arrow-json library works to perform very effi...

0121

Micah Wylde @micahw.com · 23/01/2025

Our team at Arroyo recently needed to rebuild our (very ad-hoc) analytics infra to account for our growth. We spent some time working out the best way to set up a near-real-time data lake today, and ended up with a pretty sweet approach we're calling the LOAD stack: www.arroyo.dev/blog/buildin...

arroyo.dev

Building a near-real-time data lake with the LOAD stack

The LOAD stack (log storage/object storage/Arroyo/DuckDB) makes it easy to build an affordable real-time data lake with minimal operational overhead. This tutorial will guide you through the process o...

172

Micah Wylde @micahw.com · 19/12/2024

Arroyo 0.13.0 is now available! This one includes some big improvements to the core engine a (including the operator chaining work I wrote about previously: bsky.app/profile/mica...) and a bunch of other features. All the details on our blog: www.arroyo.dev/blog/arroyo-...

arroyo.dev

Announcing Arroyo 0.13.0

Arroyo 0.13 is now available! This release introduces support for reading source metadata, a RabbitMQ connector, improved CDC support, operator chaining, along with many other improvements.

140

Micah Wylde @micahw.com · 05/12/2024

Is #DataBS interested in the internals of streaming engines? The next release of arroyo.dev (0.13) has a new feature in the core dataflow called operator chaining which gets at some of the interesting details of how these systems work. So let’s dive in to streaming dataflow 🧵

arroyo.dev

Arroyo — Cloud-native stream processing

Arroyo is the easiest way to run SQL queries against your streaming data

1375

Micah Wylde @micahw.com · 05/12/2024

nothing like a potential natural disaster to bring us all together here 🤗

000

Micah Wylde @micahw.com · 12/11/2024

Happy new Jepsen Report Day to all who celebrate! jepsen.io/analyses/buf... Confirms my priors that almost no one should be directly calling the Kafka client today—use a stream processing engine for data use cases or a durable execution engine for applications.

jepsen.io

Jepsen: Bufstream 0.1.0

140

Micah Wylde @micahw.com · 30/10/2024

If you missed #p99conf last week, talks are now available to stream on YouTube. I spoke about the design decisions that went into Arroyo's incredible performance: youtube.com/watch?v=7H4C... Come for the Rust hot takes, stay for my terrible hand-drawn architecture diagrams 😅

youtube.com

YouTube

Share your videos with friends, family, and the world

022

Micah Wylde @micahw.com · 28/10/2024

Guess we're all here now 👋

1120