Yaroslav Tkachenko @sap1ens.com · 30mI'll be speaking at the Data Streaming Summit next week. Covering one of my favourite topics: columnar data for streaming workloads. Interested in attending? Use the DSSBUDDY26 discount code to get a free ticket! 000
Yaroslav Tkachenko @sap1ens.com · 12hWant to make your streaming pipelines more efficient? I help companies make data streaming systems faster, cheaper, and more reliable - whether that means finding bottlenecks, reducing infrastructure spend, or rethinking part of the architecture. Work with me: sap1ens.com/consulting/sap1ens.comConsulting | sap1ens.comConsulting 110
Yaroslav Tkachenko @sap1ens.com · 29/09/2026Most teams that operate large-scale data streaming pipelines start to care about cost in one way or another. Streaming is perceived as "expensive", and it's frequently questioned. Today I'm attempting to address some concerns: www.streamingdata.tech/p/improving-...streamingdata.techImproving Cost Efficiency of Data Streaming PipelinesLearn practical ways to cut data streaming costs in Kafka and Flink by optimizing batching, serialization, state, network traffic, and capacity. 020
Yaroslav Tkachenko @sap1ens.com · 25/09/2026Streamhouse is old news, StreamNative has just released Open Lakestream. Lakestream is their take on materializing streaming data. streamnative.io/blog/introdu...streamnative.ioIntroducing the Stream Materialization Framework: From Streams to Governed Data AssetsLakestream's Stream Materialization Framework turns a stream into governed Iceberg, ClickHouse, or OpenSearch tables without a separate connector. 140
Yaroslav Tkachenko @sap1ens.com · 17/09/2026I've had a lot of time to reflect recently, so I decided to capture some thoughts on how my professional goals and values have evolved since becoming a full-time solopreneur. www.independentengineer.co/p/evolving-g...independentengineer.coEvolving Goals and ValuesEverything changes, including things that matter to you. 040
Yaroslav Tkachenko @sap1ens.com · 16/09/2026Just found out that Current is free this year! current.confluent.io/san-franciscocurrent.confluent.ioSan FranciscoCurrent is the biggest event in data streaming. Over two days in San Francisco, developers, data practitioners, and tech executives will come together to shape the future of Apache Kafka, Apache Flink... 000
Yaroslav Tkachenko @sap1ens.com · 11/08/2026I gave a talk about building Streamling last week at the DataFusion Community Showcase. The recording is now available! youtu.be/0-BIHyzODH8?...youtu.beDataFusion Community Showcase Vol. 3: ASAPQuery & StreamlingYouTube video by Spice AI 020
Reposted by Yaroslav TkachenkoAndrew Lamb @andrewlamb1111.bsky.social · 10/08/2026"DataFusion and Arrow allowed us to build in a few months rather than a few years" www.youtube.com/watch?v=0-BI... - Yaroslav Tkachenko from Streamling on the Community Showcase. Milind Srivastava from CMU also showed off the ASAPQuery data sketch query system.youtube.comDataFusion Community Showcase Vol. 3: ASAPQuery & StreamlingYouTube video by Spice AI 082
Yaroslav Tkachenko @sap1ens.com · 28/07/2026The New Wave of the Streaming Log Technologies www.streamingdata.tech/p/the-new-wa...streamingdata.techThe New Wave of the Streaming Log TechnologiesStreaming logs are entering a third wave: Kafka-free systems rethink performance, simplicity, routing, protocols, and object-native design. 040
Reposted by Yaroslav TkachenkoAndy Pavlo @andypavlo.bsky.social · 22/07/2026Stonebraker is not happy with existing Text2SQL tooling: cacm.acm.org/blogcacm/if-...cacm.acm.orgIf You Think You Can Do Real-World Text-to-SQL – Communications of the ACM 2316
Yaroslav Tkachenko @sap1ens.com · 22/07/2026I wrote a tutorial on using Streamling to send rich Slack notifications based on Kafka events. The tutorial highlights how you can easily combine SQL and TypeScript transformations to filter data and get it precisely into the shape you need. 130
Reposted by Yaroslav Tkachenkos2.dev @s2.dev · 20/07/2026S2 source and sink plugins are now available for Streamling. Read durable S2 streams into Arrow batches, or write Streamling output back to S2. ClickHouse ingestion, Postgres CDC, and a full Postgres → S2 → ClickHouse example: s2.dev/docs/integr... 042
Yaroslav Tkachenko @sap1ens.com · 14/07/2026ClickHouse Cloud has the best onboarding experience. 2-3 clicks, done. 010
Yaroslav Tkachenko @sap1ens.com · 25/06/2026Earlier this week, I presented a talk at the Vancouver Systems Meetup: "Streamling: Building a Lightweight Data Streaming Runtime on Apache DataFusion". 120
Yaroslav Tkachenko @sap1ens.com · 09/06/2026I'm excited to announce Streamling: lightweight, performant and extensible data streaming runtime. Streamling is built using Rust, Apache Arrow and Apache DataFusion. 130
Yaroslav Tkachenko @sap1ens.com · 20/05/2026Found a love letter to @andypavlo.bsky.social at #PgConf in Vancouver 0201
Yaroslav Tkachenko @sap1ens.com · 13/05/2026streamacademy.io/tutorial/fli...streamacademy.ioApache Flink: Postgres to Postgres Replication with Flink CDCA practical guide to building a generic, multi-table Postgres-to-Postgres replication pipeline with Flink CDC - one replication slot for any number of tables, automatic schema discovery, and a custom ... 010
Yaroslav Tkachenko @sap1ens.com · 11/05/2026www.streamingdata.tech/p/can-kafka-...streamingdata.techCan Kafka Queues Make Consumers Faster? Part 2: Head-Of-Line BlockingKafka Queues can scale consumers beyond partitions by reducing head-of-line blocking when message processing has I/O delays. 020
Reposted by Yaroslav Tkachenkormoff 🏃♂️🫖🥓 @rmoff.net · 30/04/2026Just in the nick of time, it's my round-up of Interesting Links for April 2026! There's links about all sorts of stuff around Data Engineering, CDC, Iceberg, Data architectures, and even a little hint of AI (but only the useful kind, not the gross hype-y kind). rmoff.net/2026/04/30/i...rmoff.netInteresting links - April 2026 041
Yaroslav Tkachenko @sap1ens.com · 27/04/2026www.streamingdata.tech/p/can-kafka-...streamingdata.techCan Kafka Queues Make Consumers Faster?Kafka Queues promise consumer scaling beyond partitions, but benchmarks show share consumers still lag far behind standard Kafka consumers. 010
Yaroslav Tkachenko @sap1ens.com · 26/04/2026www.independentengineer.co/p/how-to-gro...independentengineer.coHow to Grow Your NetworkLearn how software engineers can build a valuable network before going independent by helping others, nurturing work relationships, attending events, and joining communities. 020
Yaroslav Tkachenko @sap1ens.com · 20/04/2026Sharing some thoughts this morning about different paths you can take as a software engineer who's looking for more independence, and perhaps, eventually becoming a solopreneur. It's not just about building a product or becoming a consultant. www.independentengineer.co/p/software-e...independentengineer.coThe Many Ways Software Engineers Can Go IndependentHow software engineers can go independent: consulting, productized services, SaaS, courses, newsletters, and other realistic solopreneur paths. 010
Yaroslav Tkachenko @sap1ens.com · 17/04/2026Data Streaming Academy has tutorials now! The first tutorial covers reading and writing Kafka consumer offsets in Apache Flink using the State Processor API. streamacademy.io/tutorial/fli... 010
Yaroslav Tkachenko @sap1ens.com · 16/04/2026Recently, I spent a lot of time benchmarking various Iceberg sinks (Spark, Flink, Kafka Connect) and trying to beat @supermetal-inc.bsky.social for moving data from Postgres to Iceberg. Here's my report: thenewstack.io/postgres-ice...thenewstack.ioPostgres to Iceberg in 13 minutes: How Supermetal compares to Flink, Kafka Connect, and SparkSupermetal's new Iceberg sink snapshots Postgres data in 13 minutes versus 90+ for Flink, Kafka Connect, and Spark in this CDC benchmark. 020
Yaroslav Tkachenko @sap1ens.com · 15/04/2026Is this peak bubble? techcrunch.com/2026/04/15/a...techcrunch.comAfter sale of its shoe business, Allbirds pivots to AI | TechCrunchAllbirds is ditching wool sneakers for AI servers, rebranding as NewBird AI after locking in a $50M convertible financing facility. 000
Yaroslav Tkachenko @sap1ens.com · 06/04/2026Second post in The Independent Engineer (my new newsletter!) focuses on a very important question: how to (re)discover your calling as a software engineer? I believe specialization is important, especially if you'd like to explore the idea of becoming a solopreneur in the future. 120
Yaroslav Tkachenko @sap1ens.com · 06/04/2026Why does it take AWS 40 (!) minutes to provision a 3-node MSK cluster? Do they manually bring a new server rack for this? I understand why a rolling update might be slow, but creating from scratch? Really? 110
Yaroslav Tkachenko @sap1ens.com · 30/03/2026I'm starting a new newsletter! For a while, I wanted to share my experience building an independent business. Specifically, building a business as a software engineer. A business where you keep building systems, solving problems, writing code - whatever you really enjoy doing. 140
Yaroslav Tkachenko @sap1ens.com · 25/03/2026It's beautiful github.com/nicolasmd87/...github.comGitHub - nicolasmd87/aether: A compiled actor-based programming language with type inference, designed for concurrent systems.A compiled actor-based programming language with type inference, designed for concurrent systems. - nicolasmd87/aether 040
Yaroslav Tkachenko @sap1ens.com · 23/03/2026The Advanced Apache Flink training is now available as a self-paced, on-demand course! I ran the first cohort of the training in January and also had the chance to conduct a few corporate training sessions with the same program. I’ve received a lot of great feedback, iterated and polished content. 120
Reposted by Yaroslav TkachenkoAndrew Lamb @andrewlamb1111.bsky.social · 16/03/2026Recording and slides available from the DataFusion Stockholm Meetup: Recording: youtu.be/9u4cNmL14Xs Slides: github.com/apache/dataf...youtu.beApache DataFusion Meetup Stockholm, March 2026YouTube video by Andrew Lamb 042
Yaroslav Tkachenko @sap1ens.com · 08/03/2026I'm not asking much, can we open two files with the same name in 2026? 110
Reposted by Yaroslav Tkachenkormoff 🏃♂️🫖🥓 @rmoff.net · 27/02/2026✍🏻 Interesting Links in the Data & AI World, February 2026 Links from @gunnarmorling.dev, @nbuesing.bsky.social, @ssp.sh, @sap1ens.com, @neelesh.bsky.social, @rstephens.me, @justincormack.bsky.social, @sogrady.org, @simonwillison.net, @cassidoo.co, and more! 🔗 rmoff.net/2026/02/27/i...rmoff.netInteresting links - February 2026 073
Yaroslav Tkachenko @sap1ens.com · 17/02/2026Can a Rust project be 3x faster than a popular Java project right out of the box? I’m a big fan of RBIR: Rewrite Bigdata in Rust. So when @supermetal-inc.bsky.social recently launched Kafka sink support, I decided to take it for a spin and compare its performance with Debezium and Flink CDC. 110
Yaroslav Tkachenko @sap1ens.com · 02/02/2026Dream Theater not getting a Grammy in 2026 tells you everything you need to know about the recognition of the Rock/Metal bands in Grammys. 210
Yaroslav Tkachenko @sap1ens.com · 23/01/2026Engineers don't watch this talk enough and it shows (it's not about Python btw) www.youtube.com/watch?v=RZ4S...youtube.com 020
Reposted by Yaroslav TkachenkoSupermetal @supermetal-inc.bsky.social · 14/01/2026@goldskyio.bsky.social × @supermetal-inc.bsky.social 30 TB. 18.9B rows. Seven figures saved. Supermetal handled the Postgres → Postgres migration so Goldsky could keep shipping. www.supermetal.io/customers/go...supermetal.ioHow Goldsky Migrated Terabytes of Mission-Critical Web3 DataGoldsky avoided months of re-indexing time and seven-figure infrastructure TCO by directly migrating massive Graph Node PostgreSQL deployments with Supermetal 021