Sign in

Igfasouza

@igfasouza.bsky.social
55 followers 61 following 837 posts

Software Engineer in a Data Engineer team 🇧🇷🇮🇪 Views are my own •Java •Kafka •Raspberry PI •Maker

PostsRepliesMedia
Igfasouza @igfasouza.bsky.social · 46m
Beyond the Partition Limit: How KIP-932 Enables Elastic Consumer Scaling #apachekafka offsetzero.hashnode.dev/beyond-the-p...
offsetzero.hashnode.dev
000
Igfasouza @igfasouza.bsky.social · 01/10/2026
Handling Long-Running Tasks in KafkaShareConsumer #apachekafka offsetzero.hashnode.dev/handling-lon...
offsetzero.hashnode.dev
000
Igfasouza @igfasouza.bsky.social · 30/09/2026
Improving Cost Efficiency of Data Streaming Pipelines #apachekafka www.streamingdata.tech/p/improving-...
streamingdata.tech
Improving Cost Efficiency of Data Streaming Pipelines
Learn practical ways to cut data streaming costs in Kafka and Flink by optimizing batching, serialization, state, network traffic, and capacity.
000
Igfasouza @igfasouza.bsky.social · 29/09/2026
Consumer Recovery Contracts With Testcontainers and JUnit #apachekafka foojay.io/today/testin...
foojay.io
Testing Kafka Consumer Recovery Contracts With Testcontainers and JUnit
Use Testcontainers and JUnit to test Kafka consumer recovery contracts for duplicate delivery, malformed records, and offset-boundary failures.
000
Igfasouza @igfasouza.bsky.social · 28/09/2026
What Still Belongs in a Kafka Broker? #apachekafka medium.com/fresha-data-...
medium.com
What Still Belongs in a Kafka Broker?
Let’s suppose a Kafka cluster is handling roughly the same amount of traffic it did last year, but we now want to keep six months of data…
000
Igfasouza @igfasouza.bsky.social · 27/09/2026
We moved file ingestion to Kafka and lost the progress bar #apachekafka medium.com/@prashant.mo...
medium.com
We moved file ingestion to Kafka and lost the progress bar
A customer uploaded a 900,000-row CSV and asked us how far along it was. We couldn’t tell them.
000
Igfasouza @igfasouza.bsky.social · 26/09/2026
A Decade with Kafka: Why I Keep Coming Back to the Same Architectural Choice #apachekafka medium.com/@divyaprakas...
medium.com
A Decade with Kafka: Why I Keep Coming Back to the Same Architectural Choice
This article reflects lessons learned from my experience designing data, telemetry, and observability platforms over the past decade.
000
Igfasouza @igfasouza.bsky.social · 25/09/2026
Kafka Doesn’t Replicate — It Mirrors. Here’s Why That Changes Everything #apachekafka medium.com/@milaram.doc...
medium.com
Kafka Doesn’t Replicate — It Mirrors. Here’s Why That Changes Everything
A simple guide to why Kafka copies data differently than your database does — and why that’s a good thing.
000
Igfasouza @igfasouza.bsky.social · 24/09/2026
Kafka is not a queue #apachekafka elearning.morpheralabs.com/docs/part-07...
elearning.morpheralabs.com
Chapter 2: Building with Kafka | The Data Foundry
Imagine you've just joined an online shop as its first data engineer.
001
Igfasouza @igfasouza.bsky.social · 23/09/2026
Spring Boot Transactional Outbox with Kafka & PostgreSQL #apachekafka github.com/KHolodilin/s...
github.com
GitHub - KHolodilin/spring-transactional-outbox-kafka: Production-ready Java implementation of the Transactional Outbox Pattern for Spring Boot 4, Kafka and PostgreSQL, with idempotency, batch publish...
Production-ready Java implementation of the Transactional Outbox Pattern for Spring Boot 4, Kafka and PostgreSQL, with idempotency, batch publishing, recovery and low database polling. - KHolodilin...
000
Igfasouza @igfasouza.bsky.social · 22/09/2026
Kafka Isn’t What You Think It Is 😵 #apachekafka medium.com/@charansubra...
medium.com
Kafka Isn’t What You Think It Is 😵
Ask ten people what Kafka is, and you’ll get ten confident, slightly wrong answers. The business stakeholder thinks it’s a magic pipe that…
000
Igfasouza @igfasouza.bsky.social · 21/09/2026
Kafka has officially integrated AI. #apachekatka freedium-mirror.cfd/https://medi...
freedium-mirror.cfd
Kafka has officially integrated AI. - Freedium
Kafka Has Officially Entered the AI Era
000
Igfasouza @igfasouza.bsky.social · 20/09/2026
Apache Kafka: Zero to Production #apachekafka youtube.com/playlist?lis...
youtube.com
Apache Kafka: Zero to Production - YouTube
Apache Kafka runs the backend of almost every large-scale app you use daily — but most tutorials jump straight into commands without ever explaining why Kafk...
000
Igfasouza @igfasouza.bsky.social · 19/09/2026
Kafka Without ZooKeeper: Inside the KRaft Migration Every Engineer Should Understand #apachekafka medium.com/@dimples2609...
medium.com
Kafka Without ZooKeeper: Inside the KRaft Migration Every Engineer Should Understand
Friend link
000
Igfasouza @igfasouza.bsky.social · 18/09/2026
How to configure Kafka Authentication in Strimzi #apachekafka axual.com/blog/how-to-...
axual.com
Learn how to configure Kafka Authentication in Strimzi | Axual Blog
Configure Kafka authentication in Strimzi using mTLS, custom principal builders, tenant-specific listeners, OAuth, and secure multi-tenant isolation.
000
Igfasouza @igfasouza.bsky.social · 17/09/2026
JMeter + Pepper-Box Plugin for Kafka Performance Testing: Choose or Not to Choose? #apachekafka www.itechguides.com/jmeter-peppe...
itechguides.com
000
Igfasouza @igfasouza.bsky.social · 16/09/2026
Why Kafka Isn't Actually Losing Your Order #apachekafka prakashraj.me/blog/why-kaf...
prakashraj.me
Why Kafka Isn't Actually Losing Your Order | Prakash Raj Blog
Kafka only promises order inside a single partition. Here's how that works in practice, why things still arrive jumbled anyway, and how to keep order intact without getting burned in production.
000
Igfasouza @igfasouza.bsky.social · 15/09/2026
kafka-explained An interactive Node.js + React app that shows how Apache Kafka works, using a live simulation you can steer: #apachekafka github.com/ChauhanPares...
github.com
GitHub - ChauhanParesh/kafka-explained: An interactive Node.js + React simulator that shows how Kafka topics, partitions, offsets, brokers, producers, consumer groups and hot partitions actually work.
An interactive Node.js + React simulator that shows how Kafka topics, partitions, offsets, brokers, producers, consumer groups and hot partitions actually work. - ChauhanParesh/kafka-explained
000
Igfasouza @igfasouza.bsky.social · 14/09/2026
Kafka — Beyond Exactly Once #apachekafka medium.com/@kapoorabhin...
medium.com
Kafka — Beyond Exactly Once
Part 2 of the Creating Reliable Business Systems series
000
Igfasouza @igfasouza.bsky.social · 13/09/2026
Kafka Consumer Lag Is Not an Autoscaling Strategy #apachekafka medium.com/@sauravrana_...
medium.com
Kafka Consumer Lag Is Not an Autoscaling Strategy
Kafka Consumer Lag Is Not an Autoscaling Strategy While building my Distributed Job Scheduler, I came across an interesting question: How do you handle consumer lag for an ExecutorService, and what …
000
Igfasouza @igfasouza.bsky.social · 11/09/2026
Diskless Kafka: What Happens When Brokers Stop Owning the Data? #apachekafka softwaremill.com/diskless-kaf...
softwaremill.com
Diskless Kafka: Object Storage, KIP-1150, and Kafka’s Future
Diskless Kafka moves the active log to object storage. See how KIP1150, WarpStream, AutoMQ and Redpanda reshape Kafka’s write path, metadata and cost.
001
Igfasouza @igfasouza.bsky.social · 10/09/2026
Kafka Query Engine: Lightweight Text Search for Kafka #apachekafka medium.com/@prakashpsgc...
medium.com
Kafka Query Engine: Lightweight Text Search for Kafka
Kafka is great at storing and streaming events, but one thing it doesn’t provide out of the box is search.
000
Igfasouza @igfasouza.bsky.social · 09/09/2026
Distributed Tracing Across Kafka and gRPC With OpenTelemetry Context Propagation #apachekafka martinuke0.github.io/posts/2026-0...
martinuke0.github.io
Distributed Tracing Across Kafka and gRPC With OpenTelemetry Context Propagation
How to propagate OpenTelemetry context across Kafka and gRPC in production: W3C Trace Context, headers, and the gotchas that break traces.
001
Reposted by Igfasouza
Foojay, the Friends Of OpenJDK @foojay.io · 07/09/2026
New on Foojay: @igfasouza.bsky.social shows how to connect Raspberry Pi Pico to Quarkus using the Firmata protocol. Learn to control hardware with Java in this practical guide. foojay.io/today/bringi... #Java #Quarkus #RaspberryPi #IoT
foojay.io
Bringing Raspberry Pico to Quarkus with Firmata - foojay
Java applications are usually associated with APIs, databases, messaging systems, and cloud infrastructure. But Java can also interact with the physical - by Igor De Souza
011
Igfasouza @igfasouza.bsky.social · 08/09/2026
Apache Kafka vs Apache Iggy: Same Log, Different Engine #apachekafka softwaremill.com/apache-kafka...
softwaremill.com
Apache Kafka vs Apache Iggy: A Technical Comparison
Compare Kafka and Iggy architectures, durability guarantees, and benchmarks to understand which streaming engine fits your workload before you decide.
000
Igfasouza @igfasouza.bsky.social · 07/09/2026
Break Kafka before production does #apachekafka getkhaos.dev
getkhaos.dev
Khaos: Kafka Load Testing & Chaos Engineering
Khaos is an open-source Kafka traffic generator, load-testing tool, and chaos engineering CLI. Generate realistic Kafka traffic and reproduce failure scenarios like consumer lag, hot partitions, rebal...
000
Igfasouza @igfasouza.bsky.social · 06/09/2026
RabbitMQ vs Kafka simulator #apachekafka ulearn-it.vercel.app/topics/rabbi...
ulearn-it.vercel.app
RabbitMQ vs Kafka
Interactive system architecture demonstrations — load balancers, rate limiters, message queues, and RabbitMQ vs Kafka. Watch each mechanism run live, tweak it, break it on purpose.
000
Igfasouza @igfasouza.bsky.social · 05/09/2026
Scaling a Kafka Consumer From 4K to 25K Events per Second While Preserving Ordering #apachekafka hackernoon.com/scaling-a-ka...
hackernoon.com
Scaling a Kafka Consumer From 4K to 25K Events per Second While Preserving Ordering | HackerNoon
A production redesign using batching, safe offset commits, and record-level fallback raised Kafka consumer throughput from 4K to 25K without adding partitions.
000
Igfasouza @igfasouza.bsky.social · 04/09/2026
Deploying Apache Kafka Streams next to your data #apachekafka aiven.io/blog/apache-...
aiven.io
Deploying Apache Kafka® Streams next to your data
Get a complete, working Apache Kafka® Streams example for Avro messages, and deploy it as an Aiven App linked to your Karapace schema registry.
000
Igfasouza @igfasouza.bsky.social · 03/09/2026
Log-first or Table-first? Apache Kafka, Fluss, and Streaming Tables #apachekafka softwaremill.com/log-first-or...
softwaremill.com
Log-first or Table-first? Apache Kafka, Fluss, & Streaming Tables
Explore the architectural differences between logfirst systems like Apache Kafka and table-first engines like Apache Fluss for streaming data pipelines.
000
Igfasouza @igfasouza.bsky.social · 02/09/2026
Can a #Java upgrade make your Kafka Streams applications faster without touching your code? #apachekafka Check out how #Java26's G1 GC improvements deliver lower latency, higher throughput & better performance in containers. javapro.io/2026/09/01/j...
javapro.io
Java 26 GC: Making Data Streaming Faster and More Reliable in Containers - JAVAPRO International
Imagine running a Java application that processes massive amounts of data in real time. Example financial transactions flowing…
000
Igfasouza @igfasouza.bsky.social · 01/09/2026
Now Streaming Is Going To Become Mainstream #apachekafka ralphmdebusmann.substack.com/p/now-stream...
ralphmdebusmann.substack.com
Now Streaming Is Going To Become Mainstream
Kafi Streams 1.0.0
000
Igfasouza @igfasouza.bsky.social · 31/08/2026
Awesome Event-Driven Architecture #apachekafka A curated list of awesome articles and resources to learn about event-driven architecture. github.com/lutzh/awesom...
000
Igfasouza @igfasouza.bsky.social · 30/08/2026
Kafka protocol practical guide #apachekafka ivanyu.me/blog/2024/09...
000
Igfasouza @igfasouza.bsky.social · 29/08/2026
Course | Apache Kafka Fundamentals. #apachekafka youtube.com/playlist?lis...
youtube.com
Course | Apache Kafka Fundamentals - YouTube
Apache Kafka Fundamentals Course with Tim Berglund
000
Igfasouza @igfasouza.bsky.social · 28/08/2026
Kafka interview questions ... #apachekafka youtube.com/@satyverse08...
000
Igfasouza @igfasouza.bsky.social · 27/08/2026
Kafka- Zero to Hero #apachekafka youtube.com/playlist?lis...
youtube.com
Kafka- Zero to Hero - YouTube
You want to learn Kafka? This is the course for you! If you are a Data Architect or a Microservices Developer, you can't miss out on what Kafka has to offer....
000
Igfasouza @igfasouza.bsky.social · 26/08/2026
Apache Fluss: The Foundation of the Unified Streaming Lakehouse www.ververica.com/asset-librar...
ververica.com
Apache Fluss®: The Foundation of the Unified Streaming Lakehouse
Apache Fluss is an open-source storage layer designed to unify the infrastructure for real-time analytics. This paper traces the architecture that makes that possible
000
Igfasouza @igfasouza.bsky.social · 26/08/2026
I thought Kafka kept the order I published #apachekafka www.onkarsawarna.dev/blog/i-thoug...
onkarsawarna.dev
I thought Kafka kept the order I published
A topic is a named log sliced into partitions. The API writes the row, then publishes. Order lives on a partition, not on the topic.
000
Igfasouza @igfasouza.bsky.social · 25/08/2026
Apache Kafka: The Complete Guide from Zero to Production #apachekafka pub.towardsai.net/apache-kafka...
pub.towardsai.net
Apache Kafka: The Complete Guide from Zero to Production
Everything you need to understand Apache Kafka, from your first event to building scalable, fault-tolerant streaming systems.
000
Igfasouza @igfasouza.bsky.social · 24/08/2026
Kafka Consumer Lag Is the Wrong Metric — Measure This Instead #apachekafka www.linkedin.com/pulse/kafka-...
linkedin.com
Kafka Consumer Lag Is the Wrong Metric — Measure This Instead
Kafka offset-based consumer lag is misleading when message sizes vary. How to instrument time-based lag metrics for an accurate view of consumer group health.
000
Igfasouza @igfasouza.bsky.social · 23/08/2026
Distributed Systems lecture series m.youtube.com/playlist?lis...
m.youtube.com
Distributed Systems lecture series - YouTube
Accompanying lecture notes: https://www.cl.cam.ac.uk/teaching/2122/ConcDisSys/dist-sys-notes.pdf These videos form an 8-lecture series on distributed systems...
000
Igfasouza @igfasouza.bsky.social · 23/08/2026
Human Error: The Real Reason Kafka's Durability Isn't a Backup Strategy #apachekafka cymo.eu/insights/hum...
cymo.eu
Human Error: The Real Reason Kafka's Durability Isn't a Backup Strategy | Cymo
Kafka is robust against hardware failure, but not human error. Learn why DIY scripts and cluster linking fail and what a true backup solution requires.
011
Igfasouza @igfasouza.bsky.social · 22/08/2026
Inside a Kafka Order Workflow: Outbox, Retries, DLQs and Idempotency #apachekafka malkomich.github.io/2026-08-07-i...
malkomich.github.io
Inside a Kafka Order Workflow: Outbox, Retries, DLQs and Idempotency | malkomich
Build an event-driven order workflow using Java, Spring Boot, and Kafka. Learn Outbox Pattern, retry topics, DLQs, idempotency, consumer groups, and Exactly ...
000
Igfasouza @igfasouza.bsky.social · 21/08/2026
I’ve been building a small Raspberry Pi lab to get a deeper, hands-on understanding of Apache Kafka Tiered Storage. 🚀 #apachekafka github.com/igfasouza/pi...
000
Igfasouza @igfasouza.bsky.social · 20/08/2026
Gateway API and TLSRoute support in Strimzi #apachekafka strimzi.io/blog/2026/07...
strimzi.io
Gateway API and TLSRoute support in Strimzi
Strimzi provides a way to run an Apache Kafka cluster on Kubernetes in various deployment configurations.
000
Igfasouza @igfasouza.bsky.social · 19/08/2026
Kafka Cost Optimization: Cut Your Bill Without Cutting Reliability #apachekafka learnkafka.com/blog-kafka-c...
learnkafka.com
Kafka Cost Optimization: Cut Your Bill Without Cutting Reliability | LearnKafka
Reduce Kafka infrastructure costs with right-sizing, tiered storage, compression, cross-AZ traffic reduction, topic cleanup, and MSK/Confluent Cloud optimizations.
010
Igfasouza @igfasouza.bsky.social · 18/08/2026
One Weird Trick to Make Rapid Storage 40x Faster #apachekafka www.warpstream.com/blog/one-wei...
warpstream.com
One Weird Trick to Make Rapid Storage 40x Faster
GCP Rapid Buckets should have made WarpStream faster on GCP, but they didn't, until we found the trick. Here's how we cut P99 file write latency by 3x with a few small, surgical changes.
000
Igfasouza @igfasouza.bsky.social · 17/08/2026
Building Real-Time Streaming Pipelines with Apache Kafka and Apache Flink #apachekafka intellifysolutions.com/blog/buildin...
intellifysolutions.com
Building Real-Time Streaming Pipelines with Apache Kafka and Apache Flink - Intellify Solutions
Explore the future of database architecture in "Rethinking OLTP." Discover how modern transaction processing scales to meet today's real-time demands.
000
Igfasouza @igfasouza.bsky.social · 16/08/2026
Kafka and AI walk into a bar... #apachekafka softwaremill.com/managing-kaf...
softwaremill.com
Managing Kafka with AI and MCP: A Hands-On mcp-confluent Guide
Discover how to integrate Kafka, AI, and the Model Context Protocol (MCP). Learn how to set up mcpconfluent with Claude Code, manage clusters, and overcome CLI hurdles.
000