iiinova — where your digital self keeps evolving

Yuan Tang @terrytangyuan.xyz · 27/08/2026

Excited to share that our accompanying paper for the Inference Perf project (under Kubernetes SIGs) has been published in the Journal of Open Source Software! Paper: joss.theoj.org/papers/10.21... Repo: github.com/kubernetes-s...

joss.theoj.org

Inference Perf: A Benchmarking Tool for GenAI Inference

Chandrasekar et al., (2026). Inference Perf: A Benchmarking Tool for GenAI Inference. Journal of Open Source Software, 11(124), 10314, https://doi.org/10.21105/joss.10314

130

Yuan Tang @terrytangyuan.xyz · 24/08/2026

📢 𝗧𝗵𝗲 𝗦𝘁𝗮𝘁𝗲 𝗼𝗳 𝗠𝗼𝗱𝗲𝗹 𝗦𝗲𝗿𝘃𝗶𝗻𝗴 𝗖𝗼𝗺𝗺𝘂𝗻𝗶𝘁𝗶𝗲𝘀: 𝗔𝘂𝗴𝘂𝘀𝘁 𝗘𝗱𝗶𝘁𝗶𝗼𝗻 𝗶𝘀 𝗼𝘂𝘁!

110

Yuan Tang @terrytangyuan.xyz · 21/08/2026

I'm thrilled to share that I've been invited to become Omnigent's first external maintainer! It's remarkable that the project is only two months old and I feel incredibly fortunate to contribute at such an early stage and collaborate with the Databricks team and broader community.

140

Yuan Tang @terrytangyuan.xyz · 12/08/2026

I’m speaking at #KubeCon + #CloudNativeCon North America, November 9-12 in Salt Lake City, Utah! Check out our session to learn more about the work we are doing at the CNCF TAG Workloads Foundation and meet TAG leads in person! events.linuxfoundation.org/kubecon-clou...

050

Yuan Tang @terrytangyuan.xyz · 19/07/2026

Attending @scipyconf.bsky.social this year was a refreshing experience in more ways than one. #SciPy2026 Thanks to organizers, speakers, volunteers, and everyone I had the chance to meet. Looking forward to the next one! Photos with @hadley.nz @jameslamb.bsky.social @tracykteal.bsky.social

160

Reposted by Yuan Tang

SciPy Conference 2026 @scipyconf.bsky.social · 09/07/2026

🌱 Talk Spotlight: @terrytangyuan.xyz shares what years on the open source front lines have taught him about sustainable maintenance. Hard-won lessons for the whole community. #SciPy2026 🔗 scipy2026.scipy.org

031

Yuan Tang @terrytangyuan.xyz · 09/07/2026

Excited to be speaking at #SciPy2026 in Minneapolis next week! If you're attending, I'd love to connect. Come say hi after the talk or find me during the conference. Let's chat about open source, AI infrastructure, or whatever's on your mind. Full schedule: www.scipy2026.scipy.org/schedule

12190

Yuan Tang @terrytangyuan.xyz · 06/07/2026

Most teams over-engineer their inference stack from day one. They disaggregate before measuring. Add speculative decoding before concurrency stabilizes.

491

Yuan Tang @terrytangyuan.xyz · 30/06/2026

Part 2 of our 𝗗𝗶𝘀𝘁𝗿𝗶𝗯𝘂𝘁𝗲𝗱 𝗔𝗜 𝗜𝗻𝗳𝗲𝗿𝗲𝗻𝗰𝗲 series is now live on Red Hat Developer: 𝘖𝘱𝘵𝘪𝘮𝘪𝘻𝘪𝘯𝘨 𝘋𝘪𝘴𝘵𝘳𝘪𝘣𝘶𝘵𝘦𝘥 𝘈𝘐 𝘐𝘯𝘧𝘦𝘳𝘦𝘯𝘤𝘦: 𝘈𝘥𝘷𝘢𝘯𝘤𝘦𝘥 𝘋𝘦𝘱𝘭𝘰𝘺𝘮𝘦𝘯𝘵 𝘗𝘢𝘵𝘵𝘦𝘳𝘯𝘴. In Part 1, we covered prefill/decode phases and the 5D parallelism framework.

492

Yuan Tang @terrytangyuan.xyz · 26/06/2026

Excited to share Part 1 of our blog series on Red Hat Developer: 𝘋𝘦𝘴𝘪𝘨𝘯𝘪𝘯𝘨 𝘋𝘪𝘴𝘵𝘳𝘪𝘣𝘶𝘵𝘦𝘥 𝘈𝘐 𝘐𝘯𝘧𝘦𝘳𝘦𝘯𝘤𝘦: 𝘊𝘰𝘳𝘦 𝘊𝘰𝘯𝘤𝘦𝘱𝘵𝘴 𝘢𝘯𝘥 𝘚𝘤𝘢𝘭𝘪𝘯𝘨 𝘋𝘪𝘮𝘦𝘯𝘴𝘪𝘰𝘯𝘴. LLM inference is two workloads pretending to be one. The prefill phase is compute-bound, processing entire prompts in parallel to populate the KV cache.

262

Yuan Tang @terrytangyuan.xyz · 24/06/2026

📢 𝗧𝗵𝗲 𝗦𝘁𝗮𝘁𝗲 𝗼𝗳 𝗠𝗼𝗱𝗲𝗹 𝗦𝗲𝗿𝘃𝗶𝗻𝗴 𝗖𝗼𝗺𝗺𝘂𝗻𝗶𝘁𝗶𝗲𝘀: 𝗝𝘂𝗻𝗲 𝗘𝗱𝗶𝘁𝗶𝗼𝗻 𝗶𝘀 𝗼𝘂𝘁! We recently launched our newsletter publicly after sharing it internally at Red Hat AI for over a year.

221

Yuan Tang @terrytangyuan.xyz · 19/06/2026

I’ve had a great experience using Omnigent and have been collaborating with the Databricks team to land a few features and fixes, including: - Kubernetes & OpenShift deployment - NVIDIA OpenShell sandbox integration - Claude on Vertex AI auto-detection - Podman support

210

Yuan Tang @terrytangyuan.xyz · 12/06/2026

We will be hosting Cloud Native AI + Inference Day again as a co-located event with KubeCon North America! @cncf.io Looking forward to your proposals (by June 21) and sponsorship (by September 21)! events.linuxfoundation.org/kubecon-clou...

events.linuxfoundation.org

Cloud Native AI + Inference Day | LF Events

As we step into the era of rapid AI advancements, organizations are grappling with an unprecedented array of challenges. The rise of Large Language Models (LLMs), the development of Graph RAGs…

140

Yuan Tang @terrytangyuan.xyz · 31/05/2026

I recently received a LinkedIn Premium Business subscription that I'll be able to use indefinitely. More than the subscription itself, I'm grateful for the connections I've made through this platform over the years.

120

Yuan Tang @terrytangyuan.xyz · 16/05/2026

The job market is tough right now, especially for new graduates and talented people impacted by layoffs. I have several free 2-month LinkedIn Premium invites available, and I’d love to share them with people who could benefit from the extra visibility, networking tools, and learning resources.

241

Yuan Tang @terrytangyuan.xyz · 22/04/2026

Excited to share our latest blog post on how we're solving real-world LLM inference challenges at production scale, a collaboration between Red Hat AI and Tesla engineering teams.

162

Yuan Tang @terrytangyuan.xyz · 14/04/2026

📢 𝗧𝗵𝗲 𝗦𝘁𝗮𝘁𝗲 𝗼𝗳 𝗠𝗼𝗱𝗲𝗹 𝗦𝗲𝗿𝘃𝗶𝗻𝗴 𝗖𝗼𝗺𝗺𝘂𝗻𝗶𝘁𝗶𝗲𝘀: 𝗔𝗽𝗿𝗶𝗹 𝗘𝗱𝗶𝘁𝗶𝗼𝗻 𝗶𝘀 𝗼𝘂𝘁! Our goal with this newsletter is to give a clear, community-driven view of what’s happening across the model serving ecosystem, including updates from projects like vLLM, KServe, @llm-d.ai, @kubernetes.io, Llama Stack, and more.

132

Yuan Tang @terrytangyuan.xyz · 31/03/2026

Another fruitful KubeCon — this time in Amsterdam! I had a wonderful time presenting across 𝗳𝗼𝘂𝗿 𝘀𝗲𝘀𝘀𝗶𝗼𝗻𝘀: - Opening remarks for Cloud Native AI Day - KServe project lightning session - @kubernetes.io AI Conformance Program updates - @cncf.io TAG Workloads Foundation updates

360

Yuan Tang @terrytangyuan.xyz · 30/03/2026

🎤 𝗦𝗹𝗶𝗱𝗲𝘀 𝗮𝗿𝗲 𝗮𝘃𝗮𝗶𝗹𝗮𝗯𝗹𝗲! At #KubeCon Europe 2026 in Amsterdam, I had the pleasure of joining Stephen Rust, Kante Yin, and Marlow Warnicke for our session: 𝘌𝘹𝘱𝘭𝘰𝘳𝘦 𝘛𝘈𝘎 𝘞𝘰𝘳𝘬𝘭𝘰𝘢𝘥𝘴 𝘍𝘰𝘶𝘯𝘥𝘢𝘵𝘪𝘰𝘯: 𝘈𝘥𝘷𝘢𝘯𝘤𝘪𝘯𝘨 𝘊𝘭𝘰𝘶𝘥 𝘕𝘢𝘵𝘪𝘷𝘦 𝘌𝘹𝘦𝘤𝘶𝘵𝘪𝘰𝘯 𝘍𝘳𝘰𝘮 𝘊𝘰𝘳𝘦 𝘙𝘶𝘯𝘵𝘪𝘮𝘦 𝘛𝘰 𝘈𝘱𝘱𝘭𝘪𝘤𝘢𝘵𝘪𝘰𝘯𝘴

130

Yuan Tang @terrytangyuan.xyz · 28/03/2026

I was fortunate to get my hands on 𝘎𝘦𝘯𝘦𝘳𝘢𝘵𝘪𝘷𝘦 𝘈𝘐 𝘰𝘯 𝘒𝘶𝘣𝘦𝘳𝘯𝘦𝘵𝘦𝘴 before it’s even available for everyone to order online! It was also great to meet the co-authors and my wonderful colleagues - Daniele Zonca and @ro14nd.bsky.social at KubeCon Europe. And look whose quote showed up on the first page!

170

Reposted by Yuan Tang

Puja Abbassi @puja.dev · 26/03/2026

Maintainers track on #Kubernetes #AI conformance now transitioned from WG to sub-project of SIG architecture by Janet Kuo, @terrytangyuan.xyz and @mfahlandt.bsky.social

052

Reposted by Yuan Tang

Puja Abbassi @puja.dev · 23/03/2026

@terrytangyuan.xyz and @rajaskakodkar.bsky.social kicking off Cloud Native AI + Kubeflow #KubeCon colo

292

Yuan Tang @terrytangyuan.xyz · 19/03/2026

𝗞𝗦𝗲𝗿𝘃𝗲 𝘃𝟬.𝟭𝟳 𝗶𝘀 𝗹𝗶𝘃𝗲! 🚀 We are thrilled to announce KServe's most significant update yet. We’ve overhauled the architecture to move beyond traditional model serving.

120

Yuan Tang @terrytangyuan.xyz · 14/03/2026

Just noticed that 4 of my US patent applications have officially gone public! It’s one thing to have an idea, but it’s another thing entirely to see it documented and accessible on the USPTO website. Check them out here on my Google Scholars page! scholar.google.com/citations?hl...

140

Reposted by Yuan Tang

Cloud Native Computing Foundation (CNCF) @cncf.io · 11/03/2026

Managing high-volume LLM traffic at scale requires a focus on GPU utilization and Kubernetes-native architecture. @terrytangyuan.xyz (Yuan Tang) and Ran Pollak explore how KServe and @llm-d.ai work together to provide a production-ready inference platform. #CloudNative #AI #Kubernetes #CNCF

011

Yuan Tang @terrytangyuan.xyz · 09/03/2026

📢 𝗧𝗵𝗲 𝗦𝘁𝗮𝘁𝗲 𝗼𝗳 𝗠𝗼𝗱𝗲𝗹 𝗦𝗲𝗿𝘃𝗶𝗻𝗴 𝗖𝗼𝗺𝗺𝘂𝗻𝗶𝘁𝗶𝗲𝘀: 𝗠𝗮𝗿𝗰𝗵 𝗘𝗱𝗶𝘁𝗶𝗼𝗻 𝗶𝘀 𝗼𝘂𝘁! We launched our newsletter publicly last year to share our contributions to upstream communities from our Red Hat AI teams. We’ve gained over 𝟭𝟯𝟬𝟬 𝘀𝘂𝗯𝘀𝗰𝗿𝗶𝗯𝗲𝗿𝘀!

222

Yuan Tang @terrytangyuan.xyz · 06/03/2026

🚀 𝗕𝘂𝗶𝗹𝗱𝗶𝗻𝗴 𝗔𝗜 𝗶𝗻𝗳𝗲𝗿𝗲𝗻𝗰𝗲 𝗽𝗹𝗮𝘁𝗳𝗼𝗿𝗺𝘀? 𝗦𝘁𝗼𝗽 𝗹𝗲𝗮𝘃𝗶𝗻𝗴 𝗽𝗲𝗿𝗳𝗼𝗿𝗺𝗮𝗻𝗰𝗲 𝗼𝗻 𝘁𝗵𝗲 𝘁𝗮𝗯𝗹𝗲. The complexity of managing high-volume LLM traffic, GPU utilization, and cost control requires more than just model deployment.

120

Reposted by Yuan Tang

Cloud Native Computing Foundation (CNCF) @cncf.io · 23/02/2026

A milestone for the Kubernetes inference ecosystem, shared by @terrytangyuan.xyz (Yuan Tang): WG Serving has completed its mission and is transitioning ongoing work forward. Thank you to everyone who contributed, and to the teams carrying the work forward! #Kubernetes #CloudNative #AI

133

Yuan Tang @terrytangyuan.xyz · 14/02/2026

Looking forward to be frequently checking out the KubeCon Feed next month! #KubeCon #CloudNativeCon Could anyone drop me the link to the list of social events in Amsterdam? bsky.app/profile/did:...

140

Yuan Tang @terrytangyuan.xyz · 13/02/2026

We'd like to announce that @kubernetes.io WG Serving has succeeded and will be disbanded! Thank you everyone who have participated and contributed to the discussions and initiatives! More details: groups.google.com/a/kubernetes...

groups.google.com

[Announcement] WG Serving Has Succeeded and Will Be Disbanded

142

Reposted by Yuan Tang

Cloud Native Computing Foundation (CNCF) @cncf.io · 10/02/2026

If you’re interested in cloud native AI, KServe, or Kubernetes AI conformance, keep an eye out for Yuan Tang at KubeCon + CloudNativeCon EU—lots of ecosystem-wide work being shared this year. #CNCF #CloudNative

021

Yuan Tang @terrytangyuan.xyz · 09/02/2026

📢 𝗧𝗵𝗲 𝗦𝘁𝗮𝘁𝗲 𝗼𝗳 𝗠𝗼𝗱𝗲𝗹 𝗦𝗲𝗿𝘃𝗶𝗻𝗴 𝗖𝗼𝗺𝗺𝘂𝗻𝗶𝘁𝗶𝗲𝘀: 𝗙𝗲𝗯𝗿𝘂𝗮𝗿𝘆 𝗘𝗱𝗶𝘁𝗶𝗼𝗻 𝗶𝘀 𝗼𝘂𝘁! We launched our newsletter publicly last year to share our contributions to upstream communities from our Red Hat AI teams. We’ve gained over 𝟭𝟮𝟬𝟬 𝘀𝘂𝗯𝘀𝗰𝗿𝗶𝗯𝗲𝗿𝘀!

111

Yuan Tang @terrytangyuan.xyz · 15/01/2026

Excited to share that I'll be speaking at #KubeCon Europe in Amsterdam! You can find me in the following sessions: 1. Cloud Native AI + Kubeflow Day: Welcome + Opening Remarks: sched.co/2DZN3 2. Project Lightning Talk: Evolving KServe: sched.co/2EFyW

172

Yuan Tang @terrytangyuan.xyz · 13/01/2026

📢 𝗧𝗵𝗲 𝗦𝘁𝗮𝘁𝗲 𝗼𝗳 𝗠𝗼𝗱𝗲𝗹 𝗦𝗲𝗿𝘃𝗶𝗻𝗴 𝗖𝗼𝗺𝗺𝘂𝗻𝗶𝘁𝗶𝗲𝘀: 𝗝𝗮𝗻𝘂𝗮𝗿𝘆 𝗘𝗱𝗶𝘁𝗶𝗼𝗻 𝗶𝘀 𝗼𝘂𝘁! We launched our newsletter publicly last year to share our contributions to upstream communities from our Red Hat AI teams. We’ve gained over 𝟭𝟮𝟬𝟬 𝘀𝘂𝗯𝘀𝗰𝗿𝗶𝗯𝗲𝗿𝘀!

111

Yuan Tang @terrytangyuan.xyz · 18/12/2025

🚀 𝗠𝗼𝗿𝗲 𝗽𝗼𝘀𝗶𝘁𝗶𝗼𝗻𝘀 𝗮𝗿𝗲 𝗼𝗽𝗲𝗻 𝗮𝘁 𝗥𝗲𝗱 𝗛𝗮𝘁 𝗔𝗜! Our 𝗜𝗻𝗳𝗲𝗿𝗲𝗻𝗰𝗲 𝗘𝗻𝗴𝗶𝗻𝗲𝗲𝗿𝗶𝗻𝗴 team continues to grow with ML engineer, researcher, and developer advocate positions!

110

Yuan Tang @terrytangyuan.xyz · 09/12/2025

🎉 𝗠𝗶𝗹𝗲𝘀𝘁𝗼𝗻𝗲 𝘂𝗻𝗹𝗼𝗰𝗸𝗲𝗱! The 𝘐𝘯𝘧𝘦𝘳𝘦𝘯𝘤𝘦𝘖𝘱𝘴: 𝘚𝘵𝘢𝘵𝘦 𝘰𝘧 𝘵𝘩𝘦 𝘔𝘰𝘥𝘦𝘭 𝘚𝘦𝘳𝘷𝘪𝘯𝘨 𝘊𝘰𝘮𝘮𝘶𝘯𝘪𝘵𝘪𝘦𝘴 newsletter from Red Hat AI just reached 𝟭,𝟬𝟬𝟬 𝘀𝘂𝗯𝘀𝗰𝗿𝗶𝗯𝗲𝗿𝘀 in only 5 months!

130

Yuan Tang @terrytangyuan.xyz · 03/12/2025

🚀 𝗪𝗲’𝗿𝗲 𝗛𝗶𝗿𝗶𝗻𝗴 𝗦𝗼𝗳𝘁𝘄𝗮𝗿𝗲 𝗘𝗻𝗴𝗶𝗻𝗲𝗲𝗿𝘀 𝗶𝗻 𝟮𝟬𝟮𝟲! Our team at 𝗥𝗲𝗱 𝗛𝗮𝘁 𝗔𝗜 continues to grow, and we’re looking for passionate engineers (at multiple levels) to help us push the boundaries of 𝗔𝗜 𝗶𝗻𝗳𝗿𝗮𝘀𝘁𝗿𝘂𝗰𝘁𝘂𝗿𝗲.

241