IBM Research and Red Hat deployed a 753B open model on H100 GPUs, serving thousands of concurrent coding agents at 5-10x lower cost than commercial APIs.

Optimize inference for large language models (LLMs) with llm-d, reducing redundant computation and lowering AI workload costs.

Scale agentic AI with llm-d to achieve full infrastructure sovereignty across diverse hardware.

The llm-d router's default configuration is built on token-aware routing: keep each request on the cache-warm endpoint until a calibrated token-load limit is exceeded, then route by load alone,…

Optimize inference for large language models (LLMs) with llm-d, reducing redundant computation and lowering AI workload costs.

YouTube video by Red Hat

llm-d v0.7 shifts focus from proving capabilities to making them deployable, with changes across deployment tooling, hardware support, documentation, and continuous integration.

Open Source Distributed AI Inference (llm-d/vLLM) Meetup Boston/Cambridge Hosted by Google Cloud, Red Hat AI, and the llm-d Community Date: Thursday, May 28th…

Open Source Distributed AI Inference (llm-d/vLLM) Meetup Boston/Cambridge Hosted by Google Cloud, Red Hat AI, and the llm-d Community Date: Thursday, May 28th…

In this video, Sally from SIG Observability demonstrates how to achieve comprehensive observability for Large Language Model (LLM) deployments using llm-d distributed tracing and vLLM NIXL metrics. …

How migrating from a simple vLLM deployment to a robust MLOps platform utilizing KServe, llm-d's intelligent routing, and vLLM solved significant scaling and operational challenges in LLM deployment…

How migrating from a simple vLLM deployment to a robust MLOps platform utilizing KServe, llm-d's intelligent routing, and vLLM solved significant scaling and operational challenges in LLM deployment t...

YouTube video by CNCF [Cloud Native Computing Foundation]

We are thrilled to announce that llm-d has officially been accepted as a Cloud Native Computing Foundation (CNCF) Sandbox project! As generative AI transitions from research labs to production…

We are thrilled to announce that llm-d has officially been accepted as a Cloud Native Computing Foundation (CNCF) Sandbox project! As generative AI transitions from research labs to production…

Deep technical sessions. Live demos. Real conversations. If you're deploying, or scaling LLM inference, this is the room to be in. Join Red Hat AI, IBM,…

YouTube video by llm-d Project

llm-d Distributed Inference Meetup NYC Hosted by Red Hat AI, IBM Research, and AMD, this event takes place on March 11, 2026 in New York City. What to…

llm-d Distributed Inference Meetup NYC Hosted by Red Hat AI, IBM Research, and AMD, this event takes place on March 11, 2026 in New York City. What to…

llm-d Distributed Inference Meetup NYC Hosted by Red Hat AI, IBM Research, and AMD, this event takes place on March 11, 2026 in New York City. What to…

llm-d Distributed Inference Meetup NYC Hosted by Red Hat AI, IBM Research, and AMD, this event takes place on March 11, 2026 in New York City. What to…

YouTube video by llm-d Project

llm-d v0.5 introduces hierarchical KV-cache offloading, LoRA-aware scheduling, UCCL networking, and scale-to-zero autoscaling for sustained inference performance at scale.

Announcing the llm-d 0.5 release

This session introduces llm-d, a distributed open-source framework for LLM inference

In the llm-d open-source project, we believe a supported guide is only as good as the data backing it. In this community demo, the SIG-benchmarking team showcases the benchmarking suite that brings…

Dynamo Day