I coauthored an Anyscale technical deep dive on integrating NVIDIA Dynamo’s KV-aware selection machinery into Ray Serve LLM. The collaboration is a useful example of a systems component becoming more valuable when it can be embedded outside the system where it started. ...
I worked with Gcore’s AI team on the Dynamo side of a global inference-routing problem: when the same model is served in many data centers, how do you decide where each request should go without throwing away useful KV cache state? ...
I worked on DynoSim and coauthored the NVIDIA technical blog introducing it. DynoSim is a workload-driven discrete-event simulation of the NVIDIA Dynamo serving stack: a digital twin that combines engine timing, scheduler behavior, routing, planning, KV-cache effects, and workload traces on one virtual timeline. ...
I worked on the Flash Indexer, a high-throughput global KV-cache indexer for NVIDIA Dynamo. The problem is to track which inference workers hold which KV blocks, and then answer routing queries fast enough that the indexer itself does not become the bottleneck. ...
Dynamo is NVIDIA’s open-source datacenter-scale distributed inference serving framework for generative AI and reasoning models. Built in Rust for performance and Python for extensibility, it supports disaggregated prefill and decode, dynamic GPU scheduling, and LLM-aware request routing across multi-node multi-GPU topologies. It supports backends including TensorRT-LLM, vLLM, and SGLang. ...