Analysis
Packing small jobs onto one GPU has a plain obstacle: an allocation has to mean something when the neighbour gets hungry. NVIDIA's October 27 technical post says CUDA 13.4 supports a memory allowance that is enforced, either as a fixed amount or as a guaranteed request with room to grow when capacity is free.
Installing CUDA does not finish the job, though. The post splits the work across several parts. The scheduler picks devices and records allocations. The open-source GPU Fractioning Operator supervises MPS on each node, applies workload limits and measures every fraction. NVIDIA names KAI Scheduler 0.18.0, in its NvFractions mode, as the first integration. That is a chain with several links, and each one is something a person has to install and keep current.
Then there is the choice I'd read twice. NVIDIA describes time-slicing, with separate error domains, and simultaneous SM-sharing, with a shared error domain. Both carry memory limits, but the second is meant for workloads you trust to run together. A memory allowance is not a promise that every kind of failure stays in its own lane.
I'll give the post credit for explaining the mechanisms and configuration clearly. It does not offer an independent demonstration of savings or reliability on a particular cluster, and I have not tested any of this.
The test is yours to run. Fill the neighbours with busy work and measure latency. Check for failed allocations. Confirm your monitoring reports each workload's fraction. Change one condition at a time, on your own jobs and your own GPUs.
Source: Memory-Enforced GPU Sharing in Kubernetes with NVIDIA CUDA 13.4 · 2026-10-27
Why it matters
Denser GPU placement only saves money if your services stay predictable when the neighbours are busy, and that is something to measure on your own cluster.
