Analysis
The first launch downloads about 21 GB and builds an engine. Only after that does the "few commands" story begin.
NVIDIA's developer post, dated October 30, shows its TensorRT Edge-LLM runtime (version 0.11.1, with the trtedge command line) installing, chatting with and serving Nemotron-3.5 Lightning from a DGX Spark. The model is text-only, 30B-A3B, in NVFP4. The serve command exposes an OpenAI-compatible API, and a separate readiness endpoint confirms when it is up. Named deployment targets include Jetson Thor, DRIVE AGX Thor and IGX Thor.
The headline figure: DSpark speculative decoding with three draft tokens lifted mean generation speed from 73.4 to 126.9 tokens per second on HumanEval, according to NVIDIA. On two SpeedBench sets the gain was roughly 45 to 48 percent. Time to first token stayed near 98 milliseconds on HumanEval and about 200 to 330 milliseconds on the SpeedBench sets. Peak system memory with speculative decoding was 27.9 GB, operating system included.
Now the conditions. NVIDIA tested batch size one, a single streaming request, a 128-token output limit and thinking enabled. Speculative decoding gains also depend on how predictable the workload is.
The numbers were not independently reproduced, and the post reports no results on the Thor-class targets it lists.
So run the test yourself. Reproduce NVIDIA's figures on your own prompts first. Then change one condition at a time: longer outputs, concurrent requests, and finally the actual target device rather than the development box.
Source: Run AI Models for Edge Application Development with Just a Few Commands · 2026-10-30
Why it matters
Easy setup lowers the cost of testing a local model on your own workload, but it doesn't tell you whether the speedup survives longer answers, shared use or your target hardware.
