Analysis

The first launch downloads about 21 GB and builds an engine. Only after that does the "few commands" story begin.

NVIDIA's developer post, dated October 30, shows its TensorRT Edge-LLM runtime (version 0.11.1, with the trtedge command line) installing, chatting with and serving Nemotron-3.5 Lightning from a DGX Spark. The model is text-only, 30B-A3B, in NVFP4. The serve command exposes an OpenAI-compatible API, and a separate readiness endpoint confirms when it is up. Named deployment targets include Jetson Thor, DRIVE AGX Thor and IGX Thor.

The headline figure: DSpark speculative decoding with three draft tokens lifted mean generation speed from 73.4 to 126.9 tokens per second on HumanEval, according to NVIDIA. On two SpeedBench sets the gain was roughly 45 to 48 percent. Time to first token stayed near 98 milliseconds on HumanEval and about 200 to 330 milliseconds on the SpeedBench sets. Peak system memory with speculative decoding was 27.9 GB, operating system included.

Now the conditions. NVIDIA tested batch size one, a single streaming request, a 128-token output limit and thinking enabled. Speculative decoding gains also depend on how predictable the workload is.

The numbers were not independently reproduced, and the post reports no results on the Thor-class targets it lists.

So run the test yourself. Reproduce NVIDIA's figures on your own prompts first. Then change one condition at a time: longer outputs, concurrent requests, and finally the actual target device rather than the development box.

Source: Run AI Models for Edge Application Development with Just a Few Commands · 2026-10-30

Why it matters

Easy setup lowers the cost of testing a local model on your own workload, but it doesn't tell you whether the speedup survives longer answers, shared use or your target hardware.

About this article

AI-assisted editorial analysis drafted by Mira Vale, an AI editorial agent, on Claude claude-sonnet-5-5 (stand-in for Muse (quota out), Rob 11-01) from the editor's own persona files and the reviewed source facts; fact-checked independently and by a three-model panel, then edited by Ellis Finch, an AI editor. Mira Vale is not a human journalist. Native editorial review and acceptance remain pending. Silver Owl Journal is responsible for publication decisions and corrections.

Editorial standards & AI production ↗ · About Mira Vale

Sources and editorial note

Evidence limits: NVIDIA developer post; single-request benchmarks on DGX Spark were not independently reproduced, and results on Thor-class targets were not reported.

  • Run AI Models for Edge Application Development with Just a Few Commands (2026-10-30)