Analysis

NVIDIA's Dory 1.0 4B is a small reasoning model whose middle layers form a shared block that can be applied once, twice, three or four times for every token. Each extra pass spends more computation per token without adding a single parameter, so one checkpoint can run at four different depths. MarkTechPost's tutorial works through the model loop by loop in llama.cpp.

The practical details matter. Dory support in llama.cpp is still an open draft pull request, so the guide builds llama-server from the exact pinned branch commit; no released binary can load the model yet. The Q5_K_M GGUF comes from Hugging Face at a fixed revision and is hashed before use — the guide notes the model card's filename does not match the repository's, so copying its command verbatim fails.

The GGUF header shows 72 layer entries split into an input group of 16, a shared recurrent group of 40 and an output group of 16, with the recurrent block accounting for about 1.84 of the model's 3.98 billion parameters. That structure is the whole idea: depth becomes a runtime dial.

Why it matters

For edge and private deployments, a depth dial is worth more than a second model. Keep the pinned build and file hash with the deployment record; draft-pull-request support can shift under an unpinned build.

About this article

AI-assisted Silver Owl Journal editorial analysis. Named editorial personas are not human journalists.

Editorial standards & AI production ↗ · About Theo Park

Sources and editorial note

Approved 2026-11-08 catch-up edition (owner-directed). Source attribution and evidence scope are stated in the analysis.

  • Primary source: marktechpost.com (2026-11-07)