// HACKER NEWS — CYBERSECURITY
DFlash 2: Keep Drafting Parallel
Inference is the bottleneck of the agent era. Agents read, plan, and call
tools, often for hours or days. They consume tokens at a rate chat never
approached. Every one of those tokens takes a full forward pass over the
model. At Inco AI, we are building the inference stack scaled to the token
economics of tomorrow. This post is a sneak peek.
Our team released DFlash in January; it
now runs in SGLang, vLLM, TensorRT-LLM, and llama.cpp. NVIDIA measured
up to 15× throughput
with it on Blackwell GPUs; Google reported
3× more tokens per second
on TPUs; CoreWeave's production Kimi K2.7 Code endpoint, the fastest for
that model on Artificial Analysis,
runs DFlash by default. The ecosystem now builds on it:
NVIDIA,
Red Hat, and
Modal have all published
DFlash drafters; Meta
(Muse Glimmer),
Poolside (Laguna),
Xiaomi
(MiMo-V2.5-Pro),
and NVIDIA
(Nemotron 3.5 Lightning)
ship official drafters with their own models. On Hugging Face, DFlash models have
been downloaded more than 3.5 million times (as of August 2026).
Speculative decoding is a core piece of the modern inference
stack.1 A small draft model guesses a block of tokens,
and the target model verifies the whole block in one forward pass. Good guesses
turn one pass into several tokens; bad ones just get thrown away. For years, though, the draft itself stayed autoregressive:
one token at a time. DFlash made it one-pass too: the entire block, every
position, predicted in parallel.
DFlash 2 pushes parallel drafting one step further: over 20% more output from
every verification pass, for around 1% added cycle latency, with the output
provably unchanged. Across benchmarks the gain runs 16–25%. With the
Qwen3.8-27B drafter released today, SGLang serves at 2.7–3.4× the
throughput of autoregressive decoding at batch size 1. Predicting every
position independently leaves headroom in two places: choosing the right
tokens and holding accuracy to the end of the block. DFlash 2 recovers
both without giving up the one-pass design.
DFlash 2 already runs in the mainstream inference engines:
Download and install the prebuilt oMLX with DFlash 2 support.
Open the Model Manager and edit mlx-community/Qwen3.8-27B-4bit. Configure DFlash with the following settings:
DFlash predicts every position independently, in parallel. Each pick is
plausible on its own. Yet nothing makes them fit together, and an
incoherent block is cut short at verification.
Recent methods such as Domino and
DSpark buy coherence with sequential
heads that rewrite each position's full-vocabulary distribution.
But is that costly autoregressive correction really necessary?
No. The evidence is already in DFlash's own candidate lists. Take the
first position: DFlash's top pick is right 85.4% of the time, but the
right token is in its top 16 candidates 99.5% of the time. Even when the top pick
is wrong, the right token is usually on the list.
An oracle that always picks the right candidate from the top 16 would
lift the acceptance length from 4.27 to 6.79. That gap is pure selection
headroom. We just need to select the right path through the candidates.