Reflection Beam: 23B active parameters do not make a 23B deployment

Active parameters describe token work, not the whole machine

Reflection released Beam's weights on October 16 under Apache 2.0, with BF16, FP8, and NVFP4 checkpoints and day-zero support across several serving stacks. The headline architecture is a sparse mixture of experts with 501 billion total parameters and about 23 billion active per token. That second number is useful for understanding compute. It is dangerous when read as a memory budget.

An MoE router selects a small set of experts for each token. Arithmetic does not touch every expert on every step, so a large sparse model can spend far less compute per token than a dense model with the same total parameter count. But the next token may route to a different expert. Unless a system accepts severe transfer stalls or implements a carefully measured offload scheme, the serving tier must keep the full routed expert set available across its workers.

The current vLLM recipe makes the distinction concrete. Its published checkpoint sizes are 294 GB for NVFP4, 503 GB for FP8, and 1,003 GB for BF16. The recipe's minimum GPU-memory figures add 20 percent headroom for activations, CUDA graphs, and a working KV cache: 353 GB, 604 GB, and 1,204 GB respectively. Those are starting floors, not promises for your traffic.

This does not make Beam impractical. It makes it infrastructure. A team that already operates multi-GPU nodes may gain useful control over weights, precision, placement, prompts, and data. A team expecting a single 24 GB workstation experience should stop before download. The right decision comes from an explicit serving contract, not from treating “active” as “resident.”

The serving path has five capacity boundaries

Beam has 52 layers. The vLLM documentation describes dense MLPs in the first layers, then routed experts where each token uses 8 of 256 experts. Its attention design mixes 512-token sliding-window attention with full-attention layers without positional embeddings. The model supports a context limit of 1,048,576 tokens and uses the Harmony message format for reasoning and tool calls.

request + agent history
        |
        v
prefill workers ---- prefix / KV cache ---- offload tier
        |                    |                  |
        v                    v                  v
router -> selected experts -> attention -> next-token logits
        |                                      |
        +------- tensor / expert network -------+
                                               |
                                               v
                                   decode + optional drafter
                                               |
                                               v
                                    tool call or final answer

Weight residency is the first boundary. Precision determines the dominant artifact size, while parallelism determines how those weights are split. Quantization reduces storage and memory, but does not turn a sparse 501B checkpoint into a dense 23B checkpoint.

Interconnect is the second boundary. Tensor parallelism splits operations across GPUs. Expert parallelism places experts across workers and introduces all-to-all communication. A topology that fits can still lose latency and throughput if the interconnect cannot feed routed work efficiently.

Prefill is the third boundary. Agent sessions repeatedly ingest repository files, tool results, system instructions, and history. Long prompts favor high token throughput and prefix reuse. The model's advertised context length says what the protocol can accept; it does not say prefill is cheap.

Decode and KV cache are the fourth boundary. Every live history reserves cache state. Google Cloud's Beam guidance warns that dozens of long-context sessions can fill HBM, force cache eviction, reduce hit rates, and stall throughput. The useful capacity number is therefore concurrent cached tokens at an acceptable hit rate, not maximum context for one request.

Cold start and recovery are the fifth boundary. vLLM says first startup of the Blackwell NVFP4 path can take about ten minutes while FlashInfer autotunes the MoE kernel, close to the default engine-ready timeout. A node that works after manual patience can still fail automated rollout, autoscaling, replacement, or disaster recovery.

Choose precision as a capacity-quality decision

These figures come from the current vLLM recipe and should be versioned with the page you used. They do not include redundancy, spare capacity, orchestration overhead, host memory, network headroom, observability, or a second replica. A production budget that stops at “minimum GPU memory” has no room for maintenance or failure.

Reflection's model card is unusually useful because it publishes many evaluations across all three variants. The results are close on several listed tasks and vary on others. That is evidence worth using to design an acceptance test, not proof that quantization is lossless for your repository, language, tool schema, safety policy, or agent harness. Benchmark every release candidate with its exact chat template, parser, runtime, and checkpoint.

NVFP4 also has a hardware qualification. The vLLM recipe describes native W4A4 operation on Blackwell while Hopper serves FP4 experts through Marlin with BF16 activations. “The checkpoint loads” is not the same as “the intended kernel path runs.” Record GPU architecture, driver, CUDA, runtime build, backend, topology, and achieved kernel selection in the acceptance receipt.

Pin the runtime contract instead of copying a launch command

A minimal Blackwell NVFP4 command from the current vLLM recipe looks like this. Pin the image digest and verified dependency versions around it; the version placeholders below are deliberate.

export VLLM_ENGINE_READY_TIMEOUT_S=3600

vllm serve reflection/Beam-1.0-501B-A23B-NVFP4 \
  --revision <approved-model-revision> \
  --tensor-parallel-size 4 \
  --kv-cache-dtype fp8 \
  --enable-auto-tool-choice \
  --tool-call-parser beam \
  --reasoning-parser beam

The parser flags are part of correctness. Beam uses a structured reasoning and tool-calling format; a generic parser can turn a model-quality problem into a protocol problem. Test ordinary answers, tool calls, parallel or repeated calls if your harness permits them, malformed tool results, stop tokens, empty content, cancellation, retry, and resumed conversations.

DFlash2 speculative decoding is another separate release candidate. The drafter shares embeddings and the output head with the target model and adds memory overhead. vLLM recommends seven speculative tokens, supports up to fifteen, and notes that acceptance depends on natural-language input. The recipe explicitly warns against benchmarking with random tokens. A draft model can raise throughput when it guesses your output distribution well and waste capacity when it does not.

--speculative-config '{
  "method": "dflash",
  "model": "reflection/Beam-1.0-501B-A23B-DFlash2",
  "num_speculative_tokens": 7,
  "attention_backend": "FLASH_ATTN",
  "enable_adaptive_verification": true
}'

Run the drafter as an experiment cell, not as a default. Report accepted draft tokens, decode throughput, p50/p95/p99 latency, cache capacity, output equivalence, and failure behavior. If it helps only at a concurrency level your service rarely reaches, the extra moving part may not earn its operating cost.

Build a serving-capacity worksheet from live tokens

Start with a measured workload trace. Separate system and repository prefix tokens, new user input, retrieved context, tool-result growth, reasoning/output tokens, turn count, session duration, concurrent sessions, cache reuse, and cancellation. Averages hide exactly the long sessions that evict useful cache.

required serving envelope
  = model weights
  + runtime / graphs / activations
  + KV cache for live tokens at target concurrency
  + optional drafter
  + fragmentation and safety reserve

fleet capacity
  = healthy replicas at maintenance state
  - one defined failure unit
  - rollout / canary reserve

accepted load
  = highest arrival rate where quality, p95 latency,
    cache hit rate, error rate, and recovery all pass

Do not use the equation to invent precision. Measure resident memory and cache behavior on the exact build. Treat the model's one-million-token window as an upper bound to test selectively, not a default allocation. Most agent tasks should have a context policy that compacts, retrieves, or restarts before an unbounded transcript consumes the fleet.

Gate the whole loop, not a model leaderboard

Reflection reports strong results across software engineering, reasoning, tool calling, long context, and safety evaluations. Those tables are primary-source evidence about the vendor's test setup. Google Cloud reports up to 5x throughput and 8x lower latency for selected optimized deployments and labels the figures as customer benchmarking. Keep both types of claim in the research file with their conditions and reproduce only the decisions your system needs.

A practical acceptance suite freezes representative agent transcripts and their external environment. Include repository navigation, patch generation, command selection, tool schema conformance, failed command recovery, conflicting instructions, long tool results, context compaction, secret handling, prompt injection, cancellation, and deterministic verification. Compare BF16, FP8, and NVFP4 only where you can actually operate them.

Approve an immutable release tuple: model revision, precision, runtime image, driver, backend, parser, template, drafter setting, hardware topology, cache policy, admission policy, and acceptance-suite revision. A change in any of those fields is a new candidate, even when the marketing name remains “Beam 1.0.”

Failure modes that a successful demo will not reveal

Use a four-stage rollout

Stage 1 - Artifact and protocol. Freeze the license, model files, hashes, tokenizer, template, runtime, and parser. Prove clean startup and valid ordinary, reasoning, and tool-call responses on one supported topology.

Stage 2 - Workload acceptance. Run the same fixtures across viable precisions. Measure accepted-task quality, protocol errors, tokens, latency, memory, and cost. Include prohibited actions and failure recovery. Reject any precision that crosses a material quality or safety boundary.

Stage 3 - Capacity and chaos. Replay production-shaped histories under target concurrency. Exercise cache pressure, cancellation, queue limits, node loss, cold replacement, rolling updates, and the fallback model. Verify N-1 capacity and evidence collection.

Stage 4 - Bounded production. Route a narrow workload, keep human review where consequences require it, compare against the incumbent, and publish a daily capacity-quality ledger. Expand only after the p95, error, cache, accepted-task, and recovery gates remain stable.

• Record exact checkpoint, precision, runtime, backend, driver, GPU topology, parser, and prompt format.
• Reserve measured headroom for runtime, cache, drafter, fragmentation, rollout, and one failure unit.
• Define maximum live tokens per session and admission behavior before load arrives.
• Separate published benchmark context from local workload evidence.
• Keep quantization, drafter, cache policy, and model rollback independently reversible.
• Re-run acceptance after every model, runtime, driver, kernel, template, or topology change.

FAQ

Can Beam run like a 23B model because it activates 23B parameters?

No. The active count describes the routed compute for a token. Serving still needs access to the full expert set. Current official recipes list a 294 GB NVFP4 checkpoint and a 353 GB minimum GPU-memory floor before production redundancy.

Which precision should a team deploy?

Choose the smallest precision that passes the team's exact coding, tool, safety, and long-context fixtures on the intended hardware. Do not pay for BF16 automatically, and do not accept NVFP4 automatically.

Does the one-million-token window make context management unnecessary?

No. Large windows move pressure into prefill, KV cache, admission, latency, and cost. Context management remains a reliability and capacity control, especially with concurrent agents.

Are the Google Cloud performance numbers independently verified?

The post labels them as customer benchmarking and reports specific optimized stacks. Treat them as vendor-reported evidence and a reproduction target, not a universal result.

How does this relate to the open-weight adoption contract?

The open-weight adoption contract classifies license, artifacts, lineage, and operating evidence. This guide starts after a candidate exists and turns one named model's runtime into a measurable capacity and release contract.