What does it actually take to run China’s open frontier models?

Not published

Vendor documents supported engines but does not publish a smallest runnable GPU configuration.

Requires cluster sizing

Not responsibly specifiable before a concurrency and context benchmark; workstation deployment is not credible.

Validated reference

NVIDIA Dynamo publishes full-1M profiles using 8x GB300 or 16x GB200 per aggregated worker; disaggregated profiles use more GPUs. Primary recipe

Heterogeneous reference

KTransformers documents an 8-GPU CPU/GPU-offload launch shape with 96 CPU inference threads; this is a tutorial target, not a claimed absolute minimum. Primary recipe

Benchmark required

Start from a replicated TP8 service only after measuring the target context mix; no official 200-seat capacity claim exists.

Benchmark required

Requires redundant replicas or disaggregated prefill/decode; GPU count cannot be inferred from seats alone.

Validated reference not minimum

vLLM publishes 8x B300, 8x H200, 8x MI355X and GB200 profiles; none is labelled the absolute minimum. Primary recipe

Benchmark required

An 8-GPU worker may be a capacity building block, but replicas depend on concurrency and output-token demand.

Validated reference

8x H200 is documented with context capped at 800K to preserve KV headroom; production redundancy requires additional workers. Primary recipe

Candidate recipe under review

A 4x RTX PRO 6000 NVFP4 recipe is under review upstream; V1 does not promote an unmerged recipe to validated minimum.

Benchmark required

A quantized multi-GPU worker is plausible, but the 200-seat recommendation needs measured TTFT, throughput and context mix.

Framework support maturing

Aggregated and disaggregated vLLM recipes are still landing; pinning a nightly build may be required.

Official launch shape

Official serving examples use TP4 and 262K context. Smaller quantized short-context shapes exist, but V1 does not call them the official minimum.

Best v1 benchmark candidate

Use two measured serving replicas as the initial HA design candidate; exact GPUs remain pending load tests.

Benchmark required

Scale through replicated workers after measuring prefill-heavy and decode-heavy traffic separately.

Vendor run path not minimum

Official local instructions and engine integrations are published, but no absolute minimum GPU count is claimed. Primary recipe

Benchmark required

Use one measured tensor-parallel worker as the capacity baseline; add replicas only after concurrency tests.

Framework supported

vLLM and SGLang support are available; production still requires redundant workers and pinned encoding logic. Primary recipe

Official local minimum

The official GGUF path specifies about 120GB minimum unified memory/VRAM and recommends 128GB. Primary recipe

Production candidate

Start with one TP4 NVFP4 or TP8 FP8 worker and benchmark the actual image and context mix.

Official launch shapes

Official examples publish TP4 NVFP4 and TP8 FP8/BF16 serving shapes; replicas are still required for HA. Primary recipe

Not published

GPU and NPU paths are documented, but the vendor does not publish an absolute minimum hardware shape.

Cluster sizing required

A 1.6T model needs a measured cluster design; employee count alone is not a capacity input.

Official serving paths

Official GPU and NPU serving paths exist; topology, redundancy and throughput remain operator-specific. Primary recipe

Not published

Transformers can load the checkpoint, but the vendor does not state an absolute minimum GPU configuration.

Benchmark required

Use the official distributed recipe as a starting worker and size replicas from the real modality mix.

Official distributed recipe

The official card shows an FP8 SGLang DP2×TP8 configuration at 262K context. Primary recipe

Memory estimate not official

A 24GB-class GPU is a reasonable BF16 short-context planning target, but the vendor does not label this an official minimum.

Best low cost candidate

Benchmark one or two 24–48GB workers before considering larger models; exact replicas depend on output length.

Mainstream framework support

Official Transformers, vLLM and SGLang examples are published; add redundant replicas for availability. Primary recipe

Official tp4 launch

Official SGLang and vLLM examples use tensor parallel size 4. Primary recipe

Not a seat based service

Size by simulation jobs and trajectory length, not employee seats.

Specialized service

Expose behind a task-specific simulator contract; do not market it as a normal chat-completions quality substitute.

Consumer gpu claim no vram

The vendor says BF16 inference is optimized for consumer GPUs but publishes no exact minimum VRAM. Primary recipe

Robot fleet profile required

Size by robots, camera rate and control latency; a 200-seat office profile is not applicable.

Edge control not public api

Prefer an on-robot or near-edge safety architecture; a remote multi-tenant API is not the default production shape.

Single gpu supported no vram

The model card supports single-GPU inference but does not publish exact minimum VRAM. Primary recipe

Media queue required

Size a queued render farm by jobs per hour, resolution and duration; office-seat assumptions do not apply.

Official 8gpu reference

The official Ulysses SP8 path reports roughly one minute for a 720p synchronized clip; HA needs additional workers. Primary recipe

Not published

The official local Gradio and Python path requires CUDA 12+, but exact minimum VRAM is not published. Primary recipe

Audio workload profile required

Size by audio hours, peak simultaneous streams and latency rather than office seats.

Custom service required

The release provides local inference code, not a validated multi-tenant serving recipe; build queueing, batching and observability.