Skip to content

Recipe Format

A sparkrun recipe is a YAML file that describes everything needed to launch an inference workload: the model, the container image, the runtime engine, and its settings. Recipes are the central abstraction in sparkrun — they let you capture a known-good configuration once and replay it with a single command.

Terminal window
sparkrun run my-recipe # use the default cluster
sparkrun run my-recipe --cluster mylab # select a saved cluster
sparkrun run my-recipe -o port=9000 # override any default

Start with model, runtime, and container. Set supported engine options in defaults; sparkrun generates the serve command:

recipe_version: "2"
model: Qwen/Qwen3-1.7B
runtime: vllm
container: scitrera/dgx-spark-vllm:0.16.0-t5
defaults:
port: 8000
host: 0.0.0.0
model: nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-NVFP4
model_revision: abc123def
runtime: vllm
min_nodes: 1
container: scitrera/dgx-spark-vllm:0.16.0-t5
metadata:
description: NVIDIA Nemotron 3 Nano 30B (upstream NVFP4) -- cluster or solo
maintainer: user@example.com
model_params: 30B
model_dtype: nvfp4
defaults:
port: 8000
host: 0.0.0.0
tensor_parallel: 1
gpu_memory_utilization: 0.8
max_model_len: 200000
served_model_name: nemotron3-30b-a3b
tool_call_parser: qwen3_coder
enable_auto_tool_choice: true
trust_remote_code: true
env:
VLLM_ALLOW_LONG_MAX_MODEL_LEN: "1"

The recipe format version, independent of the installed sparkrun version:

recipe_version: "2"

"2" is the current format and the default when this field is omitted. Examples throughout these docs use v2, even when they omit the version line. Use "2" for new recipes.

"1" identifies the legacy format from eugr’s run-recipe launcher. Sparkrun retains compatibility for existing recipes; see Legacy v1 Recipes for its fields and conversion guidance. Use min_sparkrun_version below when you need a minimum core release rather than a format version.

Declare the minimum core version needed by a recipe:

min_sparkrun_version: "0.4.0"

Use a version string, not a range such as >=0.4.0. Omitting the field imposes no minimum. A compatible sparkrun checks this before workload preparation; recipe validate reports incompatibility, while inspection and export remain available. Enforcement was introduced in 0.4.0, so older releases cannot enforce this requirement.

The HuggingFace model identifier to serve.

# Standard HuggingFace model
model: Qwen/Qwen3-1.7B
# GGUF model with quantization variant (llama-cpp runtime)
model: Qwen/Qwen3-1.7B-GGUF:Q8_0

For GGUF models, the colon syntax (repo:quant) selects a specific quantization variant. sparkrun uses this to download only the matching quant files rather than the entire repository.

The model value is injected into the command template as {model} and is also used for model pre-sync (downloading and distributing weights to target hosts before launch).

Pin the model to a specific HuggingFace revision (branch, tag, or commit hash).

model: QuantTrio/MiniMax-M2.5-AWQ
model_revision: bbe738792c

This affects model download, cache checking, VRAM estimation, and model sync. Use it for reproducible deployments — without it, a model’s main branch may change between downloads.

Which inference engine to use:

ValueEngineClusteringNotes
vllmvLLMNative or RayRecommended. Set defaults.distributed_executor_backend to mp or ray; omitted selects distributed/mp.
vllm-distributedvLLMNativeConcrete implementation name; prefer vllm with backend omitted or mp.
vllm-rayvLLMRayConcrete implementation name; prefer vllm with backend ray.
sglangSGLangNativeFirst-class. Solo and multi-node.
llama-cppllama.cppN/ASolo mode. GGUF models.
trtllmTensorRT-LLMMPINVIDIA TensorRT-LLM with MPI-based multi-node orchestration.
atlasAtlasNativePure-Rust engine with its own NCCL distribution. Composes tensor and expert parallelism (tensor_parallel, ep_size). Requires a GB10 host. See Atlas runtime.
modular-maxModular MAXN/ASingle-node only. tensor_parallel maps to local --devices, not to host count.

Explicitly setting runtime is recommended for clarity and forward-compatibility. However, this field is technically optional — when omitted, sparkrun uses automatic runtime detection to infer the runtime from the command field. If neither runtime nor a recognizable command is present, the recipe defaults to vllm behavior.

For vLLM, keep runtime: vllm when switching backends and change defaults.distributed_executor_backend. See vLLM backend selection for examples.

The Docker/OCI container image to run.

container: scitrera/dgx-spark-vllm:0.16.0-t5
container: scitrera/dgx-spark-sglang:0.5.8-t5
container: scitrera/dgx-spark-llama-cpp:b8076-cu131

Omit command when the runtime supports the settings you need. Prefer defaults so sparkrun can generate engine flags and manage distributed launches. Provide command only when you need unusual CLI flags that are not mapped by the runtime. The runtime pages list the supported flags and their defaults keys: vLLM, SGLang, llama.cpp, TensorRT-LLM, Atlas, and Modular MAX.

For example, vLLM’s --reasoning-parser-plugin needs an explicit command in this version of sparkrun. The referenced parser must be available inside the image:

command: |
vllm serve {model} \
--host {host} --port {port} \
--max-model-len {max_model_len} \
-tp {tensor_parallel} \
--reasoning-parser-plugin /opt/parsers/custom.py

A command template replaces automatic serve-flag generation. Include the engine options you need in that template; arbitrary defaults keys are not all appended to it. Sparkrun still handles runtime-specific coordination and supported managed arguments. {placeholder} values come from effective configuration and CLI overrides. Substitution is iterative when a value contains another placeholder.

New recipes should set min_nodes and max_nodes directly — they’re explicit, compose cleanly with CLI overrides, and cover every case the older flags do.

Minimum number of nodes required. Defaults to 1.

On DGX Spark, each node has one GPU, so min_nodes is effectively the minimum GPU count.

Maximum number of nodes supported. Defaults to unlimited.

Set max_nodes: 1 for models or runtimes that do not support multi-node inference.

mode — explicit topology mode (auto, solo, or cluster). Inferred from min_nodes / max_nodes when omitted.

ValueMeaning
auto(default) sparkrun decides based on node counts and CLI flags
soloForces single-node. Sets min_nodes = max_nodes = 1.
clusterForces multi-node. Requires 2+ hosts.

solo_only / cluster_only — boolean shorthand for the above:

solo_only: true # equivalent to max_nodes: 1, mode: solo
cluster_only: true # equivalent to min_nodes: 2, mode: cluster

Which backend launches the workload — docker (default) or local.

executor: local

Per-executor knobs. Docker accepts container-level settings; the experimental executors take their own:

executor: local
executor_config:
working_dir: /opt/qwen3
log_dir: /var/log/sparkrun
env_file: /etc/sparkrun.env
command_prefix: nice -n 10

See Executors for the field reference and the resolution chain. The experimental executors are off by default and must be enabled via a feature flag.

A recipe can run shell commands around the serve command. Execution order:

1. builder.prepare() — build/pull the container image
2. Distribution — sync image + model to hosts
3. Container launch
4. pre_exec — inside ALL containers, before serve
5. Serve command
6. [wait for healthy] — port + HTTP check (only when post hooks exist)
7. post_exec — inside the HEAD container only
8. post_commands — on the CONTROL MACHINE
9. [stop_after_post] — optional auto-stop
HookRuns
pre_execInside every container, before serve. Sequential, fail-fast.
post_execInside the head container, after the endpoint is healthy.
post_commandsOn your control machine, after post_exec.
stop_after_postWhen true, stops the workload once post hooks finish.
pre_exec:
- pip install --no-cache-dir some-plugin
post_exec:
- curl -s localhost:8000/v1/models
post_commands:
- echo "endpoint ready on {head_host}:{port}"

A dictionary of default parameter values, including mappings/lists for JSON-valued engine flags. Every key is available as a {placeholder} in the command template and can be overridden at launch via CLI flags or -o key=value.

defaults:
port: 8000
host: 0.0.0.0
tensor_parallel: 1
gpu_memory_utilization: 0.8
max_model_len: 200000
served_model_name: nemotron3-30b-a3b

These common keys provide portable names where the runtime supports them. Consult the runtime’s flag table for its exact mappings and limitations; not every engine has an equivalent for every setting.

Default keyCLI flagDescription
port--portServe port
host-o host=XBind address
tensor_parallel--tpTensor parallelism degree
gpu_memory_utilization--gpu-memGPU memory fraction (0.0–1.0)
max_model_len--max-model-lenMaximum sequence length
served_model_name--served-model-nameModel name exposed by the API (works for all runtimes)

Use the runtime’s flag table to choose supported keys. Runtime-specific settings such as tool_call_parser and attention_backend are generated automatically when mapped. Extra keys can supply conditional selectors or explicit command placeholders, but an arbitrary key is not automatically forwarded as a flag.

Config chain precedence, from lowest to highest: runtime defaults, platform defaults, recipe defaults, matching conditional overrides, then CLI overrides (--port 9000, --served-model-name my-model, -o key=value).

Use -o key.field=value to change one field without replacing the rest of a mapping-valued default. Nested paths work too:

defaults:
speculative_config:
method: dflash
model: z-lab/Qwen3.6-35B-A3B-DFlash
num_speculative_tokens: 5
Terminal window
sparkrun run my-recipe -o speculative_config.num_speculative_tokens=3 \
-o speculative_config.draft_load_config.load_format=auto

The example preserves method and model. The root must already contain a mapping or a JSON-object string; a missing/scalar root or a path through a scalar is rejected. To supply a whole mapping first, use JSON:

Terminal window
sparkrun run my-recipe -o 'speculative_config={"method":"dflash","model":"z-lab/Qwen3.6-35B-A3B-DFlash"}' \
-o speculative_config.num_speculative_tokens=3

Dotted overrides refine the effective value, including matching conditional overrides, and do not edit the recipe file. Keep nested YAML under its root in defaults; dotted YAML keys are not expanded. -o env.KEY=value retains its separate meaning: setting a container environment variable.

Environment variables injected into the container.

env:
VLLM_ALLOW_LONG_MAX_MODEL_LEN: "1"

Values are literal unless they contain one of the explicit environment template namespaces below.

Place templates directly in env to use effective configuration or prepared launch facts:

defaults:
tensor_parallel: 2
env:
TP_SIZE: "{config.tensor_parallel}"
NODE_RANK: "{launch.node_rank}"
MODEL_PATH: "{launch.model_path}"

{config.NAME} reads a scalar from effective configuration, including CLI overrides. {launch.NAME} supports:

FieldValue
model_pathPrepared model directory or selected GGUF file, mapped into the workload
model_revisionConcrete Hugging Face snapshot commit shared by the selected nodes
runtime_cache_dirThis launch’s persistent runtime-cache directory
num_nodesNumber of selected physical hosts
node_rankZero-based host ordinal, independent of GPU/process rank
node_hostSelected host name
cluster_idLaunch identifier

Values are rendered per host after asset preparation. They reach container creation, in-container hooks, and serving; controller-side post_commands do not inherit them. Path fields support Docker and local execution. Model fields check the prepared cache; a local model path has no Hugging Face revision. runtime_cache_dir requires an enabled runtime cache.

There is no shell evaluation, controller-environment lookup, or recursive substitution. $VAR, ${VAR}, and unrelated braces remain literal. Write {{launch.node_rank}} to preserve the literal {launch.node_rank}. Missing fields and non-scalar config values produce errors. Literal CLI environment overrides (-e KEY=value or -o env.KEY=value) replace templates without interpolation. Exports retain the template source rather than per-host paths.

Dry runs leave model paths and revisions marked unresolved without probing model caches. External-script runtimes do not support these templates; custom execution strategies must provide their own template handling. MPI launches cannot use values that differ across nodes. See Plugin Development for integration requirements.

Descriptive and VRAM-estimation fields. Does not affect how the workload runs.

metadata:
description: NVIDIA Nemotron 3 Nano 30B (upstream NVFP4)
maintainer: user@example.com
model_params: 30B
model_dtype: nvfp4

Prefer auto-detection. sparkrun pulls parameter counts, layer counts, head dimensions, and dtypes from each model’s HuggingFace config — for the overwhelming majority of recipes you don’t need to set any of these fields. They exist as manual overrides for the rare cases where auto-detect fails or returns misleading numbers, e.g., heavily-quantized variants, custom MoE topologies, or repos with missing/atypical config.json metadata.

When you do need to override, the values in metadata take precedence over anything sparkrun infers from the model itself.

FieldTypeDescription
model_paramsstringParameter count: "1.7B", "30B", "397B"
model_dtypestringWeight dtype: bf16, fp16, fp8, nvfp4, int4, q8_0, etc.
kv_dtypestringKV cache dtype (defaults to bfloat16)
num_layersintNumber of transformer layers
num_kv_headsintNumber of key-value attention heads
head_dimintDimension per attention head
model_vramfloatOverride: total model weight VRAM in GB
kv_vram_per_tokenfloatOverride: KV cache bytes per token
attention_layersintHybrid attention/recurrent model: count of layers with per-token KV cache; selects hybrid sizing
recurrent_state_bytesintHybrid model: total recurrent state bytes per sequence across recurrent layers

Use sparkrun show <recipe> or sparkrun recipe vram <recipe> to see what auto-detection currently reports — start there and only fill in metadata fields when the displayed estimate is off.

For hybrid attention/recurrent models such as NemotronH and Qwen3-Next/Qwen3.5, sparkrun detects the layer layout from hybrid_override_pattern, layers_block_type, or layer_types. Only attention layers contribute per-token KV cache; Mamba2/GatedDeltaNet layers add a fixed state per sequence. Maximum context sizing subtracts one sequence’s recurrent state before calculating the remaining token budget. If that state cannot be determined, the estimate is a lower bound. Layouts containing only sliding/full attention are not treated as recurrent hybrids. These fields normally come from model configuration; manual metadata overrides remain optional.

A mod is a named directory containing a run.sh script (plus any supporting files — patches, templates, env-setup snippets, etc.) that gets copied into the container and executed before the serve command. Mods are a builder/runtime-agnostic mechanism for small, named container tweaks (e.g., applying a Nemotron parser plugin, patching a Python module, dropping in custom config).

Originally introduced for eugr/spark-vllm-docker, mods are now first-class in sparkrun: any recipe can declare them, any registry can publish them, and any runtime sees them as plain pre_exec entries after Phase 1 resolution. See Execution Flow › Phase 1.

mods is a top-level list field on the recipe:

model: nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-NVFP4
runtime: vllm
container: ghcr.io/spark-arena/dgx-vllm-eugr-nightly:latest
mods:
- mods/nemotron-nano # resolved against the recipe's location / registry
- "@eugr/mods/qwen3-tools" # explicitly scoped to the @eugr registry
- my-private-tweak # short form — leading "mods/" is optional

The leading mods/ prefix is purely a naming convention and is stripped during resolution, so nemotron-nano and mods/nemotron-nano are equivalent.

For each entry in mods:, sparkrun walks four stages in order and uses the first hit:

#StageWhat it looks for
1Explicit scoped reference (@registry/<rel>)Resolves <rel> under that registry’s configured mods_subpath. Bypasses every other stage when the user opts in explicitly.
2Adjacent to the recipe fileTries <rel> and mods/<rel> relative to the directory of the recipe YAML — works for local recipes and recipes shipped inside a registry repo.
3Same registry as the recipeIf the recipe came from a registry, tries <mods_subpath>/<rel> inside that registry (using the registry’s configured mods_subpath).
4@eugr fallbackFalls back to the eugr registry — the original source of truth for mods, kept as a safety net for legacy recipes and shared community mods.

A mod source is valid when the target directory contains a run.sh. If none of the four stages find a match, sparkrun run fails loudly with the list of paths it tried.

Local mod adjacent to a local recipe (stage 2):

./my-recipe.yaml
./mods/fix-config/run.sh
my-recipe.yaml
mods:
- fix-config

Resolved via stage 2 — dirname(my-recipe.yaml)/mods/fix-config/run.sh. No registry needed.

Mod from the same registry as the recipe (stage 3):

A registry repo at github.com/myteam/spark-recipes ships:

recipes/qwen-tools.yaml
mods/qwen3-tool-parser/run.sh

…and its .sparkrun/registry.yaml declares mods: mods. Then qwen-tools.yaml can reference:

mods:
- qwen3-tool-parser

…and sparkrun finds it inside the same registry’s mods_subpath (stage 3 — assuming it’s not adjacent at stage 2).

Cross-registry reference with @registry/ (stage 1):

my-local-recipe.yaml
mods:
- "@official/nemotron-nano" # pulled from the @official registry's mods_subpath
- "@eugr/mods/qwen3-coder" # pulled from @eugr, with explicit "mods/" prefix

Useful for mixing-and-matching mods from registries that don’t own the recipe.

Legacy fallback to @eugr (stage 4):

A recipe authored against the original eugr workflow that just lists mods: [nemotron-nano] (no registry scope, not adjacent, not in its own registry) still resolves — sparkrun reaches stage 4 and looks up nemotron-nano inside the @eugr registry’s mods_subpath. This keeps older v1 recipes working without modification.

A dictionary for runtime-specific configuration:

runtime_config:
build_args: [--some-flag]

Unknown top-level keys are automatically swept into runtime_config.

Use include: to make a small variation of a sibling or registry recipe:

# Save beside base.yaml as long-context.yaml.
include: base
min_sparkrun_version: "0.4.0"
defaults:
max_model_len: 65536
metadata:
description: A longer-context variant

A base may be a sibling name (base or base.yaml, without path separators) or a qualified registry reference such as "@official/qwen3-1.7b-vllm". Underscore-prefixed siblings such as _common.yaml can hold shared settings. Bases can include other bases, up to eight levels; cycles are rejected. URL-fetched and catalog-imported recipes require qualified registry references.

Foreign-format bases such as include: "@lil/<Entry>" are loaded through their registered adapter before merging. The adapter must be loaded and its format trusted; an unavailable or untrusted format makes the include fail.

Inherited valueHow the child changes it
Mappings such as metadataMerge key by key
Individual values in defaults or envReplace the entire setting, including a nested JSON object
Scalars and lists, including hooks and modsReplace the base value
overridesAppend child entries after the base entries
Explicit nullDelete the inherited key

Changing model preserves inherited metadata, including declared weight and KV-cache dtypes. Child metadata still merges key by key. Review parameter counts, architecture fields, model URLs, and other model-specific estimates when switching models: override changed values or set individual fields such as model_params to null to remove them.

To clear the entire inherited block and let automatic model detection refill available fields:

include: base
model: org/different-model
metadata: null

Inherited registry mods keep their source registry. Trust follows the least-trusted member of the chain, so a local wrapper does not grant trust to a third-party base. Legacy v1 recipes cannot participate in includes.

export recipe flattens includes by default; --keep-include preserves the original reference and child settings.

Use overrides: for settings that vary with a model’s launch layout or hardware:

defaults:
max_num_seqs: 8
max_model_len: 32768
overrides:
- when: {nodes: 1}
defaults: {max_num_seqs: 1}
- when: {arch: sm_121, nodes: {gte: 2}}
defaults: {max_model_len: 65536}
- when: {config: {attention_backend: flashinfer}}
env: {MY_ATTENTION_BACKEND: "{config.attention_backend}"}

Each entry can change defaults, env, or container. Conditions within when must all match. Values accept equality, a list of alternatives, or operators eq, ne, in, not_in, gt, gte, lt, and lte.

SelectorMatches
tp, pp, dp, ep, cpResolved parallelism
nodes, gpus_per_nodeLaunch host count and GPUs per host
runtime, runtime_familyRuntime implementation or family
config: {key: value}Configuration before any conditional layer is applied
platform, vendor, accelerator, archHardware identity, such as dgx-spark, nvidia, gb10, sm_121
capabilityAccelerator capabilities, such as unified-memory
memory_gbPer-accelerator memory in GiB; host RAM for unified-memory devices

Matching entries apply in order; later entries win. CLI overrides still have the highest priority. Conditional layers cannot change port, served_model_name, or parallelism keys. Hardware-wide defaults, such as CUTE_DSL_ARCH, belong to the platform and usually need no recipe override.

On mixed clusters, placement groups hosts that agree on the hardware conditions and selects a compatible group. A layout spanning incompatible groups is rejected. Unknown selectors/operators do not match and produce diagnostics. run --dry-run reports which layers apply; normal workload lookup retains the recipe’s declared identity for stop, logs, and --ensure.

Use export recipe --realize to save the effective settings for selected hardware as an ordinary recipe.

While explicitly setting runtime is recommended, sparkrun can infer the runtime from the command field when it is omitted. This is useful for quick experimentation or when adapting command snippets from documentation without worrying about which runtime value to set.

Command prefixDetected runtime
vllm serve ...vllm (then resolved to vllm-distributed or vllm-ray)
sglang serve ...sglang
python -m sglang.launch_server ...sglang
python3 -m sglang.launch_server ...sglang
llama-server ...llama-cpp
trtllm-serve ...trtllm
mpirun ... trtllm ...trtllm

If the command doesn’t match any of these patterns, the recipe falls through to vllm behavior by default. Runtimes without a command-hint pattern — atlas, modular-max — must be named explicitly via the runtime field.

An explicit runtime field always wins — command-hint detection only fires when runtime is omitted. Existing recipes that set runtime continue to work identically.

Recipes can include a benchmark: block to define default benchmark settings used by sparkrun benchmark:

benchmark:
framework: llama-benchy
pp: [2048]
depth: [0, 4096]
prefix_caching: true

See the benchmark command for details on how benchmark configuration is resolved.

sparkrun searches for recipes in this order:

  1. @spark-arena/ shortcut — @spark-arena/UUID expands to a Spark Arena URL.
  2. URL — if the argument is an HTTP/HTTPS URL, the recipe is fetched directly and cached.
  3. @registry/recipe-name scoped lookup — disambiguate recipes across registries using scoped syntax.
  4. Exact/relative file path — if the argument is a path to an existing file.
  5. Current working directory — sparkrun scans .yaml/.yml files in the CWD that are valid recipes.
  6. Registry search — flat name lookup in configured custom registries, then recursive glob.

Filenames are matched with or without .yaml/.yml extensions.

The command field supports {placeholder} substitution from the config chain:

defaults:
port: 8000
served_model_name: my-model
command: |
vllm serve {model} --port {port} --served-model-name {served_model_name}

With sparkrun run recipe -o port=9000, this renders as:

vllm serve Qwen/Qwen3-1.7B --port 9000 --served-model-name my-model

The special placeholder {model} is always available from the top-level model field.

Readiness settings resolve from built-in defaults, then global config, then recipe. Supported runtime/executor combinations can perform a streaming inference check; others retain endpoint checks. For example:

readiness:
inference: true
inference_timeout_s: 1200

An explicitly unsupported inference style fails validation. --no-ready-wait makes no readiness/TTFT claim and conflicts with required post-launch hooks. See run for completion, timing, and failure semantics.

Model manifests bind immutable revisions, selected files and workload paths. Cluster distribution.model.validation, file_selection, and provider choose validation and transfer policy; global defaults are model_cache_validation, model_file_selection, and model_distribution_provider. Known damage never becomes success through legacy fallback. Offline mode reuses/copies available artifacts without fetching missing dependencies. Runtime compilation cache policy is separate from model files. See Model Cache Management.

Image preparation resolves per-host resources and executor destinations. Native assets need no unused image default; distribution providers must verify every required host before launch. Use Plugin Development for the current builder/runtime contracts.