Recipe Format
A sparkrun recipe is a YAML file that describes everything needed to launch an inference workload: the model, the container image, the runtime engine, and its settings. Recipes are the central abstraction in sparkrun — they let you capture a known-good configuration once and replay it with a single command.
sparkrun run my-recipe # use the default clustersparkrun run my-recipe --cluster mylab # select a saved clustersparkrun run my-recipe -o port=9000 # override any defaultMinimal recipe
Section titled “Minimal recipe”Start with model, runtime, and container. Set supported engine options in
defaults; sparkrun generates the serve command:
recipe_version: "2"model: Qwen/Qwen3-1.7Bruntime: vllmcontainer: scitrera/dgx-spark-vllm:0.16.0-t5
defaults: port: 8000 host: 0.0.0.0Full recipe example
Section titled “Full recipe example”model: nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-NVFP4model_revision: abc123defruntime: vllmmin_nodes: 1container: scitrera/dgx-spark-vllm:0.16.0-t5
metadata: description: NVIDIA Nemotron 3 Nano 30B (upstream NVFP4) -- cluster or solo maintainer: user@example.com model_params: 30B model_dtype: nvfp4
defaults: port: 8000 host: 0.0.0.0 tensor_parallel: 1 gpu_memory_utilization: 0.8 max_model_len: 200000 served_model_name: nemotron3-30b-a3b tool_call_parser: qwen3_coder enable_auto_tool_choice: true trust_remote_code: true
env: VLLM_ALLOW_LONG_MAX_MODEL_LEN: "1"Core fields
Section titled “Core fields”recipe_version
Section titled “recipe_version”The recipe format version, independent of the installed sparkrun version:
recipe_version: "2""2" is the current format and the default when this field is omitted. Examples
throughout these docs use v2, even when they omit the version line. Use "2"
for new recipes.
"1" identifies the legacy format from eugr’s run-recipe launcher. Sparkrun
retains compatibility for existing recipes; see Legacy v1 Recipes
for its fields and conversion guidance. Use min_sparkrun_version below when
you need a minimum core release rather than a format version.
min_sparkrun_version
Section titled “min_sparkrun_version”Declare the minimum core version needed by a recipe:
min_sparkrun_version: "0.4.0"Use a version string, not a range such as >=0.4.0. Omitting the field imposes no
minimum. A compatible sparkrun checks this before workload preparation;
recipe validate reports incompatibility, while inspection and export remain
available. Enforcement was introduced in 0.4.0, so older releases cannot enforce
this requirement.
model (required)
Section titled “model (required)”The HuggingFace model identifier to serve.
# Standard HuggingFace modelmodel: Qwen/Qwen3-1.7B
# GGUF model with quantization variant (llama-cpp runtime)model: Qwen/Qwen3-1.7B-GGUF:Q8_0For GGUF models, the colon syntax (repo:quant) selects a specific quantization variant. sparkrun uses this to download only the matching quant files rather than the entire repository.
The model value is injected into the command template as {model} and is also used for model pre-sync (downloading and distributing weights to target hosts before launch).
model_revision
Section titled “model_revision”Pin the model to a specific HuggingFace revision (branch, tag, or commit hash).
model: QuantTrio/MiniMax-M2.5-AWQmodel_revision: bbe738792cThis affects model download, cache checking, VRAM estimation, and model sync. Use it for reproducible deployments — without it, a model’s main branch may change between downloads.
runtime
Section titled “runtime”Which inference engine to use:
| Value | Engine | Clustering | Notes |
|---|---|---|---|
vllm | vLLM | Native or Ray | Recommended. Set defaults.distributed_executor_backend to mp or ray; omitted selects distributed/mp. |
vllm-distributed | vLLM | Native | Concrete implementation name; prefer vllm with backend omitted or mp. |
vllm-ray | vLLM | Ray | Concrete implementation name; prefer vllm with backend ray. |
sglang | SGLang | Native | First-class. Solo and multi-node. |
llama-cpp | llama.cpp | N/A | Solo mode. GGUF models. |
trtllm | TensorRT-LLM | MPI | NVIDIA TensorRT-LLM with MPI-based multi-node orchestration. |
atlas | Atlas | Native | Pure-Rust engine with its own NCCL distribution. Composes tensor and expert parallelism (tensor_parallel, ep_size). Requires a GB10 host. See Atlas runtime. |
modular-max | Modular MAX | N/A | Single-node only. tensor_parallel maps to local --devices, not to host count. |
Explicitly setting runtime is recommended for clarity and forward-compatibility. However, this field is technically optional — when omitted, sparkrun uses automatic runtime detection to infer the runtime from the command field. If neither runtime nor a recognizable command is present, the recipe defaults to vllm behavior.
For vLLM, keep runtime: vllm when switching backends and change
defaults.distributed_executor_backend. See
vLLM backend selection for examples.
container (required)
Section titled “container (required)”The Docker/OCI container image to run.
container: scitrera/dgx-spark-vllm:0.16.0-t5container: scitrera/dgx-spark-sglang:0.5.8-t5container: scitrera/dgx-spark-llama-cpp:b8076-cu131command (optional)
Section titled “command (optional)”Omit command when the runtime supports the settings you need. Prefer
defaults so sparkrun can generate engine flags and manage distributed launches.
Provide command only when you need unusual CLI flags that are not mapped by
the runtime. The runtime pages list the supported flags and their defaults keys:
vLLM,
SGLang,
llama.cpp,
TensorRT-LLM,
Atlas, and
Modular MAX.
For example, vLLM’s --reasoning-parser-plugin needs an explicit command in this
version of sparkrun. The referenced parser must be available inside the image:
command: | vllm serve {model} \ --host {host} --port {port} \ --max-model-len {max_model_len} \ -tp {tensor_parallel} \ --reasoning-parser-plugin /opt/parsers/custom.pyA command template replaces automatic serve-flag generation. Include the engine
options you need in that template; arbitrary defaults keys are not all appended
to it. Sparkrun still handles runtime-specific coordination and supported managed
arguments. {placeholder} values come from effective configuration and CLI
overrides. Substitution is iterative when a value contains another placeholder.
Topology fields
Section titled “Topology fields”New recipes should set min_nodes and max_nodes directly — they’re explicit, compose cleanly with CLI overrides, and cover every case the older flags do.
min_nodes
Section titled “min_nodes”Minimum number of nodes required. Defaults to 1.
On DGX Spark, each node has one GPU, so min_nodes is effectively the minimum GPU count.
max_nodes
Section titled “max_nodes”Maximum number of nodes supported. Defaults to unlimited.
Set max_nodes: 1 for models or runtimes that do not support multi-node inference.
Deprecated: mode, solo_only, cluster_only
Section titled “Deprecated: mode, solo_only, cluster_only”mode — explicit topology mode (auto, solo, or cluster). Inferred from min_nodes / max_nodes when omitted.
| Value | Meaning |
|---|---|
auto | (default) sparkrun decides based on node counts and CLI flags |
solo | Forces single-node. Sets min_nodes = max_nodes = 1. |
cluster | Forces multi-node. Requires 2+ hosts. |
solo_only / cluster_only — boolean shorthand for the above:
solo_only: true # equivalent to max_nodes: 1, mode: solocluster_only: true # equivalent to min_nodes: 2, mode: clusterExecutor fields
Section titled “Executor fields”executor
Section titled “executor”Which backend launches the workload — docker (default) or local.
executor: localexecutor_config
Section titled “executor_config”Per-executor knobs. Docker accepts container-level settings; the experimental executors take their own:
executor: localexecutor_config: working_dir: /opt/qwen3 log_dir: /var/log/sparkrun env_file: /etc/sparkrun.env command_prefix: nice -n 10See Executors for the field reference and the resolution chain. The experimental executors are off by default and must be enabled via a feature flag.
Lifecycle hooks
Section titled “Lifecycle hooks”A recipe can run shell commands around the serve command. Execution order:
1. builder.prepare() — build/pull the container image2. Distribution — sync image + model to hosts3. Container launch4. pre_exec — inside ALL containers, before serve5. Serve command6. [wait for healthy] — port + HTTP check (only when post hooks exist)7. post_exec — inside the HEAD container only8. post_commands — on the CONTROL MACHINE9. [stop_after_post] — optional auto-stop| Hook | Runs |
|---|---|
pre_exec | Inside every container, before serve. Sequential, fail-fast. |
post_exec | Inside the head container, after the endpoint is healthy. |
post_commands | On your control machine, after post_exec. |
stop_after_post | When true, stops the workload once post hooks finish. |
pre_exec: - pip install --no-cache-dir some-plugin
post_exec: - curl -s localhost:8000/v1/models
post_commands: - echo "endpoint ready on {head_host}:{port}"Configuration fields
Section titled “Configuration fields”defaults
Section titled “defaults”A dictionary of default parameter values, including mappings/lists for JSON-valued
engine flags. Every key is available as a {placeholder} in the command template
and can be overridden at launch via CLI flags or -o key=value.
defaults: port: 8000 host: 0.0.0.0 tensor_parallel: 1 gpu_memory_utilization: 0.8 max_model_len: 200000 served_model_name: nemotron3-30b-a3bStandardized defaults
Section titled “Standardized defaults”These common keys provide portable names where the runtime supports them. Consult the runtime’s flag table for its exact mappings and limitations; not every engine has an equivalent for every setting.
| Default key | CLI flag | Description |
|---|---|---|
port | --port | Serve port |
host | -o host=X | Bind address |
tensor_parallel | --tp | Tensor parallelism degree |
gpu_memory_utilization | --gpu-mem | GPU memory fraction (0.0–1.0) |
max_model_len | --max-model-len | Maximum sequence length |
served_model_name | --served-model-name | Model name exposed by the API (works for all runtimes) |
Use the runtime’s flag table to choose supported keys. Runtime-specific settings
such as tool_call_parser and attention_backend are generated automatically
when mapped. Extra keys can supply conditional selectors or explicit command
placeholders, but an arbitrary key is not automatically forwarded as a flag.
Config chain precedence, from lowest to highest: runtime defaults, platform
defaults, recipe defaults, matching conditional overrides,
then CLI overrides (--port 9000, --served-model-name my-model, -o key=value).
Override one field of a mapping
Section titled “Override one field of a mapping”Use -o key.field=value to change one field without replacing the rest of a
mapping-valued default. Nested paths work too:
defaults: speculative_config: method: dflash model: z-lab/Qwen3.6-35B-A3B-DFlash num_speculative_tokens: 5sparkrun run my-recipe -o speculative_config.num_speculative_tokens=3 \ -o speculative_config.draft_load_config.load_format=autoThe example preserves method and model. The root must already contain a
mapping or a JSON-object string; a missing/scalar root or a path through a scalar
is rejected. To supply a whole mapping first, use JSON:
sparkrun run my-recipe -o 'speculative_config={"method":"dflash","model":"z-lab/Qwen3.6-35B-A3B-DFlash"}' \ -o speculative_config.num_speculative_tokens=3Dotted overrides refine the effective value, including matching conditional
overrides, and do not edit the recipe file. Keep nested YAML under its root in
defaults; dotted YAML keys are not expanded. -o env.KEY=value retains its
separate meaning: setting a container environment variable.
Environment variables injected into the container.
env: VLLM_ALLOW_LONG_MAX_MODEL_LEN: "1"Values are literal unless they contain one of the explicit environment template namespaces below.
Environment templates
Section titled “Environment templates”Place templates directly in env to use effective configuration or prepared
launch facts:
defaults: tensor_parallel: 2env: TP_SIZE: "{config.tensor_parallel}" NODE_RANK: "{launch.node_rank}" MODEL_PATH: "{launch.model_path}"{config.NAME} reads a scalar from effective configuration, including CLI
overrides. {launch.NAME} supports:
| Field | Value |
|---|---|
model_path | Prepared model directory or selected GGUF file, mapped into the workload |
model_revision | Concrete Hugging Face snapshot commit shared by the selected nodes |
runtime_cache_dir | This launch’s persistent runtime-cache directory |
num_nodes | Number of selected physical hosts |
node_rank | Zero-based host ordinal, independent of GPU/process rank |
node_host | Selected host name |
cluster_id | Launch identifier |
Values are rendered per host after asset preparation. They reach container
creation, in-container hooks, and serving; controller-side post_commands do
not inherit them. Path fields support Docker and local execution. Model fields
check the prepared cache; a local model path has no Hugging Face revision.
runtime_cache_dir requires an enabled runtime cache.
There is no shell evaluation, controller-environment lookup, or recursive
substitution. $VAR, ${VAR}, and unrelated braces remain literal. Write
{{launch.node_rank}} to preserve the literal {launch.node_rank}. Missing
fields and non-scalar config values produce errors. Literal CLI environment
overrides (-e KEY=value or -o env.KEY=value) replace templates without
interpolation. Exports retain the template source rather than per-host paths.
Dry runs leave model paths and revisions marked unresolved without probing model caches. External-script runtimes do not support these templates; custom execution strategies must provide their own template handling. MPI launches cannot use values that differ across nodes. See Plugin Development for integration requirements.
Metadata fields
Section titled “Metadata fields”metadata
Section titled “metadata”Descriptive and VRAM-estimation fields. Does not affect how the workload runs.
metadata: description: NVIDIA Nemotron 3 Nano 30B (upstream NVFP4) maintainer: user@example.com model_params: 30B model_dtype: nvfp4VRAM estimation fields (optional)
Section titled “VRAM estimation fields (optional)”Prefer auto-detection. sparkrun pulls parameter counts, layer counts, head dimensions, and dtypes from each model’s HuggingFace config — for the overwhelming majority of recipes you don’t need to set any of these fields. They exist as manual overrides for the rare cases where auto-detect fails or returns misleading numbers, e.g., heavily-quantized variants, custom MoE topologies, or repos with missing/atypical config.json metadata.
When you do need to override, the values in metadata take precedence over anything sparkrun infers from the model itself.
| Field | Type | Description |
|---|---|---|
model_params | string | Parameter count: "1.7B", "30B", "397B" |
model_dtype | string | Weight dtype: bf16, fp16, fp8, nvfp4, int4, q8_0, etc. |
kv_dtype | string | KV cache dtype (defaults to bfloat16) |
num_layers | int | Number of transformer layers |
num_kv_heads | int | Number of key-value attention heads |
head_dim | int | Dimension per attention head |
model_vram | float | Override: total model weight VRAM in GB |
kv_vram_per_token | float | Override: KV cache bytes per token |
attention_layers | int | Hybrid attention/recurrent model: count of layers with per-token KV cache; selects hybrid sizing |
recurrent_state_bytes | int | Hybrid model: total recurrent state bytes per sequence across recurrent layers |
Use sparkrun show <recipe> or sparkrun recipe vram <recipe> to see what auto-detection currently reports — start there and only fill in metadata fields when the displayed estimate is off.
For hybrid attention/recurrent models such as NemotronH and Qwen3-Next/Qwen3.5,
sparkrun detects the layer layout from hybrid_override_pattern,
layers_block_type, or layer_types. Only attention layers contribute per-token
KV cache; Mamba2/GatedDeltaNet layers add a fixed state per sequence. Maximum
context sizing subtracts one sequence’s recurrent state before calculating the
remaining token budget. If that state cannot be determined, the estimate is a
lower bound. Layouts containing only sliding/full attention are not treated as
recurrent hybrids. These fields normally come from model configuration; manual
metadata overrides remain optional.
A mod is a named directory containing a run.sh script (plus any supporting files — patches, templates, env-setup snippets, etc.) that gets copied into the container and executed before the serve command. Mods are a builder/runtime-agnostic mechanism for small, named container tweaks (e.g., applying a Nemotron parser plugin, patching a Python module, dropping in custom config).
Originally introduced for eugr/spark-vllm-docker, mods are now first-class in sparkrun: any recipe can declare them, any registry can publish them, and any runtime sees them as plain pre_exec entries after Phase 1 resolution. See Execution Flow › Phase 1.
Declaring mods
Section titled “Declaring mods”mods is a top-level list field on the recipe:
model: nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-NVFP4runtime: vllmcontainer: ghcr.io/spark-arena/dgx-vllm-eugr-nightly:latest
mods: - mods/nemotron-nano # resolved against the recipe's location / registry - "@eugr/mods/qwen3-tools" # explicitly scoped to the @eugr registry - my-private-tweak # short form — leading "mods/" is optionalThe leading mods/ prefix is purely a naming convention and is stripped during resolution, so nemotron-nano and mods/nemotron-nano are equivalent.
Resolution order
Section titled “Resolution order”For each entry in mods:, sparkrun walks four stages in order and uses the first hit:
| # | Stage | What it looks for |
|---|---|---|
| 1 | Explicit scoped reference (@registry/<rel>) | Resolves <rel> under that registry’s configured mods_subpath. Bypasses every other stage when the user opts in explicitly. |
| 2 | Adjacent to the recipe file | Tries <rel> and mods/<rel> relative to the directory of the recipe YAML — works for local recipes and recipes shipped inside a registry repo. |
| 3 | Same registry as the recipe | If the recipe came from a registry, tries <mods_subpath>/<rel> inside that registry (using the registry’s configured mods_subpath). |
| 4 | @eugr fallback | Falls back to the eugr registry — the original source of truth for mods, kept as a safety net for legacy recipes and shared community mods. |
A mod source is valid when the target directory contains a run.sh. If none of the four stages find a match, sparkrun run fails loudly with the list of paths it tried.
Examples
Section titled “Examples”Local mod adjacent to a local recipe (stage 2):
./my-recipe.yaml./mods/fix-config/run.shmods: - fix-configResolved via stage 2 — dirname(my-recipe.yaml)/mods/fix-config/run.sh. No registry needed.
Mod from the same registry as the recipe (stage 3):
A registry repo at github.com/myteam/spark-recipes ships:
recipes/qwen-tools.yamlmods/qwen3-tool-parser/run.sh…and its .sparkrun/registry.yaml declares mods: mods. Then qwen-tools.yaml can reference:
mods: - qwen3-tool-parser…and sparkrun finds it inside the same registry’s mods_subpath (stage 3 — assuming it’s not adjacent at stage 2).
Cross-registry reference with @registry/ (stage 1):
mods: - "@official/nemotron-nano" # pulled from the @official registry's mods_subpath - "@eugr/mods/qwen3-coder" # pulled from @eugr, with explicit "mods/" prefixUseful for mixing-and-matching mods from registries that don’t own the recipe.
Legacy fallback to @eugr (stage 4):
A recipe authored against the original eugr workflow that just lists mods: [nemotron-nano] (no registry scope, not adjacent, not in its own registry) still resolves — sparkrun reaches stage 4 and looks up nemotron-nano inside the @eugr registry’s mods_subpath. This keeps older v1 recipes working without modification.
Runtime-specific fields
Section titled “Runtime-specific fields”runtime_config
Section titled “runtime_config”A dictionary for runtime-specific configuration:
runtime_config: build_args: [--some-flag]Unknown top-level keys are automatically swept into runtime_config.
Recipe inheritance
Section titled “Recipe inheritance”Use include: to make a small variation of a sibling or registry recipe:
# Save beside base.yaml as long-context.yaml.include: basemin_sparkrun_version: "0.4.0"defaults: max_model_len: 65536metadata: description: A longer-context variantA base may be a sibling name (base or base.yaml, without path separators) or
a qualified registry reference such as "@official/qwen3-1.7b-vllm".
Underscore-prefixed siblings such as _common.yaml can hold shared settings.
Bases can include other bases, up to eight levels; cycles are rejected.
URL-fetched and catalog-imported recipes require qualified registry references.
Foreign-format bases such as include: "@lil/<Entry>" are loaded through their
registered adapter before merging. The adapter must be loaded and its format
trusted; an unavailable or untrusted format makes the include fail.
| Inherited value | How the child changes it |
|---|---|
Mappings such as metadata | Merge key by key |
Individual values in defaults or env | Replace the entire setting, including a nested JSON object |
| Scalars and lists, including hooks and mods | Replace the base value |
overrides | Append child entries after the base entries |
Explicit null | Delete the inherited key |
Changing model preserves inherited metadata, including declared weight and
KV-cache dtypes. Child metadata still merges key by key. Review parameter counts,
architecture fields, model URLs, and other model-specific estimates when switching
models: override changed values or set individual fields such as model_params
to null to remove them.
To clear the entire inherited block and let automatic model detection refill available fields:
include: basemodel: org/different-modelmetadata: nullInherited registry mods keep their source registry. Trust follows the least-trusted member of the chain, so a local wrapper does not grant trust to a third-party base. Legacy v1 recipes cannot participate in includes.
export recipe flattens includes by default; --keep-include
preserves the original reference and child settings.
Conditional overrides
Section titled “Conditional overrides”Use overrides: for settings that vary with a model’s launch layout or hardware:
defaults: max_num_seqs: 8 max_model_len: 32768overrides: - when: {nodes: 1} defaults: {max_num_seqs: 1} - when: {arch: sm_121, nodes: {gte: 2}} defaults: {max_model_len: 65536} - when: {config: {attention_backend: flashinfer}} env: {MY_ATTENTION_BACKEND: "{config.attention_backend}"}Each entry can change defaults, env, or container. Conditions within when
must all match. Values accept equality, a list of alternatives, or operators
eq, ne, in, not_in, gt, gte, lt, and lte.
| Selector | Matches |
|---|---|
tp, pp, dp, ep, cp | Resolved parallelism |
nodes, gpus_per_node | Launch host count and GPUs per host |
runtime, runtime_family | Runtime implementation or family |
config: {key: value} | Configuration before any conditional layer is applied |
platform, vendor, accelerator, arch | Hardware identity, such as dgx-spark, nvidia, gb10, sm_121 |
capability | Accelerator capabilities, such as unified-memory |
memory_gb | Per-accelerator memory in GiB; host RAM for unified-memory devices |
Matching entries apply in order; later entries win. CLI overrides still have
the highest priority. Conditional layers cannot change port,
served_model_name, or parallelism keys. Hardware-wide defaults, such as
CUTE_DSL_ARCH, belong to the platform and usually need no recipe override.
On mixed clusters, placement groups hosts that agree on the hardware conditions
and selects a compatible group. A layout spanning incompatible groups is
rejected. Unknown selectors/operators do not match and produce diagnostics.
run --dry-run reports which layers apply; normal workload lookup retains the
recipe’s declared identity for stop, logs, and --ensure.
Use export recipe --realize to
save the effective settings for selected hardware as an ordinary recipe.
Automatic runtime detection
Section titled “Automatic runtime detection”While explicitly setting runtime is recommended, sparkrun can infer the runtime from the command field when it is omitted. This is useful for quick experimentation or when adapting command snippets from documentation without worrying about which runtime value to set.
| Command prefix | Detected runtime |
|---|---|
vllm serve ... | vllm (then resolved to vllm-distributed or vllm-ray) |
sglang serve ... | sglang |
python -m sglang.launch_server ... | sglang |
python3 -m sglang.launch_server ... | sglang |
llama-server ... | llama-cpp |
trtllm-serve ... | trtllm |
mpirun ... trtllm ... | trtllm |
If the command doesn’t match any of these patterns, the recipe falls through to vllm behavior by default. Runtimes without a command-hint pattern — atlas, modular-max — must be named explicitly via the runtime field.
An explicit runtime field always wins — command-hint detection only fires when runtime is omitted. Existing recipes that set runtime continue to work identically.
Benchmark configuration
Section titled “Benchmark configuration”Recipes can include a benchmark: block to define default benchmark settings used by sparkrun benchmark:
benchmark: framework: llama-benchy pp: [2048] depth: [0, 4096] prefix_caching: trueSee the benchmark command for details on how benchmark configuration is resolved.
Recipe discovery
Section titled “Recipe discovery”sparkrun searches for recipes in this order:
@spark-arena/shortcut —@spark-arena/UUIDexpands to a Spark Arena URL.- URL — if the argument is an HTTP/HTTPS URL, the recipe is fetched directly and cached.
@registry/recipe-namescoped lookup — disambiguate recipes across registries using scoped syntax.- Exact/relative file path — if the argument is a path to an existing file.
- Current working directory — sparkrun scans
.yaml/.ymlfiles in the CWD that are valid recipes. - Registry search — flat name lookup in configured custom registries, then recursive glob.
Filenames are matched with or without .yaml/.yml extensions.
Command template substitution
Section titled “Command template substitution”The command field supports {placeholder} substitution from the config chain:
defaults: port: 8000 served_model_name: my-model
command: | vllm serve {model} --port {port} --served-model-name {served_model_name}With sparkrun run recipe -o port=9000, this renders as:
vllm serve Qwen/Qwen3-1.7B --port 9000 --served-model-name my-modelThe special placeholder {model} is always available from the top-level model field.
Readiness
Section titled “Readiness”Readiness settings resolve from built-in defaults, then global config, then recipe. Supported runtime/executor combinations can perform a streaming inference check; others retain endpoint checks. For example:
readiness: inference: true inference_timeout_s: 1200An explicitly unsupported inference style fails validation. --no-ready-wait makes
no readiness/TTFT claim and conflicts with required post-launch hooks. See
run for completion, timing, and failure semantics.
Cache and distribution policy
Section titled “Cache and distribution policy”Model manifests bind immutable revisions, selected files and workload paths.
Cluster distribution.model.validation, file_selection, and provider choose
validation and transfer policy; global defaults are model_cache_validation,
model_file_selection, and model_distribution_provider. Known damage never
becomes success through legacy fallback. Offline mode reuses/copies available
artifacts without fetching missing dependencies. Runtime compilation cache policy
is separate from model files. See Model Cache Management.
Image preparation resolves per-host resources and executor destinations. Native assets need no unused image default; distribution providers must verify every required host before launch. Use Plugin Development for the current builder/runtime contracts.