sparkrun run
Prefer a saved default cluster or --cluster NAME for host selection, including
single-node use. See Recommended Cluster Setups.
sparkrun run <recipe> [options]Launch an inference workload on one or more DGX Spark systems. sparkrun resolves the recipe, distributes container images and models, configures networking, and starts containers on all target hosts.
Jobs launch detached. By default, sparkrun follows startup logs and returns once readiness and required hooks complete. Ctrl+C cancels observers and leaves the detached workload running.
Readiness and exit status
Section titled “Readiness and exit status”| Mode | Model logs | Completion |
|---|---|---|
| Default | Follow startup | Readiness and required hooks complete |
--no-follow | Suppressed | Still waits for readiness |
--no-ready-wait | No ongoing attachment | Launch checks complete; readiness is unconfirmed |
--follow | Continue after readiness | Ctrl+C or log attachment ends |
--follow conflicts with either negative flag. --no-ready-wait also conflicts
with --foreground and required post-launch hooks. Invalid combinations exit 2
before launch. A normal ready launch exits 0; launch/readiness/hook failures exit
- Ctrl+C while waiting/following exits 130. A readiness timeout does not
necessarily stop or imply a crash in the detached workload; inspect
statusandlogs. A broken startup log stream does not determine readiness. With explicit--follow, a terminated log attachment exits 1.
--ensure matches launch intent and waits for an existing deployment’s readiness
at its recorded head/port without rerunning hooks. Partial or unrecoverable
placement fails readiness; --ensure --no-ready-wait checks presence only.
Foreground mode retains workload-lifetime behavior, including generated systemd
units. Unsupported host endpoints (currently Kubernetes) need --no-ready-wait
for CLI submission. Library submission has its own contract.
Supported vLLM/SGLang Docker launches finish with a streaming inference probe;
other declared runtime/executor combinations retain endpoint checks. A listening
port alone is not proof of successful generation. Startup TTR port-open, TTR
HTTP-ready, and TTFT measure from the serving container’s start, not total CLI
wall time. --no-timings hides the final timing/readiness block but not the live
readiness line. Unsupported observations omit measurements instead of inventing
zeros. See recipe readiness.
Options
Section titled “Options”| Option | Description |
|---|---|
--hosts / -H | Comma-separated candidate hosts; first scheduled host becomes head |
--hosts-file | File with hosts (one per line, # comments) |
--cluster | Use a saved cluster by name |
--port | Override serve port |
--tp / --tensor-parallel | Override tensor parallelism |
--pp / --pipeline-parallel | Override pipeline parallelism |
--gpu-mem / --gpu-memory-utilization / --mem-fraction-static | Override GPU memory utilization (0.0–1.0) |
--max-model-len | Override maximum model context length |
--served-model-name | Override served model name |
--image | Override container image (not recommended) |
-o / --option | Override a recipe default with -o key=value, or one mapping field with -o key.field=value (repeatable) |
--dry-run / -n | Show what would be done without executing |
--trust | Authorize recipe hooks, mods, and trust-gated execution settings. See Security & Trust. |
--foreground | Run in foreground (don’t detach) |
--ensure | Reuse matching launch intent; still wait for readiness |
--no-follow | Suppress model logs; still wait for readiness |
--follow | Keep watching model logs after readiness |
--no-ready-wait | Return after synchronous launch checks, without readiness or attachment |
--timings / --no-timings | Show/hide the final stage timing breakdown (on by default) |
--offline / --online | Reuse/copy existing artifacts without internet, or override cluster offline policy |
--runtime-cache / --no-runtime-cache | Toggle persistent compilation/autotuning caches subject to recipe policy and kill switch |
-e / --env | Repeatable environment override KEY=VALUE; preserves the literal value |
--no-auto-detect | Skip HuggingFace metadata lookup for estimates; not a complete offline policy |
--no-sync-tuning | Skip syncing tuning configs from registries |
--no-rm | Don’t auto-remove containers on exit (keeps containers after stop) |
--memory-limit | Container memory limit (e.g. 32G) |
--label | Set metadata on the container, e.g. --label com.example.key=value (repeatable) |
--rootful | Run with --privileged as root inside container (legacy behavior) |
Use -o key=value to override recipe defaults — that is the supported way to
pass engine-level settings through to the serve command.
For a mapping-valued default, -o speculative_config.num_speculative_tokens=3
preserves its other fields. The root must already be a mapping or JSON-object
string. See mapping overrides
for nested paths and whole-value examples.
Offline launches
Section titled “Offline launches”Use --offline when recipes, images, and model files are already available:
sparkrun run my-recipe --cluster mylab --offline
# Make offline operation the cluster default.sparkrun cluster update mylab --offline
# Allow online acquisition for one run, or restore the cluster's online default.sparkrun run my-recipe --cluster mylab --onlinesparkrun cluster update mylab --onlineOffline mode permits copying existing images and models within the cluster, but skips internet acquisition and registry refreshes. Missing required assets produce a preflight error. Cache the recipe registry, images, models, and any required tools ahead of time. Connectivity failures do not automatically switch the launch into offline mode.
--offline / --online override the saved cluster policy for that operation.
The same choices apply to benchmark and proxy load; offline benchmarking
also needs its framework and evaluation assets already available.
--no-auto-detect only skips metadata estimation and is not an offline policy.
See Model Cache Management for cache validation
and export for offline pinning.
Recipe sources
Section titled “Recipe sources”The <recipe> argument can be:
- A local recipe name — matched against configured registries (e.g.
qwen3-1.7b-vllm) - A file path — a local
.yamlfile (e.g../my-recipe.yaml) - A Spark Arena shortcut —
@spark-arena/<recipe-id>to run directly from Spark Arena - A URL — any
https://URL pointing to a recipe YAML
Examples
Section titled “Examples”# Normal placement in the default clustersparkrun run qwen3-1.7b-vllm
# Two-node tensor parallelsparkrun run qwen3-1.7b-vllm --tp 2
# Saved cluster with overridessparkrun run qwen3-1.7b-vllm --cluster mylab -o max_model_len=8192
# Run a recipe from Spark Arenasparkrun run @spark-arena/<recipe-id>
# Dry run to preview commandssparkrun run nemotron3-nano-30b-nvfp4-vllm --tp 2 --dry-runAdvanced options
Section titled “Advanced options”Some options are hidden in normal help; use SPARKRUN_ADVANCED=1 sparkrun run --help to expose supported advanced controls. A few compatibility flags remain hidden but callable. Visibility does not enable gated capabilities:
| Option | Description |
|---|---|
--solo | Force single-host mode regardless of how many hosts are available |
--dp / --data-parallel | Override data parallelism |
--scheduler | Placement scheduler: occupancy-sparse, occupancy-dense, or greedy. See Schedulers & Placement. |
--transfer-mode | Resource transfer mode: auto, local, push, or delegated (overrides the cluster setting) |
--collect-diagnostics <path> | Capture planning, launch, readiness, DEBUG logs, and plugin evidence in a bounded, redacted NDJSON bundle. See Run Diagnostics. |
--restart <policy> | Docker restart policy (no, always, unless-stopped, on-failure[:N]) |
--rebuild / --no-rebuild | Force the builder to produce a fresh image. For docker-pull, forces a fresh registry pull; for eugr, forces a rebuild or fresh pull according to its build path. Overrides the recipe’s builder_config.rebuild. |
--container-name | Override the deterministic workload ID (i.e. use a static container name) |
--executor-args | Arguments passed directly to the container executor, e.g. docker run (repeatable) |
--ray-port | Ray GCS port — default 46379 (Ray runtimes only) |
--init-port | vLLM/SGLang distributed init port — default 25000 |
--dashboard / --no-dashboard | Enable or disable the Ray dashboard on the head node (Ray runtimes only; binds 0.0.0.0 when on) |
--dashboard-port | Ray dashboard port — default 8265 |
# Capture full run diagnostics for debuggingsparkrun run my-recipe --collect-diagnostics run_diag.ndjson
# Set container restart policysparkrun run my-recipe --restart unless-stopped
# Pin placement behavior for this launchsparkrun run my-recipe --scheduler occupancy-dense