Skip to content

sparkrun run

Prefer a saved default cluster or --cluster NAME for host selection, including single-node use. See Recommended Cluster Setups.

Terminal window
sparkrun run <recipe> [options]

Launch an inference workload on one or more DGX Spark systems. sparkrun resolves the recipe, distributes container images and models, configures networking, and starts containers on all target hosts.

Jobs launch detached. By default, sparkrun follows startup logs and returns once readiness and required hooks complete. Ctrl+C cancels observers and leaves the detached workload running.

ModeModel logsCompletion
DefaultFollow startupReadiness and required hooks complete
--no-followSuppressedStill waits for readiness
--no-ready-waitNo ongoing attachmentLaunch checks complete; readiness is unconfirmed
--followContinue after readinessCtrl+C or log attachment ends

--follow conflicts with either negative flag. --no-ready-wait also conflicts with --foreground and required post-launch hooks. Invalid combinations exit 2 before launch. A normal ready launch exits 0; launch/readiness/hook failures exit

  1. Ctrl+C while waiting/following exits 130. A readiness timeout does not necessarily stop or imply a crash in the detached workload; inspect status and logs. A broken startup log stream does not determine readiness. With explicit --follow, a terminated log attachment exits 1.

--ensure matches launch intent and waits for an existing deployment’s readiness at its recorded head/port without rerunning hooks. Partial or unrecoverable placement fails readiness; --ensure --no-ready-wait checks presence only. Foreground mode retains workload-lifetime behavior, including generated systemd units. Unsupported host endpoints (currently Kubernetes) need --no-ready-wait for CLI submission. Library submission has its own contract.

Supported vLLM/SGLang Docker launches finish with a streaming inference probe; other declared runtime/executor combinations retain endpoint checks. A listening port alone is not proof of successful generation. Startup TTR port-open, TTR HTTP-ready, and TTFT measure from the serving container’s start, not total CLI wall time. --no-timings hides the final timing/readiness block but not the live readiness line. Unsupported observations omit measurements instead of inventing zeros. See recipe readiness.

OptionDescription
--hosts / -HComma-separated candidate hosts; first scheduled host becomes head
--hosts-fileFile with hosts (one per line, # comments)
--clusterUse a saved cluster by name
--portOverride serve port
--tp / --tensor-parallelOverride tensor parallelism
--pp / --pipeline-parallelOverride pipeline parallelism
--gpu-mem / --gpu-memory-utilization / --mem-fraction-staticOverride GPU memory utilization (0.0–1.0)
--max-model-lenOverride maximum model context length
--served-model-nameOverride served model name
--imageOverride container image (not recommended)
-o / --optionOverride a recipe default with -o key=value, or one mapping field with -o key.field=value (repeatable)
--dry-run / -nShow what would be done without executing
--trustAuthorize recipe hooks, mods, and trust-gated execution settings. See Security & Trust.
--foregroundRun in foreground (don’t detach)
--ensureReuse matching launch intent; still wait for readiness
--no-followSuppress model logs; still wait for readiness
--followKeep watching model logs after readiness
--no-ready-waitReturn after synchronous launch checks, without readiness or attachment
--timings / --no-timingsShow/hide the final stage timing breakdown (on by default)
--offline / --onlineReuse/copy existing artifacts without internet, or override cluster offline policy
--runtime-cache / --no-runtime-cacheToggle persistent compilation/autotuning caches subject to recipe policy and kill switch
-e / --envRepeatable environment override KEY=VALUE; preserves the literal value
--no-auto-detectSkip HuggingFace metadata lookup for estimates; not a complete offline policy
--no-sync-tuningSkip syncing tuning configs from registries
--no-rmDon’t auto-remove containers on exit (keeps containers after stop)
--memory-limitContainer memory limit (e.g. 32G)
--labelSet metadata on the container, e.g. --label com.example.key=value (repeatable)
--rootfulRun with --privileged as root inside container (legacy behavior)

Use -o key=value to override recipe defaults — that is the supported way to pass engine-level settings through to the serve command. For a mapping-valued default, -o speculative_config.num_speculative_tokens=3 preserves its other fields. The root must already be a mapping or JSON-object string. See mapping overrides for nested paths and whole-value examples.

Use --offline when recipes, images, and model files are already available:

Terminal window
sparkrun run my-recipe --cluster mylab --offline
# Make offline operation the cluster default.
sparkrun cluster update mylab --offline
# Allow online acquisition for one run, or restore the cluster's online default.
sparkrun run my-recipe --cluster mylab --online
sparkrun cluster update mylab --online

Offline mode permits copying existing images and models within the cluster, but skips internet acquisition and registry refreshes. Missing required assets produce a preflight error. Cache the recipe registry, images, models, and any required tools ahead of time. Connectivity failures do not automatically switch the launch into offline mode.

--offline / --online override the saved cluster policy for that operation. The same choices apply to benchmark and proxy load; offline benchmarking also needs its framework and evaluation assets already available. --no-auto-detect only skips metadata estimation and is not an offline policy. See Model Cache Management for cache validation and export for offline pinning.

The <recipe> argument can be:

  • A local recipe name — matched against configured registries (e.g. qwen3-1.7b-vllm)
  • A file path — a local .yaml file (e.g. ./my-recipe.yaml)
  • A Spark Arena shortcut — @spark-arena/<recipe-id> to run directly from Spark Arena
  • A URL — any https:// URL pointing to a recipe YAML
Terminal window
# Normal placement in the default cluster
sparkrun run qwen3-1.7b-vllm
# Two-node tensor parallel
sparkrun run qwen3-1.7b-vllm --tp 2
# Saved cluster with overrides
sparkrun run qwen3-1.7b-vllm --cluster mylab -o max_model_len=8192
# Run a recipe from Spark Arena
sparkrun run @spark-arena/<recipe-id>
# Dry run to preview commands
sparkrun run nemotron3-nano-30b-nvfp4-vllm --tp 2 --dry-run

Some options are hidden in normal help; use SPARKRUN_ADVANCED=1 sparkrun run --help to expose supported advanced controls. A few compatibility flags remain hidden but callable. Visibility does not enable gated capabilities:

OptionDescription
--soloForce single-host mode regardless of how many hosts are available
--dp / --data-parallelOverride data parallelism
--schedulerPlacement scheduler: occupancy-sparse, occupancy-dense, or greedy. See Schedulers & Placement.
--transfer-modeResource transfer mode: auto, local, push, or delegated (overrides the cluster setting)
--collect-diagnostics <path>Capture planning, launch, readiness, DEBUG logs, and plugin evidence in a bounded, redacted NDJSON bundle. See Run Diagnostics.
--restart <policy>Docker restart policy (no, always, unless-stopped, on-failure[:N])
--rebuild / --no-rebuildForce the builder to produce a fresh image. For docker-pull, forces a fresh registry pull; for eugr, forces a rebuild or fresh pull according to its build path. Overrides the recipe’s builder_config.rebuild.
--container-nameOverride the deterministic workload ID (i.e. use a static container name)
--executor-argsArguments passed directly to the container executor, e.g. docker run (repeatable)
--ray-portRay GCS port — default 46379 (Ray runtimes only)
--init-portvLLM/SGLang distributed init port — default 25000
--dashboard / --no-dashboardEnable or disable the Ray dashboard on the head node (Ray runtimes only; binds 0.0.0.0 when on)
--dashboard-portRay dashboard port — default 8265
Terminal window
# Capture full run diagnostics for debugging
sparkrun run my-recipe --collect-diagnostics run_diag.ndjson
# Set container restart policy
sparkrun run my-recipe --restart unless-stopped
# Pin placement behavior for this launch
sparkrun run my-recipe --scheduler occupancy-dense