Skip to content

sparkrun logs

Terminal window
sparkrun logs <target> [options]

Show available workload logs, with docker logs / journalctl semantics. TARGET can be a recipe name or a workload ID (from sparkrun status output).

Called bare, it dumps the workload’s logs and exits. Use -f to stay attached and stream new output — Ctrl+C detaches and never kills the inference job.

OptionDescription
--hosts / -HFull workload host list; preserve original topology
--hosts-fileFile with hosts (one per line, # comments)
--clusterUse a saved cluster by name
--tp / --tensor-parallelMatch host trimming from run
--portOverride port (to match run-time override)
--served-model-nameOverride served model name (to match run-time override)
-n / --lines / --tailNumber of log lines to show (default: all). With -f, sets the scrollback shown before following.
-f / --followStay attached and stream new lines
-a / --all-sourcesRead every worker/rank, not just the head

By default sparkrun logs reads the head. -a / --all-sources adds every worker:

  • Without -f, each source is dumped in full — head first, then workers by rank.
  • With -f, the streams interleave in arrival order.
Terminal window
# Dump everything the head has logged
sparkrun logs qwen3-1.7b-vllm --cluster mylab
# Last 200 lines, then follow
sparkrun logs qwen3-1.7b-vllm --cluster mylab -n 200 -f
# Follow head and all workers together
sparkrun logs e5f6a7b8 -f -a

Log reads never delete saved workload metadata or release model-cache claims. Workload stdout/stderr stays in the output; SSH, Docker-reader, or kubectl diagnostics are reported separately. Failed readers report an error, so a partial export must not be treated as complete. Ctrl+C stops readers, leaving the workload.

Saved native Kubernetes Job/JobSet workloads use local kubectl and their recorded connection settings. Completed containers remain readable while their logs exist. --all-sources includes regular containers and sidecars, not init containers or previous instances. A replaced pod or restarted container requires a fresh read; following does not automatically attach to replacements.

For interactive browsing, use sparkrun cluster monitor. Its log viewer supports regex search, runtime colorization, source scopes, older history, and complete file exports.

Python and daemon clients additionally support host/container/source filters, reusable targets, and cancellable readers. Python callers can page older history; LogTarget.dump() exports complete logs with cancellation cleanup and atomic publication. See Log Streaming API.