Skip to content

Creating Clusters

Clusters are named logical groups of hosts, including a single host. Groups may overlap; they do not partition the physical machines. Save all your nodes as the default cluster, then add smaller groups when useful. Prefer cluster names over explicit host lists for everyday commands. See Recommended Cluster Setups for one-, two-, four-, and eight-node examples.

Terminal window
sparkrun cluster create mylab \
--hosts 192.168.11.13,192.168.11.14 \
--default \
-d "My DGX Spark lab"
OptionDescription
--hosts / -HComma-separated list of host IPs or names
--defaultMake this the default cluster for subsequent commands
-d / --descriptionHuman-readable description
--userSSH user for connecting to hosts
--cache-dirOverride HuggingFace cache directory for this cluster
--transfer-modeResource distribution mode: auto (default), local, push, delegated
--transfer-interfaceNetwork interface for high-speed transfers between cluster hosts

The first host of the scheduled placement becomes head; occupancy scheduling can choose a different host from the first saved candidate. Host order still matters for greedy placement. Inspect the plan/status for the actual head.

Terminal window
# Save candidates; scheduling determines the actual head
sparkrun cluster create mylab --hosts 192.168.11.13,192.168.11.14

By default, sparkrun uses your current OS user for SSH. Set a per-cluster user:

Terminal window
sparkrun cluster create mylab \
--hosts 192.168.11.13,192.168.11.14 \
--user dgxuser

Or override per-command with --user.

Terminal window
sparkrun cluster create mylab \
--hosts 192.168.11.13,192.168.11.14 \
--transfer-mode delegated

Controls how container images and models are distributed to cluster hosts. Options: auto (default), local, push, delegated. See Networking for details on each mode.

Terminal window
sparkrun cluster create mylab \
--hosts 192.168.11.13,192.168.11.14 \
--cache-dir /data/hf-cache

Override the HuggingFace cache directory for this cluster. By default, sparkrun uses the standard HuggingFace cache resolution (see Troubleshooting for the full resolution order).

Modify hosts, description, or SSH user of an existing cluster:

Terminal window
sparkrun cluster update mylab --hosts 192.168.11.13,192.168.11.14,192.168.11.15
sparkrun cluster update mylab --user newuser
sparkrun cluster update mylab -d "Updated description"
# Add or remove individual hosts
sparkrun cluster update mylab --add-host 192.168.11.16
sparkrun cluster update mylab --remove-host 192.168.11.14
Terminal window
sparkrun cluster set-default mylab

When a default cluster is set, commands like sparkrun run use it automatically without needing --cluster or --hosts.

After creating a cluster, set up SSH access for multi-node inference.

Hardware inventory, per-host executor destinations, and live reservations determine placement. Ordinary workloads reserve accelerators exclusively; a low engine memory fraction does not automatically permit co-location. See Schedulers. Workload metadata is shared across direct and daemon calls, while daemon operation history is separate. Missing observations are not proof of idle capacity. Cluster model validation/selection/provider settings and runtime cache cleanup are covered in Model Cache Management.