Feature Flags
Experimental capabilities ship behind named feature flags rather than as
hidden branches. A flag’s default can vary per release channel, so an
experiment can be on for alpha users while staying off for stable — no code
change per release.
Flags control capability execution or help visibility. Plugin loading is managed
separately with sparkrun plugins enable/disable. The defaults below are for
0.4.0; a flag never installs or enables its parent plugin or qualifies new hardware.
Managing flags
Section titled “Managing flags”# All flags, their effective state, and whysparkrun setup features list --all
# Turn one on / off explicitlysparkrun setup features enable executor.localsparkrun setup features disable executor.local
# Drop the explicit override and fall back to the channel defaultsparkrun setup features reset executor.locallist labels its columns FEATURE, STATE, SOURCE, OWNER, and
DESCRIPTION. State is the effective capability value; source explains its
origin (env, config, or channel), and owner identifies the declaring plugin
or core. With no subcommand, setup features lists enabled flags; list does
the same. Add --all to include disabled flags and setup-step flags. Disabled
plugins’ static declarations remain discoverable through that full inventory.
The enabled-only default also applies to --json, which includes
resolved values, overrides, ownership, and channel information.
Built-in flags
Section titled “Built-in flags”| Flag | Stable | Beta | Alpha | Purpose |
|---|---|---|---|---|
executor.docker | on | on | on | Docker executor |
gateway | on | on | on | Shared gateway capability gate |
gateway.litellm | on | on | on | Bundled default gateway |
cli.tune | on | on | on | Runtime kernel tuning |
cli.setup.wizard, setup.steps.* | on | on | on | Hardware/executor-scoped setup steps |
cli.setup.rdma_test | on | on | on | perftest RDMA traffic tests |
builder.uv_venv | off | on | on | Host-side Python environment preparation |
cli.setup.features | off | on | on | Help visibility only; command always callable |
daemon | off | off | on | Optional durable HTTP/SSE service |
executor.k8s, cli.setup.k8s, api.run.k8s | on | on | on | Kubernetes capabilities; require the k8s plugin to be loaded |
cli.setup.rdma_test.nccl | off | off | on | Experimental NCCL collective test suite |
gateway.sparkroute | off | off | off | Opt-in on every channel; requires separately installed SparkRoute plugin |
executor.local | off | off | off | Experimental native execution |
cli.setup.tailscale | off | off | off | Tailscale setup commands |
core.external_plugins | off | off | off | Managed/manual external plugin loading |
Setup step flags cover docker, docker_group, nvidia_container, nvidia_cdi,
host_ipc, earlyoom, sudoers, ssh_identity, ssh_mesh, cx7, and rdma.
Enabled steps still require an applicable hardware/executor plan.
SPARKRUN_ADVANCED=1 exposes advanced help; it does not enable capability flags.
model-cache is visible in normal help and shell completion.
plugins add can enable the external-loader
gate after source consent; --enable separately enables plugin loading.
Bundled LiteLLM, OCI Relay, and Arena plugins load by default on every channel.
The k8s and lil plugins default to alpha only; installed integrations require
selection. Use plugins enable k8s to load Kubernetes on stable/beta. Capability
flags still apply once a plugin is loaded. Legacy whole-plugin flags such as
integration.k8s, registry.lil, integration.arena, plugins.oci_relay, and
plugins.coldsnap remain migration inputs, but no longer appear in the feature
list. Explicit plugins.enabled overrides take precedence.
LiteLLM is the default gateway. Explicitly running
setup features enable gateway.sparkroute selects SparkRoute, even if its
capability was already enabled. It requires that plugin to be loaded and the
shared gateway gate to be on. Other providers’ flags remain unchanged;
features reset does not select a gateway. See plugins
and proxy.
setup features list is authoritative for the installed package and profile.
MCP is disabled and has no registered command/service entry point.
Resolution order
Section titled “Resolution order”Highest priority first:
- Environment override —
SPARKRUN_FEATURE_<NAME>, with dots and dashes becoming underscores and the name upper-cased.executor.local→SPARKRUN_FEATURE_EXECUTOR_LOCAL=1. Handy for CI and one-off debugging. - Explicit config override —
features.<name>inconfig.yaml. - Per-channel default declared on the flag.
- The flag’s baseline default.
- Unknown flag → off. Resolution fails closed.
features: channel: alpha # optional; defaults to the self_update channel executor.local: true # explicit override beats the channel defaultThe active channel follows your update channel
(stable / beta / alpha) unless features.channel overrides it — so a
stable install can preview an entire channel’s feature set without changing
which code it actually runs.
What a disabled flag does
Section titled “What a disabled flag does”Gating is enforced at the point of use, and it fails loudly rather than silently degrading:
- A gated executor is absent from
sparkrun cluster create --executorcompletion and from executor resolution. Naming it anyway raisesExecutorUnavailableErrorpointing at the flag to enable. - A gated command group (e.g.
setup tailscale) hides itself from--helpand refuses to run with a message naming the flag. - A gated transport fails closed at use, so an already-imported cluster can’t quietly fall back to plain SSH.
Teardown is deliberately not gated: a workload launched while a flag was on stays stoppable after you turn the flag off.
Disabling gateway hides gateway help and rejects new gateway work. Status, stop,
service status, and service removal remain available for recovery. Disabling a
selected provider does not automatically select a replacement.