Modular MAX

The modular-max runtime launches Modular MAX with max serve. It supports one host, with multiple local GPUs when the host has enough devices. tensor_parallel selects local GPUs and does not add cluster nodes.

Example recipe

model: Qwen/Qwen3-1.7B
runtime: modular-max
container: modular/max-nvidia-full:latest
max_nodes: 1
defaults:
  port: 8000
  host: 0.0.0.0
  tensor_parallel: 1
  max_model_len: 8192

Choose an image version that supports your model and GPU. The example illustrates recipe structure; latest is mutable.

Configuration options

Use runtime and defaults and omit command for the supported settings below. Add a command only when you need an unusual engine flag that sparkrun does not map. These are the flags generated by sparkrun; the selected engine image must also support them.

Set values under defaults, for example port: 8000, or override them with -o port=9000. Unset values are omitted unless a runtime/platform default supplies them. Boolean toggles emit a bare flag when true and omit it when false, except where the table lists an explicit negative or valued form. Mapping/list values are rendered as one JSON argument for flags that accept JSON. Unknown keys do not automatically become engine flags.

The top-level model supplies --model. Port defaults to 8000 when unset.

Mappings: sparkrun 0.4.0 source.

Engine flag	Key under defaults	Value form
--port	port	Value
--host	host	Value
--max-length	max_model_len	Value
--max-batch-size	max_num_seqs	Value
--served-model-name	served_model_name	Value
--quantization-encoding	quantization	Value
--devices	devices	Value

devices takes a MAX device string such as gpu:0,1. When it is unset, tensor_parallel: N above one generates --devices gpu:0,...,N-1; for TP=1 the engine’s default is used. An explicit devices value takes precedence over this mapping. GPU assignments remain subject to sparkrun’s placement and device binding.

Placement

MAX is single-node only. On a DGX Spark with one GPU, tensor_parallel above one is rejected during preparation. Use a multi-GPU host for local tensor parallelism. See Schedulers & Placement for admission and Model Cache Management for model preparation.