Architecture & Source Layout
The 0.4.0 architecture separates frontend policy from public operations and executor-specific work. Use Python API and Plugin Development as public contracts; private implementation files below are navigation aids, not supported imports.
Source layout
Section titled “Source layout”| Package/module | Responsibility |
|---|---|
application.py, core/application_profile.py, core/application_identity.py | One application/profile/config binding and controller identity |
cli/ | Click commands, confirmation, terminal rendering, diagnostics |
api/ | Console-free options, operations, typed results and exceptions |
daemon/ | Authenticated HTTP/SSE, durable operation journal, workers and clients |
core/bootstrap.py, core/registration.py | Plugin initialization and shared registration transactions |
core/plugin_manager.py, core/installed_plugins.py, core/external_plugins.py | Managed acquisition, selected installed integrations, manual loading |
core/resolve.py, core/recipe.py, core/recipe_overrides.py | Recipe resolution, provenance, effective inputs |
core/scheduler.py, schedulers/, core/layout.py | Rank assignment, occupancy schedulers, explicit layouts |
core/hardware_probe.py, core/hardware_assessment.py, platforms/ | Observed hardware, fit assessment, platform/setup defaults |
core/execution.py, core/asset_preparation.py, core/build_preparation.py | Execution and prepared asset contracts |
core/images.py, core/image_preparation.py | Per-node image selection and prepared container policy |
core/model_distribution.py, core/image_distribution.py | Provider contracts and verified resource preparation |
core/coordination.py, core/ownership.py | Shared reservations and workload ownership |
core/runtime_lease.py, core/process_identity.py | Long-lived runtime leases and process identity |
core/readiness.py, core/timing.py | Readiness observations and launch timing |
orchestration/, transports/ | Executor destinations, host operations, connectivity and collectives |
runtimes/, builders/, benchmarking/ | Serve commands, environment/assets, measurements |
api/setup/, core/setup_plans.py, core/setup_steps.py | Shared probe/plan/apply/undo contracts |
api/model_cache.py, models/, core/runtime_cache.py | Model inventory and separate runtime caches |
proxy/, plugins/ | Gateway contracts and optional/bundled integrations |
feedback.py, telemetry/, diagnostics/ | Operation feedback, opt-out usage reporting, diagnostics |
Layering
Section titled “Layering”CLI / application frontend / authenticated daemon operation | application.initialize + shared context | api.plan -> live RunPlan | api.run(plan=...) | prepared assets -> verified distribution -> executor destination | workload result / logs / readiness observationsThe API does not import the CLI or own a terminal. Frontends own approval and observation policy. Planning retains selected hosts, transport/config, resource inputs, and placement; remote state can still change afterward. A daemon job retains one context/plan within its worker, while warm queries/monitors share bounded services. Independent jobs do not share one mutable operation context.
Plugin system (SAF)
Section titled “Plugin system (SAF)”Sparkrun uses scitrera-app-framework (SAF) for class-based extensions plus public registries for cross-cutting contributions. Bundled, manual, managed, and installed modules share API declaration checks and supported rollback behavior.
| Plugin type | Base class | Internal extension identifier |
|---|---|---|
| Runtime | RuntimePlugin | sparkrun.runtime |
| Builder | BuilderPlugin | sparkrun.builder |
| Benchmark framework | BenchmarkingPlugin | sparkrun.benchmarking |
| Executor | Executor | sparkrun.executor |
| Scheduler | Scheduler | sparkrun.scheduler |
| Transport | Transport | sparkrun.transport |
| Host telemetry | TelemetryProvider | sparkrun.telemetry |
These identifiers are distinct from Python’s shared sparkrun.plugins entry-
point group. Installed integrations target a module and require explicit selection.
Hardware platforms use ordered matching (specific before generic), not selector-
based replacement. Lazy CLI loaders keep initialization free of console/network work.
Conflicting contribution owners fail rather than silently replacing each other.
Bootstrap flow
Section titled “Bootstrap flow”initialize(profile, config_path) -> bind application/config identity -> initialize registry and bundled modules -> load selected installed integrations and enabled external modules -> scan declared classes / register(v) -> retain inventory and supported-registration failure detailsNo ordinary startup fetches plugin updates or performs the new full dependency/ version-range audit. API/feature/profile guards still apply. Explicit plugin management performs compatibility/load checks before activating generations.
Multiplatform architecture
Section titled “Multiplatform architecture”Hardware inventory (core/hardware.py) and probes feed platform matching,
core/backend_select.py, compatibility assessment, and scheduling. Placement uses
RankAssignment in core/scheduler.py and implementations in schedulers/; the
old core/placement.py module is not the current path. Prepared resources and
executor destinations carry per-node execution requirements through staging.
DGX Spark is the primary validated GPU workload target. Generic NVIDIA defaults and other vendor scaffolds are not universal runtime qualification. Linux target requirements remain when a portable controller/daemon runs on macOS or Windows. See Multiplatform and Executors.
Status discovery and ownership
Section titled “Status discovery and ownership”api.status() provides occupancy; api.status_report() enriches it for display.
Executor status scopes describe observation domains, while resolved destinations
identify where mutations actually occur. Failed observations must not be presented
as proof that a destination is empty. api.list_jobs() lists persisted workload
metadata, not daemon operation history.
Native workloads need ownership, verified process/filesystem identity, and recovery state before adoption or teardown. Kubernetes uses an optional plugin with its own destination and controller submission semantics. Cross-controller identity metadata does not automatically broaden existing ownership enforcement.
State boundaries
Section titled “State boundaries”Daemon operation history lives in jobs.sqlite3; daemon and direct mutating
operations share reservations in operations.sqlite3. Workload metadata remains
cache-root YAML, and cluster/config/benchmark state retains existing YAML/JSON
storage. Plugin generations live beside config, outside the Python environment.
See Daemon,
Model Cache Management, and
Execution Flow.