Skip to content

Spinner

spinner logo

Spinner is an open-source, parameterized sweep benchmark tool for High-Performance Computing (HPC). It runs user-supplied commands with varying parameters, collecting output and recording metadata in a straightforward, YAML-based configuration.

Key Features

  • Parameterized Sweeps: Define arbitrary parameter sweeps in YAML.
  • Non-Intrusive: Runs your existing scripts or applications unchanged.
  • Re-run and Timeout Handling: Automatically retry failed runs and enforce timeouts.
  • Automatic Data Collection: Captures logs, metrics, and metadata into concise dataframes.
  • YAML Configuration: Reproducible, shareable configs for HPC workloads.

Quick Start

If you prefer to use an isolated environment, you can create a virtual environment using venv or conda before installing.

python3 -m ensurepip
python3 -m pip install virtualenv
python3 -m virtualenv .venv
source .venv/bin/activate
python3 -m pip install pip --upgrade
  1. Install Spinner:
pip install spinner
  1. Create a YAML file defining parameters and commands. For a minimal example:
metadata:
  description: Sleep check
  version: "1.0"
  runs: 2
  timeout: 5
  retry: 1
  envvars: []

applications:
  sleep_test:
    command: >
        sleep {{sleep_amount}} && printf "I slept {{sleep_amount}}"

benchmarks:
  sleep_test:
    sleep_amount:
      - 1
      - 2
  1. Run Spinner:
spinner run sleep_benchmark.yaml

This runs all parameter combinations, retries failures (if configured), and stores the results in results.pkl.

Documentation & Examples

  • Examples: See Examples for a gallery of ready-to-run YAML files showcasing parameter sweeps, capturing output from commands, handling timeouts, and more.
  • SLURM HPC: Refer to Using with SLURM for tips on incorporating Spinner in batch jobs.
  • Contributing: Check out Contribute to Spinner for development guidelines, linting policy, and release automation steps.

Citation

If you use Spinner in your research, please cite it.

@inproceedings{ceccato2024spinner,
  title={Spinner: Enhancing HPC Experimentation with a Streamlined Parameter Sweep Tool},
  author={Ceccato, Rodrigo and Cl{\'e}to, Jhonatan and Leite, Gustavo and Rigo, Sandro and Diaz, Jose M Monsalve and Yviquel, Herv{\'e}},
  booktitle={2024 International Symposium on Computer Architecture and High Performance Computing Workshops (SBAC-PADW)},
  pages={1--11},
  year={2024},
  organization={IEEE}
}

License

This project is licensed under the MIT License. See the LICENSE.md in the repository root for details.