Luigi is a Python (2.7, 3.6, 3.7 tested) package that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization, handling failures, command line integration, and much more.
Run pip install luigi-k8s-jobs-runner to install the latest stable version from PyPI. Documentation for the latest release is hosted on readthedocs.
Run pip install luigi-k8s-jobs-runner[toml] to install Luigi with TOML-based configs support.