Skip to content

yaml.md

Latest commit

 

History

History
149 lines (99 loc) · 4.07 KB

yaml.md

File metadata and controls

149 lines (99 loc) · 4.07 KB

cog.yaml reference

cog.yaml defines how to build a Docker image and how to run predictions on your model inside that image.

It has three keys: build, image, and predict. It looks a bit like this:

build:
  python_version: "3.11"
  python_packages:
    - pytorch==2.0.1
  system_packages:
    - "ffmpeg"
    - "libavcodec-dev"
predict: "predict.py:Predictor"

Tip: Run cog init to generate an annotated cog.yaml file that can be used as a starting point for setting up your model.

build

This stanza describes how to build the Docker image your model runs in. It contains various options within it:

cuda

Cog automatically picks the correct version of CUDA to install, but this lets you override it for whatever reason.

For example:

build:
  cuda: "11.1"

gpu

Enable GPUs for this model. When enabled, the nvidia-docker base image will be used, and Cog will automatically figure out what versions of CUDA and cuDNN to use based on the version of Python, PyTorch, and Tensorflow that you are using.

For example:

build:
  gpu: true

When you use cog run or cog predict, Cog will automatically pass the --gpus=all flag to Docker. When you run a Docker image built with Cog, you'll need to pass this option to docker run.

python_packages

A list of Python packages to install, in the format package==version. For example:

build:
  python_packages:
    - pillow==8.3.1
    - tensorflow==2.5.0

python_requirements

A pip requirements file specifying the Python packages to install. For example:

build:
  python_requirements: requirements.txt

Your cog.yaml file can set either python_packages or python_requirements, but not both. Use python_requirements when you need to configure options like --extra-index-url or --trusted-host to fetch Python package dependencies.

python_version

The minor (3.11) or patch (3.11.1) version of Python to use. For example:

build:
  python_version: "3.11.1"

Cog supports all active branches of Python: 3.8, 3.9, 3.10, 3.11.

Note that these are the versions supported in the Docker container, not your host machine. You can run any version(s) of Python you wish on your host machine.

run

A list of setup commands to run in the environment after your system packages and Python packages have been installed. If you're familiar with Docker, it's like a RUN instruction in your Dockerfile.

For example:

build:
  run:
    - curl -L https://github.com/cowsay-org/cowsay/archive/refs/tags/v3.7.0.tar.gz | tar -xzf -
    - cd cowsay-3.7.0 && make install

Your code is not available to commands in run. This is so we can build your image efficiently when running locally.

Each command in run can be either a string or a dictionary in the following format:

build:
  run:
    - command: pip install
      mounts:
        - type: secret
          id: pip
          target: /etc/pip.conf

You can use secret mounts to securely pass credentials to setup commands, without baking them into the image. For more information, see Dockerfile reference.

system_packages

A list of Ubuntu APT packages to install. For example:

build:
  system_packages:
    - "ffmpeg"
    - "libavcodec-dev"

image

The name given to built Docker images. If you want to push to a registry, this should also include the registry name.

For example:

image: "r8.im/your-username/your-model"

r8.im is Replicate's registry, but this can be any Docker registry.

If you don't provide this, a name will be generated from the directory name.

predict

The pointer to the Predictor object in your code, which defines how predictions are run on your model.

For example:

predict: "predict.py:Predictor"

See the Python API documentation for more information.