Safe Deepseek Setup Guide

This guide will walk you through the process of setting up and running the Safe Deepseek environment. Follow these steps in order to ensure proper configuration.

Table of Contents

• Model Download Guide
• Configuration Guide
  • Docker Setup
  • Podman Setup

Prerequisites

Container Runtime

You'll need either Docker or Podman installed on your system:

Docker Setup

• Docker Engine Installation Guide
• Docker Compose Installation Guide

Podman Setup

• Podman Installation Guide
• Podman Compose Installation Guide

NVIDIA GPU Requirements

Important: Safe Deepseek requires NVIDIA GPU support. Follow these steps carefully:

1. Install NVIDIA GPU Driver

   • Install using your distribution's package manager (recommended)
   • Or download from NVIDIA Official Drivers
   • Verify driver installation:

     nvidia-smi

     This should display your GPU information

2. Install NVIDIA Container Toolkit

   Choose your package manager:

   For Ubuntu/Debian (apt):

   # Setup the repository and GPG key
   curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
     && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
       sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
       sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
   
   # Update package listing
   sudo apt-get update
   
   # Install the toolkit
   sudo apt-get install -y nvidia-container-toolkit

   For RHEL/CentOS/Rocky (yum/dnf):

   # Setup the repository
   curl -s -L https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo | \
     sudo tee /etc/yum.repos.d/nvidia-container-toolkit.repo
   
   # Install the toolkit
   sudo yum install -y nvidia-container-toolkit

3. Configure Container Runtime

   For Docker:

   # Configure the runtime
   sudo nvidia-ctk runtime configure --runtime=docker
   # Restart Docker daemon
   sudo systemctl restart docker

   For Podman:

   • Podman uses CDI (Container Device Interface) for NVIDIA GPU access
   • No additional configuration required

Without proper NVIDIA GPU setup, the container will not function correctly and will fallback to CPU-only mode. For troubleshooting and advanced configurations, refer to the official documentation.

Quick Start

Before you begin, please read our Contributing Guidelines if you plan to modify the code or contribute to the project.

1. Download Models

First, you'll need to download the required models using our downloader tool. For detailed instructions, see our Model Download Guide.

Quick steps:

1. Navigate to the downloader directory
2. Choose the appropriate executable for your system:
   • Linux: ollama-dl-linux-amd64
   • macOS Intel: ollama-dl-darwin-amd64
   • macOS M1/M2: ollama-dl-darwin-arm64
   • Windows: ollama-dl-windows-amd64.exe
3. Make it executable (Linux/macOS):

   chmod +x ollama-dl-*

4. Run the downloader:

   ./ollama-dl-* pull deepseek-r1 ./volume/models

2. Setup Volume Permissions

Run the setup script to configure proper permissions for the model volume:

./setup

This script ensures that the container has appropriate read-only access to the models stored in the volume directory.

3. Configure Your Runtime Environment

Note: Ensure you have completed the NVIDIA GPU setup before proceeding. The container requires GPU access to function properly.

Choose your preferred container runtime (Docker or Podman) and follow these steps:

1. Navigate to the corresponding configuration directory:

   • For Docker: config/docker/
   • For Podman: config/podman/

2. Copy the runtime script to your project root:

   # For Docker
   cp config/docker/docker-runtime ./
   # OR for Podman
   cp config/podman/podman-runtime ./

3. Make the script executable:

   chmod +x *-runtime

For detailed configuration options, see our Configuration Guide.

4. Run the Container

Start the container using your chosen runtime script:

# For Docker
./docker-runtime
# OR for Podman
./podman-runtime

For detailed setup instructions:

• Docker: See our Docker Setup Guide
• Podman: See our Podman Setup Guide

4. Test the Installation

Verify the setup by running a test prompt:

# For Podman
podman exec -it <container-name> ollama run modelname "Your test prompt"

# For Docker
docker exec -it <container-name> ollama run modelname "Your test prompt"

API Access

The service is accessible via HTTP at:

http://localhost:11434/api/tags

You can integrate this endpoint with:

• IDE extensions
• Applications
• OpenWebUI
• Other compatible tools

Additional Notes

GPU Support (WSL 2.0 / Ubuntu)

For NVIDIA GPU support, install the NVIDIA Container Toolkit:

• Installation Guide

Credits

• Golang Parallel Download
• Ollama Model Direct Download