Skip to content

README.md

Latest commit

 

History

History
90 lines (66 loc) · 3.04 KB

README.md

File metadata and controls

90 lines (66 loc) · 3.04 KB

Pillar Service

简体中文

Service Architecture Diagram.png

Project Overview

Pillar Service is a multifunctional service platform based on FastAPI, supporting image caption generation, text translation, and more. It is suitable for rapid integration and extension of AI inference services.

Key Features

  • FastAPI-based, fast response, and easy to extend
  • Supports image captioning (JoyCaption) and text translation
  • Model cache and memory management
  • Detailed health check and logging

Environment Requirements

  • Python 3.11
  • Conda (Anaconda or Miniconda recommended)
  • NVIDIA GPU and driver (for GPU acceleration)
  • Dependencies listed in requirements.txt

Installation & Running

# 1. Create and activate conda environment
conda create -n pillar_env python=3.11
conda activate pillar_env

# 2. Configure Tsinghua conda-forge mirror (recommended for China)
conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge
conda config --set show_channel_urls yes

# 3. Install CUDA dependencies
conda install -y cudatoolkit cudnn 

# 4. Install PyTorch (example: CUDA 12.4)
conda install -y pytorch torchvision torchaudio pytorch-cuda=12.4 -c pytorch -c nvidia

# 5. Install Python dependencies
pip install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple

# 6. Install bitsandbytes and accelerate
pip install bitsandbytes --prefer-binary -i https://pypi.tuna.tsinghua.edu.cn/simple
pip install accelerate -i https://pypi.tuna.tsinghua.edu.cn/simple

# 7. Check CUDA availability
python -c "import torch; print('[INFO] torch.cuda.is_available():', torch.cuda.is_available())"

# 8. Start the service
run_server.bat

Main Directory Structure

Pillar_Service/
├── server/
│   ├── main.py              # FastAPI entry point
│   ├── api/                 # API routes and service management
│   └── ...
├── Pillar/
│   ├── dto/                 # Data Transfer Objects (DTO)
│   ├── service/             # Business service implementations
│   └── util/                # Utilities and constants
├── requirements.txt         # Python dependencies
├── run_server.bat           # Startup script (Windows)
└── README.md                # Project documentation

API Overview

  • /translate
    Text translation API, POST request, parameter: text.
  • /joycaption/generate
    Image caption generation API, POST request, parameters: image and related generation options.
  • /health
    Health check API, GET request.

For detailed parameters and return formats, please refer to code comments or API documentation.

FAQ & Support

  • Slow or failed dependency installation: Use Tsinghua, USTC, or other China mirrors for pip/conda.
  • bitsandbytes device not supported: Make sure you have an NVIDIA GPU, CUDA driver, and PyTorch CUDA version.
  • For other issues, please refer to the documentation or submit an issue.

Feel free to open an issue or contact us if you have any questions!