Skip to content

feature: cache installed dependancies to speed up build time #11

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
varit05 merged 2 commits into from
May 5, 2026
Merged

feature: cache installed dependancies to speed up build time #11

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
71ffbed
feature: cache installed dependancies to speed up build time
May 5, 2026
abfb16c
add nltk package as dependancies
May 5, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 13 additions & 6 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,20 +25,27 @@ jobs:
uses: actions/setup-python@v5
with:
python-version: "3.14"
cache: 'pip'

- name: Cache pip dependencies
- name: Cache full virtual environment
uses: actions/cache@v4
id: pip-cache
id: cache-venv
with:
path: ~/.cache/pip
key: ${{ runner.os }}-pip-${{ hashFiles('pyproject.toml', 'requirements*.txt') }}
path: .venv
key: ${{ runner.os }}-py3.14-venv-${{ hashFiles('pyproject.toml') }}
restore-keys: |
${{ runner.os }}-pip-
${{ runner.os }}-py3.14-venv-
Comment on lines 25 to +37

@coderabbitai coderabbitai Bot May 5, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

cat -n .github/workflows/ci.yml | sed -n '20,42p'

Repository: varit05/Streaming-Video-RAG

Length of output: 859

🏁 Script executed:

cat -n .github/workflows/ci.yml | sed -n '39,50p'

Repository: varit05/Streaming-Video-RAG

Length of output: 560

🌐 Web query:

actions/setup-python@v5 outputs python-version

💡 Result:

The actions/setup-python@v5 action provides a python-version output that returns the precise installed Python (or PyPy/GraalPy) version as a string, particularly useful when the python-version input is a range (e.g., "3.9.0 - 3.12.0" resolves to e.g. "3.12.0") [1][2]. Other outputs include python-path (absolute path to the executable) and cache-hit (boolean for cache restoration) [1][2]. Example usage: - uses: actions/setup-python@v5 id: python-setup with: python-version: '3.x' - run: echo "Python version: ${{ steps.python-setup.outputs.python-version }}" This echoes the exact version installed, such as "3.12.3" [1].

Citations:

Use dynamic Python version in .venv cache key to prevent incompatible virtualenv restoration.

On lines 35 and 37, the hardcoded py3.14 in the cache key and restore-keys fallback creates a maintenance burden and can restore incompatible virtualenv caches if the Python version changes. Use steps.setup-python.outputs.python-version instead, which is provided by actions/setup-python@v5 and contains the exact installed version.

Additionally, adding id: setup-python to the setup-python step (line 25) is required to access its outputs.

Suggested patch 
       - name: Set up Python
         uses: actions/setup-python@v5
+        id: setup-python
         with:
           python-version: "3.14"
           cache: 'pip'
@@
       - name: Cache full virtual environment
         uses: actions/cache@v4
         id: cache-venv
         with:
           path: .venv
-          key: ${{ runner.os }}-py3.14-venv-${{ hashFiles('pyproject.toml') }}
+          key: ${{ runner.os }}-py${{ steps.setup-python.outputs.python-version }}-venv-${{ hashFiles('pyproject.toml') }}
           restore-keys: |
-            ${{ runner.os }}-py3.14-venv-
+            ${{ runner.os }}-py${{ steps.setup-python.outputs.python-version }}-venv-
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/ci.yml around lines 25 - 37, The cache key uses a
hardcoded "py3.14" which can restore an incompatible virtualenv; add id:
setup-python to the actions/setup-python@v5 step and update the cache key and
restore-keys to reference the actual installed version via
steps.setup-python.outputs.python-version (e.g. replace "py3.14" with something
like "py-${{ steps.setup-python.outputs.python-version }}-venv") so the .venv
cache is version-specific and automatically follows the configured Python
install.


- name: Install dependencies
if: steps.cache-venv.outputs.cache-hit != 'true'
run: |
python -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
pip install ".[dev]"
pip install --no-cache-dir ".[dev]"

- name: Activate virtual environment
run: echo "$GITHUB_WORKSPACE/.venv/bin" >> $GITHUB_PATH

- name: Run Ruff linter
run: ruff check .
Expand Down
18 changes: 13 additions & 5 deletions .github/workflows/docker-publish.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,19 +20,27 @@ jobs:
uses: actions/setup-python@v5
with:
python-version: "3.14"
cache: 'pip'

- name: Cache pip dependencies
- name: Cache full virtual environment
uses: actions/cache@v4
id: cache-venv
with:
path: ~/.cache/pip
key: ${{ runner.os }}-pip-${{ hashFiles('pyproject.toml', 'requirements*.txt') }}
path: .venv
key: ${{ runner.os }}-py3.14-venv-${{ hashFiles('pyproject.toml') }}
restore-keys: |
${{ runner.os }}-pip-
${{ runner.os }}-py3.14-venv-

Comment on lines 20 to 33

@coderabbitai coderabbitai Bot May 5, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

# First, check if the file exists and examine lines 20-33
if [ -f .github/workflows/docker-publish.yml ]; then
  echo "=== docker-publish.yml lines 20-33 ==="
  sed -n '20,33p' .github/workflows/docker-publish.yml
  echo ""
  echo "=== Full file for context ==="
  cat -n .github/workflows/docker-publish.yml
else
  echo "File not found: .github/workflows/docker-publish.yml"
fi

Repository: varit05/Streaming-Video-RAG

Length of output: 3787

🏁 Script executed:

# Check if ci.yml exists and has similar caching patterns
if [ -f .github/workflows/ci.yml ]; then
  echo "=== Searching for cache patterns in ci.yml ==="
  grep -n "cache.*py3\|restore-keys\|.venv" .github/workflows/ci.yml || echo "No matches found"
else
  echo "ci.yml not found"
fi

Repository: varit05/Streaming-Video-RAG

Length of output: 525

🌐 Web query:

actions/setup-python v5 outputs python-version

💡 Result:

The actions/setup-python action (including v5) exposes an output named python-version. You can read it in a workflow like this: steps: - uses: actions/setup-python@v5 id: setup with: python-version: "3.9.0 - 3.12.0" - run: echo ${{ steps.setup.outputs.python-version }} The output is defined as “The installed Python or PyPy version. Useful when given a version range as input.” [1][2] Related outputs you may also see are python-path (absolute path to the Python/PyPy executable) and cache-hit (boolean). [1][2]

Citations:

Harden .venv cache key to use exact interpreter version; avoid partial virtualenv restores.

The cache key on line 30 hardcodes py3.14 as a literal string, and the restore-keys on lines 31–32 allows partial restoration. If the runner's Python patch version changes (e.g., 3.14.0 → 3.14.1), a partially restored .venv may contain packages/bytecode compiled for an incompatible interpreter, causing flaky CI failures.

Use the dynamic python-version output from actions/setup-python@v5 instead. Add id: setup-python to the setup step and reference ${{ steps.setup-python.outputs.python-version }} in the cache key. This ensures the cache key updates automatically when the actual installed Python version changes and prevents partial mismatches.

The same issue exists in .github/workflows/ci.yml (lines 35–37) and requires the same fix.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/docker-publish.yml around lines 20 - 33, The cache key
hardcodes "py3.14" and allows partial restores which can mismatch the actual
interpreter; update the actions/setup-python@v5 step by adding id: setup-python
(or ensure it already has that id) so you can reference the real installed
interpreter via steps.setup-python.outputs.python-version, then replace the
literal "py3.14" in the cache key and restore-keys with a dynamic value like ${{
runner.os }}-py${{ steps.setup-python.outputs.python-version }}-venv-${{
hashFiles('pyproject.toml') }} (and tighten or remove loose restore-keys to
avoid partial restores); apply the identical change to the CI workflow (ci.yml)
where the same cache key pattern is used.

- name: Install dependencies
if: steps.cache-venv.outputs.cache-hit != 'true'
run: |
python -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
pip install ".[dev]"
pip install --no-cache-dir ".[dev]"

- name: Activate virtual environment
run: echo "$GITHUB_WORKSPACE/.venv/bin" >> $GITHUB_PATH

- name: Run Ruff linter
run: ruff check .
Expand Down
1 change: 1 addition & 0 deletions pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ dependencies = [

# RAG
"tiktoken>=0.7",
"nltk>=3.8.1",

# Storage
"sqlalchemy>=2.0",
Expand Down
Loading