From af73101a66cee56a1a9e894cff0307a37eeb1bfb Mon Sep 17 00:00:00 2001 From: Jackson Falgoust Date: Thu, 25 Jun 2026 08:11:44 -0400 Subject: [PATCH 1/4] Feature/installer mount (#49) * Adds installer scripts and configuration * Remove installer directory * changes installer to zip file * removes installer folder * updates installer to work with empty .installer_state.env file * Driver check for cuda and rocm. * Adds --mount flag to mount folder to project without rebuilding * Makes sure folder is mounted before opening localhost * test run of installers * changes compose files for arm64 builds * Stores login token for mount authentication * Mount to notbook within project, and unmount * Add CliAuthSession entity, DbSet, model config, and EF migration Persistence layer for the new CLI device-code authorization flow. Adds CliAuthSession entity with SessionId (PK), DeviceSecretHash, Status enum (Pending/Approved/Consumed), nullable UserId FK to User, CreatedAt, and ExpiresAt. Includes ExpiresAt index for cleanup queries and cascade delete from User. Co-Authored-By: Claude Opus 4.8 * Add optional lifetimeOverride to JwtTokenService.IssueToken Support caller-specified JWT lifetimes for the CLI device-code flow, which needs short-lived (~10 min) tokens. When lifetimeOverride is null, behavior is unchanged (uses configured LifetimeMinutes). Non-positive overrides throw ArgumentOutOfRangeException. Co-Authored-By: Claude Opus 4.8 * Add CliAuthService for secure CLI device-code auth flow Implement ICliAuthService with CreateSessionAsync, ApproveAsync, and IssueTokenAsync driving the CLI device-code authorization flow. The device secret is hashed (SHA256) before storage and verified with constant-time comparison before any status branching, preventing session-ID-only callers from learning approval state. Tokens are single-use (Consumed on issuance). Includes DI registration and 14 unit tests covering creation, cleanup, approval, secret verification, token issuance, single-use enforcement, and JWT claim validation. Co-Authored-By: Claude Opus 4.8 * Add CliAuth HTTP endpoints and remove insecure on-disk token write Expose ICliAuthService over three endpoints (POST /api/cli/sessions, POST /{sessionId}/approve, GET /{sessionId}/token) for the browser device-code approval flow. Remove the .cli-auth-token file write and unused IStoragePathResolver parameter from the login handler. Add nine integration tests covering the full happy path and security negatives. Co-Authored-By: Claude Opus 4.8 * Add CliAuthorize page, protected route, and api.ts approve call Implements the browser-side CLI authorization flow: a new CliAuthorize page where logged-in users approve command-line mount requests, a protected route at /cli/authorize, and the api.cli.approveSession() method. Includes component tests covering all states (missing session, approve success, 404/410 errors, generic errors, deny). Co-Authored-By: Claude Opus 4.8 * Replace installer on-disk-token auth with browser device-code flow Rewrites acquire_token() to use the POST /api/cli/sessions + browser-approval + polling pattern instead of reading a token file from disk. The token is held only in a shell variable (AUTH_TOKEN) and never written to disk. Also generalizes open_browser() to accept an optional URL argument, removes the CLI_TOKEN_FILE variable, and adds .gitignore entries for the token file. Co-Authored-By: Claude Opus 4.8 * Require admin to approve CLI auth sessions Mount creation requires admin, so allowing any approved user to approve a CLI session led to a 403 dead-end after the browser approval. Tighten the /api/cli/sessions/{id}/approve endpoint to RequireAdmin and add a non-admin -> 403 integration test. Co-Authored-By: Claude Opus 4.8 * Doc for secure CLI authentication and authorization window close after approval * Mount folder to project root. * fixes on windows * Fix windows bugs * update README.md * updates installer.zip * tests * update installer.zip --------- Co-authored-by: Claude Opus 4.8 --- .gitignore | 4 + docker/build/guideants-ai/Dockerfile.cpu | 2 + ...ocker-compose.cpu.api-only-local-build.yml | 1 + docker/docker-compose.cpu.yml | 1 + docker/docker-compose.ghcr-cpu.yml | 1 - docker/docker-compose.ghcr-slim.yml | 1 - docker/docker-compose.slim.yml | 1 + docs/project-root-host-folder-mounts.md | 257 ++ docs/secure-cli-auth.md | 267 ++ installer.zip | Bin 0 -> 77197 bytes installer/README.md | 232 ++ installer/docker/.env | 30 + installer/docker/docker-compose.cpu.yml | 305 ++ installer/docker/docker-compose.cuda.yml | 337 ++ installer/docker/docker-compose.ghcr-cpu.yml | 253 ++ .../docker/docker-compose.ghcr-cuda13.yml | 275 ++ installer/docker/docker-compose.ghcr-rocm.yml | 254 ++ installer/docker/docker-compose.ghcr-slim.yml | 151 + .../docker-compose.host-mounts.generated.yml | 130 + installer/docker/docker-compose.rocm.yml | 311 ++ installer/docker/docker-compose.slim.yml | 181 + .../docker/volumes/content-files/.gitkeep | 0 .../volumes/searxng/config/limiter.toml | 3 + .../volumes/searxng/config/settings.yml | 409 ++ .../volumes/searxng/config/settings.yml.new | 2800 +++++++++++++ .../docker/volumes/searxng/data/.gitkeep | 0 installer/guideants.sh | 1139 +++++ installer/scripts/guideants-host-mount.ps1 | 403 ++ installer/scripts/guideants-host-mount.sh | 444 ++ installer/stop_guideants.sh | 75 + scripts/guideants-host-mount.sh | 0 src/client/src/components/AppContent.tsx | 2 + .../hostMounts/MapHostFolderDialog.tsx | 30 +- .../notebook/sidebar/NotebookFolderTree.tsx | 1 + .../components/project/sidebar/FolderTree.tsx | 81 +- .../project/sidebar/ProjectSidebar.tsx | 80 +- src/client/src/hooks/useProjectHostMounts.ts | 66 + src/client/src/pages/CliAuthorize.tsx | 131 + .../src/pages/__tests__/CliAuthorize.test.tsx | 132 + src/client/src/services/api.ts | 6 + src/client/src/types/hostFolderMount.ts | 8 + src/client/src/types/project.ts | 4 + src/client/src/utils/hostMountDisplayState.ts | 28 +- .../ApplicationDbContext.cs | 10 + ...260623200101_AddCliAuthSession.Designer.cs | 3702 +++++++++++++++++ .../20260623200101_AddCliAuthSession.cs | 54 + .../ApplicationDbContextModelSnapshot.cs | 47 +- .../Models/CliAuthSession.cs | 31 + .../Models/CliAuthSessionStatus.cs | 9 + .../GuideAntsApi.DataModel/Models/User.cs | 1 + .../Endpoints/CliAuthEndpointsTests.cs | 309 ++ .../Services/Auth/CliAuthServiceTests.cs | 361 ++ .../Services/Auth/JwtTokenServiceTests.cs | 119 + .../Configuration/StartupConfiguration.cs | 1 + .../GuideAntsApi/Endpoints/AuthEndpoints.cs | 1 + .../Endpoints/CliAuthEndpoints.cs | 121 + .../GuideAntsApi/Models/ProjectFolderDtos.cs | 8 +- src/server/GuideAntsApi/Program.cs | 1 + .../Services/Auth/CliAuthService.cs | 260 ++ .../Services/Auth/JwtTokenService.cs | 11 +- .../Components/HostMountDirectoryScanner.cs | 132 + .../Components/NotebookFileService.cs | 126 +- .../Components/ProjectFolderService.cs | 216 +- 63 files changed, 14210 insertions(+), 146 deletions(-) create mode 100644 docs/project-root-host-folder-mounts.md create mode 100644 docs/secure-cli-auth.md create mode 100644 installer.zip create mode 100644 installer/README.md create mode 100644 installer/docker/.env create mode 100644 installer/docker/docker-compose.cpu.yml create mode 100644 installer/docker/docker-compose.cuda.yml create mode 100644 installer/docker/docker-compose.ghcr-cpu.yml create mode 100644 installer/docker/docker-compose.ghcr-cuda13.yml create mode 100644 installer/docker/docker-compose.ghcr-rocm.yml create mode 100644 installer/docker/docker-compose.ghcr-slim.yml create mode 100644 installer/docker/docker-compose.host-mounts.generated.yml create mode 100644 installer/docker/docker-compose.rocm.yml create mode 100644 installer/docker/docker-compose.slim.yml create mode 100644 installer/docker/volumes/content-files/.gitkeep create mode 100644 installer/docker/volumes/searxng/config/limiter.toml create mode 100644 installer/docker/volumes/searxng/config/settings.yml create mode 100644 installer/docker/volumes/searxng/config/settings.yml.new create mode 100644 installer/docker/volumes/searxng/data/.gitkeep create mode 100755 installer/guideants.sh create mode 100644 installer/scripts/guideants-host-mount.ps1 create mode 100755 installer/scripts/guideants-host-mount.sh create mode 100755 installer/stop_guideants.sh mode change 100644 => 100755 scripts/guideants-host-mount.sh create mode 100644 src/client/src/hooks/useProjectHostMounts.ts create mode 100644 src/client/src/pages/CliAuthorize.tsx create mode 100644 src/client/src/pages/__tests__/CliAuthorize.test.tsx create mode 100644 src/server/GuideAntsApi.DataModel/Migrations/20260623200101_AddCliAuthSession.Designer.cs create mode 100644 src/server/GuideAntsApi.DataModel/Migrations/20260623200101_AddCliAuthSession.cs create mode 100644 src/server/GuideAntsApi.DataModel/Models/CliAuthSession.cs create mode 100644 src/server/GuideAntsApi.DataModel/Models/CliAuthSessionStatus.cs create mode 100644 src/server/GuideAntsApi.IntegrationTests/Endpoints/CliAuthEndpointsTests.cs create mode 100644 src/server/GuideAntsApi.Tests/Services/Auth/CliAuthServiceTests.cs create mode 100644 src/server/GuideAntsApi.Tests/Services/Auth/JwtTokenServiceTests.cs create mode 100644 src/server/GuideAntsApi/Endpoints/CliAuthEndpoints.cs create mode 100644 src/server/GuideAntsApi/Services/Auth/CliAuthService.cs create mode 100644 src/server/GuideAntsApi/Services/Components/HostMountDirectoryScanner.cs diff --git a/.gitignore b/.gitignore index cf19e217..c80f14a0 100644 --- a/.gitignore +++ b/.gitignore @@ -426,3 +426,7 @@ FodyWeavers.xsd /src/server/docker/volumes/content-files /tmp /scan-results.txt + +# CLI auth token (never commit) +installer/docker/volumes/content-files/.cli-auth-token +*.cli-auth-token diff --git a/docker/build/guideants-ai/Dockerfile.cpu b/docker/build/guideants-ai/Dockerfile.cpu index be7a1753..04a7f10b 100644 --- a/docker/build/guideants-ai/Dockerfile.cpu +++ b/docker/build/guideants-ai/Dockerfile.cpu @@ -143,6 +143,8 @@ COPY requirements.txt /tmp/requirements.txt RUN --mount=type=cache,target=/root/.cache/pip \ pip install --upgrade pip "setuptools<82" wheel \ + && echo "setuptools<82" > /tmp/build-constraints.txt \ + && export PIP_CONSTRAINT=/tmp/build-constraints.txt \ && pip install \ torch==2.11.0 torchaudio==2.11.0 torchvision==0.26.0 \ --index-url https://download.pytorch.org/whl/cpu \ diff --git a/docker/docker-compose.cpu.api-only-local-build.yml b/docker/docker-compose.cpu.api-only-local-build.yml index 97aaca32..2732612b 100644 --- a/docker/docker-compose.cpu.api-only-local-build.yml +++ b/docker/docker-compose.cpu.api-only-local-build.yml @@ -166,6 +166,7 @@ services: documentserver: image: ${GA_DOCUMENTSERVER_IMAGE:-documentserver:latest} + platform: linux/amd64 container_name: documentserver environment: - JWT_ENABLED=${GA_DOCUMENTSERVER_JWT_ENABLED:-false} diff --git a/docker/docker-compose.cpu.yml b/docker/docker-compose.cpu.yml index a7734c26..957cdea0 100644 --- a/docker/docker-compose.cpu.yml +++ b/docker/docker-compose.cpu.yml @@ -171,6 +171,7 @@ services: documentserver: image: ${GA_DOCUMENTSERVER_IMAGE:-documentserver:latest} + platform: linux/amd64 container_name: documentserver environment: - JWT_ENABLED=${GA_DOCUMENTSERVER_JWT_ENABLED:-false} diff --git a/docker/docker-compose.ghcr-cpu.yml b/docker/docker-compose.ghcr-cpu.yml index b48ea5de..0db378ac 100644 --- a/docker/docker-compose.ghcr-cpu.yml +++ b/docker/docker-compose.ghcr-cpu.yml @@ -128,7 +128,6 @@ services: docling-serve: image: ${DOCLING_SERVE_CPU_IMAGE:-quay.io/docling-project/docling-serve-cpu:v1.21.0} - platform: linux/amd64 pull_policy: always container_name: docling-serve environment: diff --git a/docker/docker-compose.ghcr-slim.yml b/docker/docker-compose.ghcr-slim.yml index c84c95d7..e7fd8de9 100644 --- a/docker/docker-compose.ghcr-slim.yml +++ b/docker/docker-compose.ghcr-slim.yml @@ -28,7 +28,6 @@ services: docling-serve: image: ${DOCLING_SERVE_CPU_IMAGE:-quay.io/docling-project/docling-serve-cpu:v1.21.0} - platform: linux/amd64 pull_policy: always container_name: docling-serve environment: diff --git a/docker/docker-compose.slim.yml b/docker/docker-compose.slim.yml index 8b4bb3bd..f5e7902e 100644 --- a/docker/docker-compose.slim.yml +++ b/docker/docker-compose.slim.yml @@ -56,6 +56,7 @@ services: documentserver: image: ${GA_DOCUMENTSERVER_IMAGE:-documentserver:latest} + platform: linux/amd64 container_name: documentserver environment: - JWT_ENABLED=${GA_DOCUMENTSERVER_JWT_ENABLED:-false} diff --git a/docs/project-root-host-folder-mounts.md b/docs/project-root-host-folder-mounts.md new file mode 100644 index 00000000..9b5e1582 --- /dev/null +++ b/docs/project-root-host-folder-mounts.md @@ -0,0 +1,257 @@ +# Project-Root Host Folder Mounts + +This document covers the project-root host folder mount feature. It builds on +the notebook-level mount system documented in +[`host-folder-mounts.md`](./host-folder-mounts.md). + +## What this feature does + +Project-root host folder mounts let an admin mount a folder from the Docker host +so that it appears in two places simultaneously: + +1. **At the root of the project file tree** — browsable, expandable into its + real files and subfolders. +2. **Inside every notebook in the project** — as a symlinked folder at the + notebook root, with full read-write access (existing notebook mount behavior). + +This includes notebooks created or copied *after* the mount was set up. + +Before this feature, project-wide mounting was only reachable through a +notebook's "Map host folder" dialog by switching a scope dropdown to "All +notebooks in project." The result never appeared in the project's own tree. This +feature moves project-wide mounting to the **project screen** and makes the +mounted folder a first-class entry in the project file tree. + +### Notebook dialog simplification + +The notebook "Map host folder" dialog no longer has a scope selector. It always +mounts into the current notebook only. Project-wide mounting is done exclusively +from the project screen. + +## How to use it + +### Mount a host folder to the project + +1. Open a project and **right-click the project root folder** in the sidebar. +2. Select **"Map host folder here"** (visible to admins only). +3. Enter the **absolute host path** of the folder to mount. Optionally set a + custom leaf name (defaults to the last segment of the host path). +4. Click **Create mapping**. A command dialog appears with the generated + `guideants-host-mount.sh apply ...` (or `.ps1`) command. +5. Run the displayed command on the Docker host. This adds bind-mount entries to + `docker-compose.host-mounts.generated.yml` and restarts the affected services. +6. Back in the app, right-click the project root and select **"Check mapped + folders"** (reconcile). The mounted folder now appears at the project root and + can be expanded to browse its contents. The same folder also appears inside + every notebook in the project. + +### Context menu actions + +**On the project root folder (admin):** + +| Action | Description | +|---|---| +| Map host folder here | Opens the mount creation dialog | +| Check mapped folders | Reconciles all project-scope mounts | + +**On a mount root node (admin):** + +| Action | Description | +|---|---| +| Remove mapped folder | Returns the host remove command | +| Show apply command | Re-displays the apply command | +| Show remove command | Re-displays the remove command | +| Check mapped folders | Reconciles this specific mount | + +**On subfolders within a mount:** + +Mount subfolders are browsable (expandable/collapsible) but read-only. The +context menu shows "Read-only (host mount)" — no rename, delete, upload, or +create operations. + +### Mount status badges + +Mount root nodes display a colored status badge: + +| Badge | Meaning | +|---|---| +| **Linked** (green) | Mount is active and working | +| **Pending restart** (amber) | Mount created but host command not yet run | +| **Missing source** (orange) | Container mount path is absent | +| **Link error** (red) | Symlink or mount error | +| **Pending removal** (slate) | Marked for removal, host command not yet run | + +### Remove a project mount + +1. Right-click the mount root in the project tree. +2. Select **Remove mapped folder**. +3. Run the displayed remove command on the host. +4. Reconcile. The mount disappears from both the project tree and all notebooks. + +Host folder contents are never deleted — only symlinks and compose volume entries +are removed. + +### Installer CLI + +The interactive installer (`./installer/guideants.sh`) mount flow offers the +project-wide option as **"Entire project (project root + every notebook)"**. The +API payload is unchanged (`scope: "Project"`). + +## How it works + +### Architecture overview + +The feature reuses the existing project-scope mount machinery. No new mount type, +database table, or API endpoint is introduced. + +``` + HostFolderMount (Scope=Project) + | + +-------------+-------------+ + | | + Project file tree Per-notebook links + (virtual overlay) (HostFolderMountLink per notebook) + | | + Scans ContainerSourcePath Symlink at notebook root + /app/HostMounts/{key} /app/ContentFiles/{proj}/{nb}/{leaf} +``` + +### Backend + +**Mount creation** uses the existing `HostFolderMountService.CreateMountAsync` +with `Scope=Project` and `NotebookId=null`. This creates a `HostFolderMountLink` +for every notebook in the project. No change to this path. + +**New-notebook back-fill** is already implemented. When a notebook is created or +copied after a project mount exists, `ApplyProjectScopedMappingsToNewNotebookAsync` +adds a link for every active project-scope mount. No change to this path. + +**Project tree overlay** is new. `ProjectFolderService.GetFolderTreeAsync` now: + +1. Builds the normal DB-backed folder tree (unchanged). +2. Queries active project-scope mounts (`Scope == Project`, status not + `Removed` or `PendingRemoval`). +3. For each mount, scans `ContainerSourcePath` (`/app/HostMounts/{mountKey}`) + using `HostMountDirectoryScanner` and injects a root-level `FolderTreeDto` + node with children representing the mount's file/folder structure. + +Mount nodes carry metadata: +- `IsHostMount: true` on the mount root +- `MountId` and `MountStatus` on the mount root +- `IsLinked: true` on child folders within the mount +- Unique deterministic IDs (SHA256 of mount ID + path) for stable + expand/collapse state in the UI + +**Shared scanner.** `HostMountDirectoryScanner` is a static helper extracted from +`NotebookFileService.BuildLinkedMountFileDtosAsync`. Both the notebook and +project trees use the same budgeted directory scan with identical constraints: + +| Parameter | Default | Config key | +|---|---|---| +| Max files | 5,000 | `FileStorage:LinkedMountTreeMaxFiles` | +| Max depth | 3 | `FileStorage:LinkedMountTreeMaxDepth` | +| Scan timeout | 2,500 ms | `FileStorage:LinkedMountTreeScanBudgetMs` | + +The scanner filters out temporary script files (`{hash}_script.sh/ps1/py`) and +`__pycache__/` directories. Notebook-specific filters (`Resources/`, +`.guideants/`) are applied in `NotebookFileService` after scanning, not in the +shared scanner. + +**DTO additions.** `FolderTreeDto` gained four optional fields: + +```csharp +public record FolderTreeDto( + ... + bool IsHostMount = false, + Guid? MountId = null, + HostFolderMountStatus? MountStatus = null, + bool IsLinked = false +); +``` + +These mirror the notebook tree DTO's mount flags so the frontend badge/display +utilities work identically. + +### Frontend + +**`MapHostFolderDialog`** is now scope-agnostic. The scope ` setScope(event.target.value as HostFolderMountScope)} - disabled={isSubmitting} - className="w-full rounded border px-3 py-2 text-sm focus:outline-none focus:ring-2 focus:ring-blue-500" - data-testid="host-mount-scope-select" - > - - - - -