Skip to content

Add web dashboard for managing data transfer jobs#3

Open
fluidnumerics-joe wants to merge 1 commit intomainfrom
feature/dashboard
Open

Add web dashboard for managing data transfer jobs#3
fluidnumerics-joe wants to merge 1 commit intomainfrom
feature/dashboard

Conversation

@fluidnumerics-joe
Copy link
Member

Summary

  • Adds a FastAPI-based web dashboard for creating and monitoring xfer data transfer jobs
  • Implements Google Workspace SSO authentication with configurable domain restrictions
  • Admin-controlled settings (shards, concurrency, partitions, resources) separate from user inputs
  • Two-stage job submission keeps manifest creation off the login node (runs as Slurm batch job)
  • Real-time progress monitoring via state directory polling
  • Log streaming via Server-Sent Events
  • Job cancellation support

New Files

  • src/xfer/dashboard/ - Dashboard package (FastAPI app, services, templates)
  • docs/dashboard.md - Comprehensive documentation
  • dashboard.yaml.example - Example configuration file

Installation

pip install -e ".[dashboard]"

Test plan

  • Install with dashboard extras and verify dependencies resolve
  • Create dashboard.yaml from example and configure Google OAuth
  • Run xfer-dashboard and verify server starts
  • Test Google SSO login flow with allowed domain
  • Test domain restriction (user from non-allowed domain should be rejected)
  • Create a test transfer job and verify manifest Slurm job is submitted
  • Monitor job progress and verify shard completion updates
  • Test log streaming for manifest and shard logs
  • Test job cancellation

🤖 Generated with Claude Code

Implements a FastAPI-based web interface that allows users to create
and monitor xfer data transfer jobs via browser. Key features:

- Google Workspace SSO authentication with domain restrictions
- Admin-controlled settings (shards, concurrency, partitions, resources)
- User-focused job creation (source, destination, tag only)
- Two-stage job submission (manifest creation as Slurm batch job)
- Real-time progress monitoring via state directory polling
- Log streaming via Server-Sent Events
- Job cancellation support

Includes comprehensive documentation at docs/dashboard.md.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant