sync-gitlab-github

Bidirectional mirroring between GitHub and GitLab repositories.

How it works

This project provides two mirroring mechanisms with different strengths.

Mechanism	How it syncs	Latency	Scope	Initial sync	Divergence detection
Daemon (cron)	Compares all refs, pushes what's behind	Minutes	All branches + tags	Yes	Yes (halt + log)
CI jobs (bootstrap)	Mirrors each push event	Seconds	The pushed ref only	No	No (just fails)

The daemon is fully self-sufficient — it handles initial sync, ongoing sync, and divergence detection for all branches and tags.

CI is a fast-path optimisation — it gives near-real-time sync on push events, but it can't perform initial sync of pre-existing content and has no retry or divergence detection. CI needs the daemon (or a manual one-off daemon run) to handle the first sync.

Recommended setup: Use both. Run the daemon for initial sync and as a safety net. Use CI for low-latency mirroring of day-to-day pushes.

Consistency model

Single-writer-at-a-time: if both sides receive different commits on the same ref, the daemon halts that repo pair and logs a divergence error. No automatic merges. Manual repair required.

Project structure

repos.yaml                         Repo pair configuration
mirror_daemon.py                   Cron daemon (polls and syncs all refs)
bootstrap.py                       Setup tool (repos, CI files, secrets)
ci-templates/
  github-mirror.yml                GitHub Action template
  gitlab-mirror.yml                GitLab CI job template
tests/
  test_mirror_daemon.py            Daemon integration tests
  test_bootstrap.py                Bootstrap unit tests
pyproject.toml                     Project config (uv)
Makefile                           Developer convenience targets

Option A: Daemon only

The daemon is a standalone Python script that syncs all branches and tags for every repo pair in repos.yaml. It handles everything: initial sync of pre-existing content, ongoing sync, new branches/tags, and divergence detection. No CI configuration needed in the mirrored repos.

Prerequisites

• Python 3.10+ with uv
• GITHUB_TOKEN (PAT with repo scope)
• GITLAB_TOKEN (PAT with write_repository scope)

Configure repo pairs

Edit repos.yaml:

repos:
  - github: myorg/repo1
    gitlab: myorg/repo1
  - github: myorg/repo2
    gitlab: myorg/repo2

Deploy

sudo mkdir -p /var/tmp/mirror-cache
sudo cp mirror_daemon.py repos.yaml /var/tmp/

Add a cron entry:

*/5 * * * * GITHUB_TOKEN=ghp_... GITLAB_TOKEN=glpat-... cd /var/tmp && /usr/bin/python3 mirror_daemon.py >> /var/log/mirror-daemon.log 2>&1

Or run it once manually:

export GITHUB_TOKEN="ghp_..."
export GITLAB_TOKEN="glpat-..."
make daemon

Option B: Daemon + CI (recommended)

Use the daemon for initial sync and integrity, plus CI for near-real-time mirroring on every push.

Prerequisites

• Everything from Option A, plus:
• gh CLI (authenticated)
• glab CLI (authenticated)

Setup

1. Set environment variables:

export GITHUB_TOKEN="ghp_..."          # GitHub PAT with repo scope
export GITLAB_TOKEN="glpat-..."        # GitLab PAT with write_repository scope
export MIRROR_BOT_USER_GITHUB="..."    # GitHub username of the mirror bot
export MIRROR_BOT_USER_GITLAB="..."    # GitLab username of the mirror bot

2. Run bootstrap to install CI workflows and set secrets:

make install
make bootstrap

This will, for each repo pair in repos.yaml:

1. Create repos on both platforms if they don't exist

2. Set secrets/variables on GitHub (GITLAB_TOKEN, GITLAB_REPO, MIRROR_BOT_USER)

3. Set CI variables on GitLab (GITHUB_TOKEN, GITHUB_REPO, MIRROR_BOT_USER)

4. Copy CI workflow files into each repo (the push triggers the first CI mirror)

5. Deploy the daemon (see Option A) as the safety net.

The daemon catches anything CI misses (failed jobs, network errors, token expiry) and provides divergence detection.

Loop prevention

Only relevant when using CI.

Bot usernames are configurable via MIRROR_BOT_USER CI variables on each platform. When GitHub CI pushes to GitLab, GitLab's pipeline skips if the pusher matches the bot user. Same in reverse.

Use dedicated bot accounts whose tokens are used for mirroring to keep the filter reliable.

Deletion policy

Branch and tag deletions are not propagated. CI pushes only the triggered ref. The daemon only pushes refs that exist; it never prunes.

Divergence handling

If the daemon detects that both sides have different commits on the same ref with no ancestor relationship, it:

1. Logs an error identifying the diverged ref(s)
2. Halts all syncing for that repo pair
3. Exits with a non-zero status

To recover: manually force-push one side to match the other, then let the daemon run again.

Note: CI alone does not detect divergence — a non-fast-forward push simply fails in the CI log. Use the daemon for proper divergence detection.

Development

make install   # install dependencies
make test      # run tests
make lint      # syntax check

Makefile targets

Target	Description
install	Install project + dev dependencies
test	Run pytest
lint	Syntax check Python files
daemon	Run the mirror daemon locally
bootstrap	Run the bootstrap tool
clean	Remove .venv and caches