GithubPull

GitHub App demo: ingest repository snapshots and incremental file exports for Search/RAG workflows.

Prerequisites

• Python 3.11+
• git on your PATH
• ngrok (for receiving GitHub webhooks locally)
• A GitHub account and an existing repository to use as the demo target

Local setup

python -m venv .venv
source .venv/bin/activate   # Windows: .venv\Scripts\activate
pip install -r requirements.txt
cp .env.example .env

Edit .env with your app credentials and target repo (see Configuration).

Run the service

uvicorn app.main:app --reload --port 8000

Verify:

curl http://localhost:8000/health
# {"status":"ok"}

Interactive API docs: http://localhost:8000/docs

---

GitHub App setup (operator runbook)

1. Create the GitHub App

1. Open New GitHub App (personal account).
2. GitHub App name: e.g. GithubPull Demo (must be unique on GitHub).
3. Homepage URL: http://localhost:8000 (placeholder is fine for a demo).
4. Webhook
   • Active: checked
   • Webhook URL: https://<your-ngrok-subdomain>.ngrok-free.app/webhook (set after ngrok is running; see below)
   • Webhook secret: generate a random string → copy into .env as WEBHOOK_SECRET
5. Permissions → Repository permissions
   • Contents: Read-only
   • Metadata: Read-only (often auto-selected)
   • Pull requests: Read-only
6. Subscribe to events
   • Installation
   • Push
   • Pull request
7. Where can this GitHub App be installed? Only on this account (enough for a single-repo demo).
8. Click Create GitHub App.

2. Generate credentials

1. On the app settings page, note the App ID → APP_ID in .env.
2. Generate a private key → downloads a .pem file → save as e.g. ./github-app.pem (already in .gitignore).
3. Set PRIVATE_KEY_PATH=./github-app.pem in .env.

3. Configure the demo repository

In .env:

REPO_OWNER=your-github-username
REPO_NAME=your-existing-repo

Only webhooks for this repository are processed; all others return 200 and are logged as ignored.

4. Expose localhost with ngrok

In a second terminal:

ngrok http 8000

Copy the HTTPS forwarding URL (e.g. https://abc123.ngrok-free.app) and set the app Webhook URL to:

https://abc123.ngrok-free.app/webhook

Save the app settings. GitHub sends a ping event; your server should log webhook ping ok.

Note: Free ngrok URLs change each session unless you use a reserved domain. Update the GitHub App webhook URL whenever ngrok restarts.

5. Install the app on your repo

1. App settings → Install App → install on your account.
2. Select Only select repositories → choose the repo matching REPO_OWNER/REPO_NAME.
3. Confirm install.

6. Test webhook delivery

1. Ensure uvicorn is running and ngrok is forwarding to port 8000.
2. GitHub App settings → Advanced → Recent Deliveries.
3. Open a delivery → Redeliver and check your terminal logs.

Expected logs:

• ping → webhook ping ok
• Events for other repos → ignored event=...
• Events for your repo → accepted event=... and installation token minted... (if credentials are set)

---

Configuration

Variable	Description
APP_ID	GitHub App ID
PRIVATE_KEY_PATH	Path to the app private key .pem
WEBHOOK_SECRET	Webhook secret from GitHub App settings
REPO_OWNER	Allowlisted repository owner
REPO_NAME	Allowlisted repository name

---

API routes

Method	Path	Description
GET	/health	Liveness check
POST	/webhook	GitHub webhook receiver (signature required)
POST	/debug/snapshot?installation_id=	Manual full snapshot (no webhook)

Manual snapshot example (find installation ID in the app’s install URL or webhook payload):

# Query param or INSTALLATION_ID in .env
curl -X POST "http://localhost:8000/debug/snapshot?installation_id=12345678"

Tests

pip install -r requirements.txt
pytest

Data layout

data/{owner}__{repo}/
  git/                         # bare clone
  state.json                   # default_branch, last_exported_sha
  snapshots/{sha}/files/       # full snapshot (install)
  updates/{sha}/files/         # push incremental export
  prs/{number}/{head_sha}/files/

---

Implementation plan

Issue	Feature
#1	Bootstrap service
#2	Operator runbook (this README)
#3	Webhook ingress ✓
#4	Installation snapshot ✓
#5	Push incremental export ✓
#6	PR export ✓
#7	Debug snapshot endpoint ✓

second pr export test line