[#20] Live demo guide and runtime screenshot pack

.gitignore

@@ -30,3 +30,6 @@ Thumbs.db
 __pycache__/
 *.py[cod]
 
+# Screenshot generator runtime output
+docs/screenshots/runtime/
+

README.md

@@ -188,15 +188,15 @@ See [**START_HERE.md**](START_HERE.md) for guided navigation.
 - [x] **#12** — Architecture documentation
 - [x] **#13** — CI/CD pipeline (GitHub Actions + SonarQube + Telegram notifications)
 
-### 🚀 Phase 2 — Active/Planned (Issues #14–#20)
+### ✅ Phase 2 — Completed (Issues #14–#20)
 
 - [x] **#14** — Support additional event types (`UserProfileChanged`, `EntityCreated`, `EntityUpdated`, `DataDeleted`)
 - [x] **#15** — Authentication & Authorization (JWT + Spring Security + RBAC)
 - [x] **#16** — Advanced filtering, search, date range picker, CSV export
 - [x] **#17** — Event timeline visualization
-- [ ] **#18** — Reconciliation Service (batch integrity checking + Quartz scheduler)
-- [ ] **#19** — Kubernetes manifests + Helm chart
-- [ ] **#20** — Live demo scenario + Q&A documentation
+- [x] **#18** — Reconciliation Service (batch integrity checking + Quartz scheduler)
+- [x] **#19** — Kubernetes manifests + Helm chart
+- [x] **#20** — Live demo scenario + Q&A documentation
 
 **Full roadmap and issue tracking:** [**docs/ROADMAP.md**](docs/ROADMAP.md)
 
@@ -226,6 +226,7 @@ All documentation is located in the `docs/` directory:
 | [**docs/DEPLOYMENT.md**](docs/DEPLOYMENT.md) | Quickstart + environment setup |
 | [**docs/ROADMAP.md**](docs/ROADMAP.md) | Complete GitHub Issues roadmap (Phase 1–4) |
 | [**docs/TESTING_SCENARIOS.md**](docs/TESTING_SCENARIOS.md) | Live demo scenarios + curl commands + screenshots |
+| [**docs/LIVE_DEMO_QA.md**](docs/LIVE_DEMO_QA.md) | Interview-ready live demo script + architecture Q&A |
 | [**docs/EVENT_HASH_CANONICAL_MIGRATION.md**](docs/EVENT_HASH_CANONICAL_MIGRATION.md) | Event hash canonicalization + recovery procedures |
 | [**CONTRIBUTING.md**](CONTRIBUTING.md) | Contribution workflow, PR guidelines |
 

START_HERE.md

@@ -25,10 +25,13 @@ Use this file as the single entry point for project setup and work planning.
 
 ## 4) Current status
 
-- MVP Phase 1 is complete (Issues #1–#13)
-- Next open implementation item is Issue #14 (additional event types)
+- MVP Phase 1 is complete (Issues #1-#13)
+- Phase 2 items (#14-#20) are complete in roadmap; use docs below for demo preparation and next planning
 
 ## 5) Next action
 
-Confirm local prerequisites from section 1, then start with the next open issue in [`docs/ROADMAP.md`](docs/ROADMAP.md).
+Confirm local prerequisites from section 1, then:
+
+1. Use [`docs/LIVE_DEMO_QA.md`](docs/LIVE_DEMO_QA.md) for interview-ready demo flow.
+2. Use [`docs/ROADMAP.md`](docs/ROADMAP.md) to plan the next issue beyond phase 2.
 

docs/LIVE_DEMO_QA.md

@@ -0,0 +1,238 @@
+# Live Demo Scenario and Q&A (Issue #20)
+
+This document is an interview-ready script for demonstrating the Distributed Audit Ledger end to end.
+
+## Audience and Goal
+
+- Audience: interviewer, tech lead, architect, or security engineer.
+- Goal: prove immutable audit flow from command API to on-chain anchoring and read-side integrity checks.
+- Duration: 12-20 minutes (core path: ~10 minutes, Q&A: ~10 minutes).
+
+## Demo Narrative (What to Say)
+
+1. "I send a command to the command service."
+2. "The command service emits a domain event to Kafka."
+3. "Event store persists immutable JSON payload + canonical SHA-256 hash."
+4. "Audit writer anchors the same hash in the smart contract."
+5. "Query service exposes audit logs and integrity status (`ON_CHAIN`, `MISMATCH`, `PENDING`)."
+
+## Prerequisites
+
+- Docker stack from `deploy/` is running.
+- Backend services are running on ports `8081`-`8084`.
+- `AUDIT_LEDGER_CONTRACT_ADDRESS` is configured for blockchain-aware services.
+- Tools: `curl`, `jq`.
+- Optional (for Steps 5–6): `docker` CLI, `psql` client.
+
+Quick health check:
+
+```bash
+curl http://localhost:8081/actuator/health
+curl http://localhost:8082/actuator/health
+curl http://localhost:8083/actuator/health
+curl http://localhost:8084/actuator/health
+```
+
+## Demo Script (Core Path)
+
+### Step 0: Obtain JWT token
+
+```bash
+TOKEN=$(curl -s -X POST http://localhost:8081/auth/login \
+  -H "Content-Type: application/json" \
+  -d '{"username":"admin","password":"admin123!"}' | jq -r '.accessToken')
+```
+
+### Step 1: Send one command
+
+```bash
+CMD=$(curl -s -X POST http://localhost:8081/commands/user/login \
+  -H "Content-Type: application/json" \
+  -H "Authorization: Bearer $TOKEN" \
+  -d '{"userId":"demo.user@example.com"}')
+echo "$CMD"
+EVENT_ID=$(echo "$CMD" | jq -r '.eventId')
+```
+
+Expected: response contains `success=true` and `eventId`.
+
+### Step 2: Read events from query service
+
+```bash
+curl -s "http://localhost:8084/api/audit-logs?userId=demo.user@example.com&limit=5" \
+  -H "Authorization: Bearer $TOKEN"
+```
+
+Expected:
+
+- At least one event with `eventType=USER_LOGGED_IN`.
+- `eventHash` is present.
+- Note: `integrityStatus` in the list always shows `PENDING` by default. The live on-chain status is resolved by the integrity-check endpoint (Step 4).
+
+### Step 3: Resolve audit ID for integrity check
+
+Use the `eventId` captured in Step 1 to find the exact record:
+
+```bash
+AUDIT_ID=$(curl -s "http://localhost:8084/api/audit-logs?userId=demo.user@example.com&limit=20" \
+  -H "Authorization: Bearer $TOKEN" | jq -r --arg eid "$EVENT_ID" '.[] | select(.eventId == $eid) | .id')
+echo "$AUDIT_ID"
+```
+
+Expected: numeric ID (for example `42`).
+
+### Step 4: Run integrity check
+
+```bash
+curl -s "http://localhost:8084/api/audit-logs/$AUDIT_ID/integrity-check" \
+  -H "Authorization: Bearer $TOKEN"
+```
+
+Expected: `status` eventually becomes `ON_CHAIN`.
+
+If `MISMATCH` appears immediately after command, wait a few seconds and retry (asynchronous anchor timing).
+
+### Step 5: Show Kafka flow proof (optional)
+
+```bash
+docker exec dal-kafka kafka-topics --bootstrap-server localhost:9092 --list
+```
+
+Expected topics include `user.login.events`.
+
+### Step 6: Show DB evidence (optional)
+
+```bash
+psql -h localhost -U postgres -d audit_ledger \
+  -c "SELECT id, event_id, event_type, user_id, event_hash, created_at FROM audit.events ORDER BY id DESC LIMIT 5;"
+```
+
+Expected: latest event row with non-empty `event_hash`.
+
+## UI Walkthrough (2-4 minutes)
+
+Use screenshot pack and/or running Angular app:
+
+1. Command accepted response: `docs/screenshots/01-command-accepted.png`
+2. Audit list response: `docs/screenshots/02-audit-logs-list.png`
+3. Integrity `ON_CHAIN`: `docs/screenshots/03-integrity-on-chain.png`
+4. Integrity `MISMATCH`: `docs/screenshots/04-integrity-mismatch.png`
+5. Kafka topics: `docs/screenshots/05-kafka-topics.png`
+6. Postgres events: `docs/screenshots/06-postgres-audit-events.png`
+7. Health endpoints: `docs/screenshots/07-health-endpoints.png`
+8. Angular dashboard: `docs/screenshots/08-angular-dashboard.png`
+
+## Demo Variants
+
+### Fast path (5-7 minutes)
+
+- Run Step 1, 2, 4 only.
+- Explain asynchronous eventual consistency in one sentence.
+
+### Deep technical path (15-20 minutes)
+
+- Run full core path.
+- Add Kafka consumer groups and lag checks.
+- Discuss canonical hash serialization and why it prevents DB/on-chain divergence.
+
+## Typical Interview Q&A
+
+### 1) Why blockchain here if you already store data in PostgreSQL?
+
+- PostgreSQL stores operational history.
+- Blockchain adds external tamper-evidence with immutable anchoring.
+- Integrity endpoint cross-checks DB hash against on-chain evidence.
+
+### 2) How do you guarantee hash consistency across services?
+
+- Both `event-store-service` and `audit-writer-service` use `CanonicalObjectMapperFactory.create()`.
+- Canonical JSON (stable field order and timestamp formatting) guarantees byte-identical hashing.
+
+### 3) Why Kafka between services?
+
+- Decouples write pipeline stages.
+- Supports backpressure/retries and independent scaling.
+- Preserves event-driven architecture boundaries.
+
+### 4) What does `PENDING` mean?
+
+`PENDING` appears in two different contexts with different meanings:
+
+- **Audit log list** (`GET /api/audit-logs`): `integrityStatus` is always `"PENDING"` — it is a hard-coded placeholder set by `AuditEventDtoMapper.defaultIntegrityStatus()`. It says nothing about whether `event_hash` is present or the record is valid; it simply means "not checked yet".
+- **Integrity-check endpoint** (`GET /api/audit-logs/{id}/integrity-check`): `PENDING` means the `event_hash` column is null or blank in `audit.events`. This is a legacy or corrupt-row condition — the event-store service writes `event_hash` synchronously during persistence, so a blank hash indicates the row was never properly processed. `PENDING` is **not** returned when the blockchain is unreachable; that scenario results in an error response instead.
+
+### 5) What does `MISMATCH` mean?
+
+- Stored DB hash is not observed on-chain.
+- Could indicate delayed anchoring, write failure, or tampering.
+
+### 6) How do you handle duplicate blockchain writes?
+
+- Contract rejects duplicate hashes.
+- Service uses retries + dead-letter topic for failed processing.
+
+### 7) What are your failure isolation points?
+
+- Command acceptance is isolated from downstream anchoring latency.
+- Event store and audit writer consume asynchronously and can recover independently.
+
+### 8) Which service owns schema migrations?
+
+- Only `event-store-service` owns Flyway migrations for `audit.events`.
+- Other services must not introduce independent Flyway schemas.
+
+### 9) Is this exactly-once processing?
+
+- End-to-end exactly-once is hard in distributed systems.
+- Design is effectively-once at business level via idempotency keys, duplicate hash rejection, and unique constraints.
+
+### 10) How do you scale read traffic?
+
+- `query-service` is read-optimized and stateless.
+- Scale horizontally behind load balancer.
+- Keep writes and reads separated (CQRS).
+
+### 11) Why WebFlux + Reactor?
+
+- Non-blocking model for I/O heavy services.
+- Better resource use under concurrent traffic.
+- Fits event-driven architecture.
+
+### 12) How do you secure this in production?
+
+- JWT + role-based access control.
+- Secret management for private keys and contract addresses.
+- TLS, network policies, audit logging, and limited DB privileges.
+
+## Risk/Trade-off Talking Points
+
+- Eventual consistency: integrity status may lag right after command acceptance.
+- Operational complexity: adds Kafka + blockchain infra.
+- Cost/performance: anchoring every event can be expensive on public networks; batching is a future optimization.
+
+## Troubleshooting During Demo
+
+### Integrity never reaches ON_CHAIN
+
+- Check `AUDIT_LEDGER_CONTRACT_ADDRESS` for `audit-writer-service` and `query-service`.
+- Verify Ganache RPC is reachable at `http://127.0.0.1:8545`.
+- Ensure private key env var is set for `audit-writer-service`.
+
+### No events in query service
+
+- Verify Kafka topic exists: `user.login.events`.
+- Check event-store consumer logs and PostgreSQL connectivity.
+
+### 401/403 from APIs
+
+- Re-authenticate and include JWT token.
+- Confirm role permissions for requested endpoint.
+
+## Related Docs
+
+- `docs/ARCHITECTURE.md`
+- `docs/CQRS_FLOW.md`
+- `docs/TESTING_SCENARIOS.md`
+- `docs/DEPLOYMENT.md`
+- `docs/EVENT_HASH_CANONICAL_MIGRATION.md`
+

docs/ROADMAP.md

@@ -542,6 +542,7 @@ CREATE TABLE audit.events (
 **ID:** #20  
 **Labels:** `docs`  
 **Depends on:** #12, #18  
+**Status:** ✅ Done  
 **Description:**
 Подготовить готовый сценарий для демонстрации на собеседовании.
 
@@ -553,13 +554,17 @@ CREATE TABLE audit.events (
 - Потенциальные вопросы и ответы
 
 **Subtasks:**
-- [ ] #20.1 - Написать guide
-- [ ] #20.2 - Prepare curl команды
-- [ ] #20.3 - Prepare скриншоты UI
-- [ ] #20.4 - Q&A документ
+- [x] #20.1 - Написать guide
+- [x] #20.2 - Prepare curl команды
+- [x] #20.3 - Prepare скриншоты UI
+- [x] #20.4 - Q&A документ
 
 **Expected PR:** PR-20 (Demo scenario documentation)
 
+**Deliverables:**
+- `docs/LIVE_DEMO_QA.md`
+- `docs/TESTING_SCENARIOS.md`
+
 ---
 
 ## Labels Reference

docs/screenshots/README.md

@@ -22,6 +22,37 @@ Use lowercase kebab-case names and keep files in PNG format:
 - Keep terminal width wide enough so commands/results are readable
 - Prefer one screenshot per scenario outcome
 
+## Runtime Regeneration
+
+Prerequisites:
+
+```pwsh
+pip install Pillow
+```
+
+Requires **Python 3.9+**, a running local stack (`deploy/docker-compose.yml`) and all four backend services on ports `8081`–`8084`. Credentials are read from env vars `DEMO_USERNAME` (default: `admin`) and `DEMO_PASSWORD` (default: `admin123!`).
+
+The script connects to Docker containers and PostgreSQL using the following env vars (shown with defaults matching `deploy/docker-compose.yml`):
+
+| Env var               | Default        | Purpose                            |
+|-----------------------|----------------|------------------------------------|
+| `DEMO_USERNAME`       | `admin`        | Auth login username                |
+| `DEMO_PASSWORD`       | `admin123!`    | Auth login password                |
+| `POSTGRES_CONTAINER`  | `dal-postgres` | Docker container name for psql     |
+| `POSTGRES_DB`         | `audit_ledger` | Database name                      |
+| `POSTGRES_USER`       | `postgres`     | PostgreSQL user                    |
+| `KAFKA_CONTAINER`     | `dal-kafka`    | Docker container name for Kafka    |
+| `SCREENSHOT_TIMESTAMP`| `1`            | Set `0` for deterministic PNGs     |
+| `CAPTURE_OUTPUT`      | `docs/screenshots/runtime/capture.json` | Path for `capture.json` output     |
+
+Use the generator to refresh screenshots from live local services:
+
+```pwsh
+python docs/screenshots/generate_runtime_screenshots.py
+```
+
+The script stores raw capture data in `docs/screenshots/runtime/capture.json` (gitignored).
+
 ## Current Status
 
 The screenshot pack files are present in this folder:
@@ -37,3 +68,7 @@ The screenshot pack files are present in this folder:
 
 If needed for final demo polish, replace generated images with runtime-captured screenshots while keeping the same filenames.
 
+Note for `08-angular-dashboard.png`:
+- If Angular app on `http://localhost:4200` is running, the screenshot will include a live frontend probe result.
+- If frontend is not running, the file captures the probe error so the demo pack still remains complete and reproducible.
+