docs: consolidate branches into main — fix architecture drift, complete sprint docs, add Quick Start

README.md

@@ -9,6 +9,7 @@
 
 ## Table of Contents
 
+- [Quick Start](#quick-start)
 - [Core Loop: Ingest -> Analyze -> Report](#core-loop-ingest---analyze---report)
 - [Analytics Scope](#analytics-scope)
 - [Core Features](#core-features)
@@ -40,6 +41,20 @@
 [↑ Back to Top](#uap-analyticsbot)
 
 
+## Quick Start
+
+Get the pipeline running in four commands. For full installation options and output formats, see the [User Guide](docs/USER_GUIDE.md).
+
+```bash
+git clone https://github.com/aj1126/UAP_AnalyticsBot.git
+cd UAP_AnalyticsBot
+npm install
+npm start -- /path/to/your/documents --format=md
+```
+
+[↑ Back to Top](#uap-analyticsbot)
+
+
 ## Core Loop: Ingest -> Analyze -> Report
 
 UAP AnalyticsBot is a file-first analytics system built around a repeatable three-stage loop:

docs/SPRINT_01.md

@@ -4,22 +4,21 @@
 This sprint focuses on eliminating main-thread bottlenecks during WebAssembly PDF rasterization and deepening the analytical accuracy of the Diagnostic tier.
 
 ## 🛠️ Task 1: Worker Pool Implementation (Performance)
-- [ ] Create src/ingestion/worker.js to house the mupdf and Tesseract logic.
-- [ ] Update src/ingestion/file-ingestion.js to utilize 
-ode:worker_threads to spawn a pool sized dynamically to the host's CPU core count.
-- [ ] Implement a thread-safe queue for the directory walker to dispatch file paths to idle workers.
+- [x] Create src/ingestion/worker.js to house the mupdf and Tesseract logic.
+- [x] Update src/ingestion/file-ingestion.js to utilize node:worker_threads to spawn a pool sized dynamically to the host's CPU core count.
+- [x] Implement a thread-safe queue for the directory walker to dispatch file paths to idle workers.
 
 ## 🛠️ Task 2: State Caching (Memoization)
-- [ ] Implement a fingerprinting function in the ingestion node using file stats (size + mtime).
-- [ ] Create a non-destructive .analytics_cache.json to store parsed data arrays.
-- [ ] Add a short-circuit boolean check to bypass worker dispatch if the fingerprint matches the cache.
+- [x] Implement a fingerprinting function in the ingestion node using file stats (size + mtime).
+- [x] Create a non-destructive .analytics_cache.json to store parsed data arrays.
+- [x] Add a short-circuit boolean check to bypass worker dispatch if the fingerprint matches the cache.
 
 ## ✨ Task 3: TF-IDF Engine (Diagnostic Tier)
-- [ ] Update src/analytics/diagnostic.js to calculate Inverse Document Frequency across the total ileCount.
-- [ ] Map the TF-IDF weight for each word in individual files.
-- [ ] Output the top 5 most highly weighted (unique) keywords per file into the final Diagnostic JSON.
+- [x] Update src/analytics/diagnostic.js to calculate Inverse Document Frequency across the total fileCount.
+- [x] Map the TF-IDF weight for each word in individual files.
+- [x] Output the top 5 most highly weighted (unique) keywords per file into the final Diagnostic JSON.
 
 ## Definition of Done
-- [ ] All 	est/pipeline.test.js unit tests passing.
-- [ ] Pipeline runs fully non-destructively (source files unmodified).
-- [ ] Performance benchmark shows >50% reduction in processing time for a cached directory.
+- [x] All test/pipeline.test.js unit tests passing.
+- [x] Pipeline runs fully non-destructively (source files unmodified).
+- [x] Performance benchmark shows >50% reduction in processing time for a cached directory.

docs/SPRINT_02.md

@@ -4,20 +4,19 @@
 This sprint introduces string entity normalization loops to prevent metadata token fragmentation and establishes cross-document vector mapping using similarity metrics.
 
 ## 🛠️ Task 1: Named Entity Token Unification (NLP)
-- [ ] Refactor xtractLocations in src/ingestion/file-ingestion.js to clear formatting symbols and possessives.
-- [ ] Enforce upper or title-case uniformity over extracted proper noun objects before array set compilation.
+- [x] Refactor extractLocations in src/ingestion/file-ingestion.js to clear formatting symbols and possessives.
+- [x] Enforce upper or title-case uniformity over extracted proper noun objects before array set compilation.
 
 ## 🛠️ Task 2: Cosine Similarity Scoring Nodes (Diagnostic Tier)
-- [ ] Create a vector generator module inside src/analytics/diagnostic.js.
-- [ ] Calculate a multi-dimensional matrix cross-referencing document vectors based on unique TF-IDF weights.
-- [ ] Append a elatedDocuments array listing high-correlation file matches into individual descriptive metadata blocks.
+- [x] Create a vector generator module inside src/analytics/diagnostic.js.
+- [x] Calculate a multi-dimensional matrix cross-referencing document vectors based on unique TF-IDF weights.
+- [x] Append a relatedDocuments array listing high-correlation file matches into individual descriptive metadata blocks.
 
 ## ✨ Task 3: Nonlinear Forecasting Tweaks (Predictive Tier)
-- [ ] Refactor orecastNextValue in src/analytics/predictive.js to replace standard linear delta metrics with weighted moving averages.
-- [ ] Adjust time-series arrays to support empty intervals gracefully when data contains chronological gaps.
+- [x] Refactor forecastNextValue in src/analytics/predictive.js to replace standard linear delta metrics with weighted moving averages.
+- [x] Adjust time-series arrays to support empty intervals gracefully when data contains chronological gaps.
 
 ## Definition of Done
-- [ ] Test cases validate that casing discrepancies (e.g., 'Roswell' vs 'ROSWELL') match safely.
-- [ ] The CLI output provides semantic vector cross-references in the final JSON payload.
-- [ ] All 
-pm run docs:generate checkpoints complete successfully.
+- [x] Test cases validate that casing discrepancies (e.g., 'Roswell' vs 'ROSWELL') match safely.
+- [x] The CLI output provides semantic vector cross-references in the final JSON payload.
+- [x] All npm run docs:generate checkpoints complete successfully.

docs/SPRINT_03.md

@@ -4,25 +4,21 @@
 This sprint focuses on the outward-facing reliability and usability of the bot. It introduces automated testing pipelines, expanded export formats for data scientists, and power-user terminal controls.
 
 ## 🛠️ Task 1: Continuous Integration (CI/CD)
-- [ ] Create .github/workflows/test.yml to trigger on push and pull_request.
-- [ ] Configure the action to run 
-pm ci and 
-pm test across latest Node.js versions.
-- [ ] Ensure the action verifies 
-pm run docs:check to enforce documentation standards.
+- [x] Create .github/workflows/test.yml to trigger on push and pull_request.
+- [x] Configure the action to run npm ci and npm test across latest Node.js versions.
+- [x] Ensure the action verifies npm run docs:check to enforce documentation standards.
 
 ## ✨ Task 2: CSV Data Export Generation
-- [ ] Create src/delivery/csv-generator.js.
-- [ ] Implement flat-mapping logic to translate nested Diagnostic and Predictive JSON arrays into tabular CSV rows.
-- [ ] Update src/index.js to accept --format=csv and route the JSON telemetry to the new generator.
+- [x] Create src/delivery/csv-generator.js.
+- [x] Implement flat-mapping logic to translate nested Diagnostic and Predictive JSON arrays into tabular CSV rows.
+- [x] Update src/index.js to accept --format=csv and route the JSON telemetry to the new generator.
 
 ## 🛠️ Task 3: Advanced CLI Argument Parsing
-- [ ] Implement a lightweight argument parser in src/index.js.
-- [ ] Add --clear-cache to delete the local .analytics_cache.json before a run.
-- [ ] Add --workers=<num> to allow users to override default OS CPU core limits.
+- [x] Implement a lightweight argument parser in src/index.js.
+- [x] Add --clear-cache to delete the local .analytics_cache.json before a run.
+- [x] Add --workers=<num> to allow users to override default OS CPU core limits.
 
 ## Definition of Done
-- [ ] GitHub Actions badge displays "Passing" on the README.
-- [ ] Running with --format=csv outputs a valid, spreadsheet-readable file in the data_exports directory.
-- [ ] 
-pm run docs:generate is updated to reflect the new command-line flags.
+- [x] GitHub Actions badge displays "Passing" on the README.
+- [x] Running with --format=csv outputs a valid, spreadsheet-readable file in the data_exports directory.
+- [x] npm run docs:generate is updated to reflect the new command-line flags.

docs/architecture.md

@@ -15,20 +15,21 @@ The repository currently ships a Node.js CLI-centered analytics flow:
 
 ## Current Runtime Boundaries
 
-Implemented today:
-
-- recursive read-only ingestion for `.txt`, `.md`, `.json`, `.csv`, and `.log`
-- multithreaded parsing with fingerprint-based cache reuse for compatible ingestions
-- tokenization plus lightweight date/location extraction
+Implemented in the active system:
+
+- recursive read-only ingestion for `.txt`, `.md`, `.json`, `.csv`, `.log`, `.pdf`, `.png`, `.jpg`, and `.jpeg`
+- multithreaded worker pool (`node:worker_threads`) with fingerprint-based memoization cache (`.analytics_cache.json`)
+- PDF text extraction via `pdf-parse` with automatic OCR fallback (MuPDF + Tesseract) for rasterized/corrupted documents
+- image OCR via `tesseract.js` for `.png`/`.jpg`/`.jpeg` files
+- Named Entity Recognition (NER) for dates and locations via the `compromise` NLP library, with regex fallback for structured fields
+- TF-IDF weighting and Cosine Similarity cross-linking in the diagnostic tier (`src/analytics/diagnostic.js`)
+- weighted moving average forecasting with empty-interval fill in the predictive tier (`src/analytics/predictive.js`)
 - descriptive, diagnostic, predictive, and prescriptive analytics modules
-- JSON, Markdown, and CSV report delivery through the Node CLI
-- directory watch mode that re-runs the pipeline after file changes
-
-Not yet implemented in the active system:
-
-- binary or multimedia extraction
-- Named Entity Recognition (NER)
-- dashboard or background scheduling
+- JSON, Markdown, and CSV report delivery through the Node CLI (`--format=md`, `--format=csv`)
+- directory watch mode via `chokidar` (`--watch`) that re-runs the pipeline after file changes
+- `--workers=<n>` and `--clear-cache` CLI flags
+- local web GUI (`src/gui/server.js`) for browser-based directory browsing and analysis
+- telemetry pipeline (`src/telemetry/`) for GitHub webhook ingestion, drift detection, and subagent handoff simulation
 
 ## Planned Expansion