Pdf native ingestion

docs/ROADMAP.md

@@ -27,4 +27,11 @@ This document tracks the active expansion phases for the AnalyticsBot pipeline.
 **Goal:** Allow the bot to monitor a directory and update analytics dynamically.
 - [x] Install filesystem watcher (`npm install chokidar`).
 - [x] Implement a `--watch` flag in the CLI orchestrator.
-- [x] Hook `chokidar` file events (add, change, unlink) into the `pipeline.js` to update the corpus index without full application restarts.
+- [x] Hook `chokidar` file events (add, change, unlink) into the `pipeline.js` to update the corpus index without full application restarts.
+
+## Stage 5: Advanced PDF & Structural Analysis
+**Goal:** Improve PDF ingestion by preserving page-level context and metadata.
+- [x] Refactor `readFileData` to support page-based text content extraction.
+- [x] Implement PDF metadata extraction (Author, CreationDate) to augment `Metadata Extractor`.
+- [x] Update `test/pipeline.test.js` with a mock PDF fixture to validate multi-page parsing.
+- [x] Test Resulting in successfull pdf file ingestion, parse and output.

package.json

@@ -22,6 +22,7 @@
   "dependencies": {
     "chokidar": "^5.0.0",
     "compromise": "^14.15.1",
+    "mupdf": "^1.27.0",
     "pdf-parse": "^2.4.5",
     "tesseract.js": "^7.0.0"
   }