feat: update report service and report generation template [26T1-DOC-RR-001]#230
feat: update report service and report generation template [26T1-DOC-RR-001]#230raaidrushdy wants to merge 7 commits into
Conversation
- Updated the current report_service.py to match the new template - Add AutoAudit_Report_Template.docx for Word report generation - Add run_test.py smoke test for local testing - Add fake_dataset.json sample dataset - Add README_report_service.md with schema docs and onboarding notes
raaidrushdy
requested review from
6igby,
du-dhartley and
liyunze-coding
as code owners
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ab222f3d0b
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| return out | ||
|
|
||
|
|
||
| def generate_single_finding_pdf( |
There was a problem hiding this comment.
generate_pdf entry point
Existing entry points still do from security.reports.report_service import generate_pdf in security/evidence_backend/reportgenerator.py and security/evidence_ui/app.py, but this rewrite only exposes the renamed single-finding functions. In an environment with the report dependencies installed, those modules will fail at import time with ImportError before users can generate evidence reports; keep a generate_pdf wrapper/alias or update those callers in the same change.
Useful? React with 👍 / 👎.
| dataset = args[0] if args and not args[0].startswith("--") else "fake_dataset_template_ready.json" | ||
| template = args[1] if len(args) > 1 and not args[1].startswith("--") else "AutoAudit_Report_Template.docx" |
There was a problem hiding this comment.
The default CLI path uses fake_dataset_template_ready.json and AutoAudit_Report_Template.docx, but this commit adds security/reports/fake_dataset.json and updates security/results/report_template.docx instead. Running the documented no-argument command from security/reports therefore fails immediately with FileNotFoundError, so the smoke-test/default path is not usable unless the caller supplies both paths manually.
Useful? React with 👍 / 👎.
- Add generate_pdf() alias so existing callers in evidence_backend/ reportgenerator.py and evidence_ui/app.py continue to work without changes (maps to generate_single_finding_pdf) - Fix CLI default dataset path from fake_dataset_template_ready.json to fake_dataset.json to match the file actually committed
|
Fixed both. Added a generate_pdf backward-compat alias that maps to generate_single_finding_pdf with the same signature both callers use. Also corrected the CLI default filename to fake_dataset.json. |
|
@raaidrushdy Can this be run on the real tenant to avoid needing to use sample data? We have the tenant, we should use it rather than risk an intermediate layer that could potentially have issues. |
|
yeah, the fake dataset is just so anyone can test the report locally without needing tenant access. it never goes near the actual pipeline. when the engine runs against the real tenant that output feeds straight into the report generator, the sample data has nothing to do with that. |
|
@raaidrushdy I understand what the fake dataset is for, but we should generate the dataset from a real tenant, unless that has been done already. |
|
@du-dhartley end-to-end testing against the real tenant requires the engine to run against the tenant first and produce real output, and to my knowledge, that hasn't happened yet, so there's no real dataset to test against at this stage. once the engine has run against the tenant and produced output, I can test the report generator against it. I'm happy to do that as a follow-up once that data exists. |
|
@raaidrushdy End to end testing against a real tenant has been available and working since T3 2025. All of the scans that I've been running have been against the same tenant that we have documented and discussed in our teams chats. |
|
hi @du-dhartley, would i be able to get the login for bitwarden? thank you! |
…on from real tenant scan data
CI: Security
One or more checks failed. View logs |
|
tested against the real sandbox tenant, spun up the stack, ran a scan, and generated the report from the actual results. the unfilled fields in the detailed findings (ISO mapping, impact, root cause, etc.) are expected; the data isn't coming from the scan results, it needs to come from the GRC pipeline separately. not a bug in the report service. also added generate_report_from_scan.py to the branch, it takes the scan results from the API and transforms them into the format the report service expects. Works live against the API or from saved JSON files. README updated with how to use it. 
|
2 participants
Summary
Builds the AutoAudit M365 compliance report pipeline end-to-end. This includes a redesigned report template, a fully reworked report_service.py aligned to the new template, bug fixes across the service, and all supporting files needed to generate, test, and understand the pipeline.
Team Contributions
This PR was completed as part of task 26T1-DOC-RR-001 and reflects contributions from the full report generation workstream team. Teammates can link this PR to their 10.1P evidence against the same planner card.
Type of Change
Affected Components
/backend-api/frontend/engine(collectors / policies)/security/infrastructure/.github/workflows/docsMotivation
Task 26T1-DOC-RR-001 - build the AutoAudit M365 compliance report template. The previous report_service.py was built against a different template structure and had accumulated bugs that prevented the correct generation of reports. This PR redesigns the template from scratch, aligns the service with it, resolves all known bugs, and adds the tooling teammates need to run and validate the pipeline locally and against the actual M365 tenant.
Testing Done
Ran run_test.py against fake_dataset.json with the new template and confirmed the generated .docx opened correctly in Word with all placeholders filled. Also validated the pipeline against the actual M365 tenant to confirm end-to-end report generation works against live compliance data.
Security Considerations
No security impact. This change only affects offline report generation from already-collected compliance data. No auth, secrets, API permissions, or data exposure involved.
Breaking Changes
Rollback Plan
Checklist
Screenshots