Skip to content

Commit 602ff41

Browse files
committed
chore: update AGENTS.md and README.md
1 parent 7fca701 commit 602ff41

2 files changed

Lines changed: 206 additions & 158 deletions

File tree

AGENTS.md

Lines changed: 71 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
# AGENTS.md — UTMStack Repository Guide
22

3+
See [org-level AGENTS.md](../AGENTS.md) for branch strategy and CI/CD overview.
4+
35
## Overview
46

57
SIEM/XDR platform. Multi-language monorepo:
@@ -15,21 +17,23 @@ SIEM/XDR platform. Multi-language monorepo:
1517
| `plugins/*/` | Go 1.25.5 | `go build` (17 modules) |
1618
| `shared/` | Go 1.25.1 | — (shared library) |
1719
| `installer/` | Go 1.25.1 | `go build` (+ ldflags) |
18-
| `user-auditor/` | Java 11 | Maven |
19-
| `web-pdf/` | Java 11 | Maven |
20+
| `user-auditor/` | Java 11 (Spring Boot 2.7) | Maven |
21+
| `web-pdf/` | Java 11 (Spring Boot 2.7) | Maven |
2022

2123
## Build Commands
2224

23-
### Backend (Java)
25+
### Backend (Java 17)
2426
```bash
2527
cd backend
26-
mvn -s settings.xml -B # Run Spring Boot dev server (port 8080)
27-
mvn -B -Pprod clean package -s settings.xml # Production WAR → target/utmstack.war
28+
mvn -s settings.xml -B # Spring Boot dev server (port 8080)
29+
mvn -B -Pprod clean package -s settings.xml # Production WAR → target/utmstack.war
2830
```
29-
- Maven settings: `backend/settings.xml` authenticates to GitHub Packages via `MAVEN_TK` env var (GitHub PAT with `read:packages`).
31+
- `settings.xml` authenticates to GitHub Packages via `MAVEN_TK` env var (GitHub PAT with `read:packages`).
3032
- Spring profiles: `dev` (default), `prod`, `tls`. Config in `backend/src/main/resources/config/`.
3133
- Docker dev database: `docker-compose -f backend/src/main/docker/mysql.yml up -d`
3234
- **No `src/test/` directory** — tests are embedded in `src/main/java/`.
35+
- **`jib-maven-plugin` image mismatch**`pom.xml` references `eclipse-temurin:11-jre-focal` but the backend requires Java 17. The `Dockerfile` correctly uses `eclipse-temurin:17`. Jib builds will fail or produce broken images.
36+
- **`proto-command.txt`** contains the protoc command for gRPC code generation.
3337

3438
### Frontend (Angular 7)
3539
```bash
@@ -44,6 +48,7 @@ npm run lint # ng lint (TSLint, NOT ESLint)
4448
- Linter is **TSLint** (`tslint.json`), not ESLint. Uses `codelyzer` rules.
4549
- Output: `dist/utm-stack`. Styles use SCSS (`angular.json`).
4650
- **Builds need 8 GB heap**: `NODE_OPTIONS=--max_old_space_size=8192`.
51+
- Frontend Dockerfile serves via nginx (see `frontend/nginx/`).
4752

4853
### Go Components
4954
Each module is independent. Build from its directory:
@@ -72,7 +77,12 @@ CI injects `$AGENT_SECRET_PREFIX` for all three. Without it, these services cann
7277
### Plugins
7378
Each plugin under `plugins/*/` is a standalone Go module. Build binary named `com.utmstack.<name>.plugin`.
7479

75-
**16 plugins** are copied into `event_processor.Dockerfile`. The `plugins/` directory currently has 17 modules — `compliance-orchestrator` exists but is not yet in the Dockerfile.
80+
**16 plugins** are in `event_processor.Dockerfile`. The `plugins/` directory has 17 modules — `compliance-orchestrator` is in the directory but not yet in the Dockerfile.
81+
82+
Plugin list in Dockerfile: `alerts`, `aws`, `azure`, `bitdefender`, `config`, `events`, `gcp`, `geolocation`, `inputs`, `o365`, `sophos`, `stats`, `soc-ai`, `modules-config`, `crowdstrike`, `feeds`.
83+
84+
### user-auditor and web-pdf
85+
Both are small Spring Boot 2.7.14 microservices (Java 11). Each has its own `pom.xml`, `Dockerfile`, and `compose.yml`. Built independently of the main backend.
7686

7787
### Installer
7888
```bash
@@ -88,32 +98,67 @@ https://storage.googleapis.com/utmstack-updates/dependencies/geolocation/
8898
```
8999
`geolocation/` is gitignored — must be populated from GCS before Docker build.
90100

101+
## Architecture Overview
102+
103+
```
104+
endpoint agent ──gRPC──▶ agent-manager ──▶ backend (Java REST API) ↔ Angular frontend
105+
106+
event processor (Go, plugin-loaded)
107+
├── plugins (17 Go modules)
108+
└── filters/ + rules/ (YAML)
109+
110+
utmstack-collector / as400 (log ingestion)
111+
```
112+
113+
- **Backend** serves the REST API. WAR packaging. `filters/` and `rules/` are YAML files copied into the container at `/utmstack/filters` and `/utmstack/rules`.
114+
- **Event processor** is the core Go-based log correlation engine. Loads compiled plugin binaries at runtime. `event_processor.Dockerfile` expects all plugins pre-built alongside it.
115+
- **Frontend** is a standalone Angular app served by nginx in a separate container.
116+
- **Agent** runs on endpoints (Windows/Linux/macOS). Communicates with `agent-manager` via gRPC. Has Windows resource files (`rsrc_windows_*.syso`) for embedded icons.
117+
- **Collector** (`utmstack-collector/`) and **AS400** (`as400/`) are separate log collection services. `as400/` is a near-identical copy of `utmstack-collector/` (same structure, different parser).
118+
- **etc/** contains ISO build configs and OpenSearch configuration.
119+
120+
## Testing
121+
122+
- **Go**: `go test -v ./...` in each module directory. CI runs this via `reusable-golang.yml` before build.
123+
- **Frontend**: `npm test` (Karma + Jasmine). Note: `e2e` script exists but uses Protractor (deprecated).
124+
- **Backend**: No separate `src/test/` tree. Tests live in `src/main/java/` alongside production code.
125+
- **Go deps check**: `bash .github/scripts/go-deps.sh --check --discover` (discovers all `go.mod` files, checks for outdated direct deps and out-of-sync `go.sum`).
126+
91127
## CI / CD
92128

93129
### PR Checks
94-
`.github/workflows/pr-checks.yml` — runs on PRs to `release/**`, `v10`, `v11`. Triggers Go dependency checks, AI review, and approver job.
130+
`.github/workflows/pr-checks.yml` — runs on PRs to `release/**`, `v10`, `v11`. Three jobs:
131+
1. `go_deps` — dependency freshness check across all Go modules
132+
2. `ai_review` — matrix of ThreatWinds AI prompts (`.github/ai-prompts/`)
133+
3. `approver` — consolidates results, sticky comments, optional formal review + auto-merge
134+
135+
**AI prompts:** `security.md`, `bugs.md`, `architecture.md`. Default model: `gemini-3-flash-lite`. Add new prompts by dropping `.md` into `.github/ai-prompts/`.
136+
137+
**Tiered approval:**
138+
- Tier 1: AI approves, auto-merge on `release/**`
139+
- Tier 2: AI flags issues, `REQUEST_CHANGES`
140+
- Tier 3: Critical path (crypto, auth, migrations, gRPC, CI/CD) → human review
141+
142+
See `.github/workflows/README.md` for full CI/CD documentation (secrets, approver setup, deployment flows, hotfix procedure).
95143

96144
### Deployment Pipelines
97-
- **v11** (`.github/workflows/v11-deployment-pipeline.yml`) — active line. Triggers: push to `release/v11**` (dev), push to `v11` (rc), `release.released` (prod).
98-
- **v10** (`.github/workflows/v10-deployment-pipeline.yml`) — legacy. Triggers: push to `release/v10**` (dev), push to `v10` (rc), `release.released` with `v10.*` tag (prod).
145+
- **v11** (`.github/workflows/v11-deployment-pipeline.yml`) — active line
146+
- **v10** (`.github/workflows/v10-deployment-pipeline.yml`) — legacy (EOL Dec 2026)
147+
148+
Images published to `ghcr.io/utmstack/utmstack/<image>:<tag>`.
99149

100150
### Reusable Workflows
101-
- `reusable-java.yml` — Maven build + Docker push to `ghcr.io/utmstack/utmstack/<image>:<tag>`
102-
- `reusable-golang.yml``go test ./...`, `go build`, Docker push
151+
- `reusable-java.yml` — Maven build + Docker push
152+
- `reusable-golang.yml``go test ./...` + `go build` + Docker push
103153
- `reusable-node.yml` — Node 14.16.1, `npm install && npm run-script build`, Docker push
104154
- `reusable-sign-agent.yml` — Windows (jsign + GCP KMS) and macOS (codesign + notarytool) signing
105-
- `reusable-basic.yml` — Docker build-only
106-
107-
### Agent Signing Workflow
108-
`installer-release.yml` builds the installer binary with ldflags using `CM_ENCRYPT_SALT` and `CM_SIGN_PUBLIC_KEY` secrets.
155+
- `reusable-basic.yml` — Docker build only
109156

110-
## Key Architecture Notes
157+
## Filters, Rules, and Plugins
111158

112-
- **Event processor** is the core Go-based log correlation engine. Loads compiled plugin binaries at runtime. `event_processor.Dockerfile` expects all plugins pre-built alongside it.
113-
- **Backend** serves the REST API. WAR packaging. `filters/` and `rules/` are YAML files copied into the container at `/utmstack/filters` and `/utmstack/rules`.
114-
- **Frontend** is a standalone Angular app in a separate container.
115-
- **Agent** runs on endpoints (Windows/Linux/macOS). Communicates with `agent-manager` via gRPC.
116-
- **Collector** (`utmstack-collector/`) and **AS400** (`as400/`) are separate log collection services.
159+
- **filters/** and **rules/** — YAML files, organized by vendor/domain (25+ categories each: antivirus, aws, azure, cisco, crowdstrike, fortinet, generic, windows, etc.)
160+
- Plugin/filerule/rule documentation: [UTMStack Wiki](https://github.com/utmstack/UTMStack/wiki)
161+
- Custom plugin development guide: `UTMStack.wiki/Custom-Plugin-Development.md`
117162

118163
## Gotchas
119164

@@ -125,4 +170,7 @@ https://storage.googleapis.com/utmstack-updates/dependencies/geolocation/
125170
- **Installer build requires ldflags** — see `installer/build.sh` for `DEFAULT_BRANCH`, `INSTALLER_VERSION`, `REPLACE`, `PUBLIC_KEY`.
126171
- **Geolocation data must be downloaded** — event processor Docker build fails without `./geolocation/` CSV files from GCS.
127172
- **`.plugin` binaries are gitignored** — build artifacts, not committed.
128-
- **`jib-maven-plugin` image mismatch** — pom.xml references `eclipse-temurin:11-jre-focal` but the backend requires Java 17.
173+
- **`jib-maven-plugin` image mismatch**`pom.xml` references `eclipse-temurin:11-jre-focal` but the backend requires Java 17.
174+
- **`as400/` is a copy of `utmstack-collector/`** — nearly identical structure (agent, collector, config, conn, database, logservice, models, serv, utils). Changes to one usually need mirroring in the other.
175+
- **`CONTRIBUTING.md` is stale** — mentions PEP 8 (Python) but there is no Python code in the repo.
176+
- **`shared/` is a library, not buildable standalone** — consumed via `replace` directives. Don't try to `go build` it directly.

0 commit comments

Comments
 (0)