Skip to content
This repository was archived by the owner on May 28, 2026. It is now read-only.
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
364fe4b
refactor(readme). update team-level readme for dev branch
PostboxRetinal Apr 22, 2026
82ab4b5
refactor(readme). fixed spanish typos
PostboxRetinal Apr 22, 2026
eb9c8df
feat: Add Logstash SIEM pipeline configuration and supporting scripts
JuanJaramillo12004 Apr 26, 2026
bbbd420
feat: Add new SIEM detection rules for SSH brute force, after-hours l…
JuanJaramillo12004 Apr 29, 2026
c418ec4
feat: Update Docker and configuration files for improved compatibilit…
JuanJaramillo12004 Apr 29, 2026
b58db96
feat: Enhance Docker Compose and import-rules script for improved Kib…
JuanJaramillo12004 Apr 29, 2026
167ec0f
feat: Update rules-import service to use curl image and implement aut…
JuanJaramillo12004 Apr 29, 2026
72a608d
cambios a logstash, ya inicia
Joseligos Apr 29, 2026
a0ee27a
metricbeat, puede que no se utilice
Joseligos Apr 29, 2026
b2ca489
refactor(infra). bumped STACK version + removed literal burned constants
PostboxRetinal Apr 29, 2026
fe01b1b
feat: add system diagnostic script and incident response playbooks fo…
xJuanes21 Apr 29, 2026
ca12270
refactor(arch). lots of critical changes for bind mounts and linux sp…
PostboxRetinal Apr 29, 2026
e7131e4
refactor(metricbeat). removed metricbeat partial integration
PostboxRetinal Apr 30, 2026
7a4695b
refactor(docs). updated custom instructions + README general guidance
PostboxRetinal Apr 30, 2026
cf2418b
fix(kibana). bumped security enablinb HTTP/2 + TLS, security stuff tbh
PostboxRetinal May 1, 2026
37462d1
refactor(kibana). removed unneeded AI noisy dep 'fleet'
PostboxRetinal May 1, 2026
11e6081
refactor(kibana). removed unneded report export option
PostboxRetinal May 1, 2026
76deb89
feat(kibana): move queries to HTTPS
PostboxRetinal May 1, 2026
7acaafe
refactor(logging). added deprecated warn bypasses + geoip fixes
PostboxRetinal May 1, 2026
c8a5908
fix(docs). removed all kind of unwanted emojis
PostboxRetinal May 1, 2026
1467a3e
refactor(instructions). merged with andrej-karpathy-skill's file
PostboxRetinal May 2, 2026
6048012
feat: add cross-platform container management script and project docu…
xJuanes21 May 7, 2026
8457042
fix(compose). removed not working wrapper files
PostboxRetinal May 7, 2026
1c24339
feat: set setup.template.overwrite on true
xJuanes21 May 7, 2026
20021b1
downgrade a v8.3.3 del stack. ya funcionan las rules-import, falta co…
Joseligos May 7, 2026
7ae9008
refactor(siem). updated templates priority and compose ES roles. Tele…
PostboxRetinal May 7, 2026
db44e9c
arreglados errores que no permitian que arrancara el container de kib…
Joseligos May 10, 2026
1677e71
dashboards agregados bien. estan 'rotos' toca arreglarlos.
Joseligos May 11, 2026
2d45c15
feat: Enhance Kibana dashboard import and data view creation
JuanJaramillo12004 May 11, 2026
92a17a2
generate-test-logs funcionando en dataviews. reglas funcionando. dash…
Joseligos May 11, 2026
60b14ab
archivos faltantes
Joseligos May 11, 2026
c97938c
refinamiento de dashboard ejecutivo (pendiente tendencia de alertas p…
xJuanes21 May 11, 2026
2127513
arreglo de mapa de ip sospechosas
xJuanes21 May 11, 2026
affc807
problema de inicializacion de stack mitigado, ambos dashboards refina…
xJuanes21 May 11, 2026
02c078a
cambios en cómo se generan los logs. Ahora estan divididos en 3 archi…
Joseligos May 14, 2026
200ed2a
reglas importadas correctamente. demás logs añadidos pero sin visuali…
Joseligos May 15, 2026
003e5fd
refactor(cleanup.) removed root elk folder
PostboxRetinal May 15, 2026
4e0c4bf
fix(logs). back to normal, try switching data timestamp
PostboxRetinal May 15, 2026
e22712b
refactor(provisioning). patched and exported provisioning files. Miss…
PostboxRetinal May 15, 2026
4682e4c
fix(filters). updated to UTC-5 TMZ
PostboxRetinal May 15, 2026
52cd07c
feat(dashboards). added new and polished gauges (mano se ve una re ch…
PostboxRetinal May 15, 2026
f9a1b85
feat(triage). added polished executive fresh gauges (chimba^2)
PostboxRetinal May 15, 2026
217e971
feat(ci/cd). added python, YAML and shell lint checkers for main branch
PostboxRetinal May 15, 2026
a167769
refactor(gitignore). added missing ruff_cache dir
PostboxRetinal May 15, 2026
2f464c1
feat(sentinel). added bicep IaC files + docs
PostboxRetinal May 15, 2026
f757481
cambios para que funcione en windows
Joseligos May 16, 2026
0b69e43
bugfix(sentinel). stack should boot now as expected, Linux systems mu…
PostboxRetinal May 16, 2026
70b79a8
feat(format). normalized LF format along all repo files
PostboxRetinal May 16, 2026
c384df3
cambios para que se vean todas las alertas de forma correcta
Joseligos May 16, 2026
e845f3a
Merge PR #1 from PostboxRetinal/feat/ml_sentinel
PostboxRetinal May 16, 2026
95fa181
refactor(gitignore). removed unexistent file paths
PostboxRetinal May 16, 2026
512ae87
fix(parsing). added TLS bypass by default to parsing py script
PostboxRetinal May 16, 2026
028afaf
feat(docs). final documentation
PostboxRetinal May 20, 2026
e8f1076
refactor(readme). salvedaad de entornos para despliegue
PostboxRetinal May 20, 2026
a6ceb5d
feat. diapositivas
PostboxRetinal May 20, 2026
a1de334
fix(docs). merge fro README
PostboxRetinal May 20, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
root = true

[*]
charset = utf-8
end_of_line = lf
insert_final_newline = true
14 changes: 14 additions & 0 deletions .env.sentinel.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# Copiar a .env.sentinel y completar con las salidas del despliegue Azure CLI.
# No agregar .env.sentinel al repositorio.

AZURE_TENANT_ID=
AZURE_CLIENT_ID=
AZURE_CLIENT_SECRET=

SENTINEL_DCE_ENDPOINT=
SENTINEL_DCR_IMMUTABLE_ID=
SENTINEL_STREAM_NAME=Custom-CloudSIEM

SENTINEL_POLL_SECONDS=10
SENTINEL_BATCH_SIZE=500
SENTINEL_LOOKBACK_MINUTES=15
1 change: 1 addition & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
* text=auto eol=lf
40 changes: 40 additions & 0 deletions .github/AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# AGENTS.md

## Repo Shape
- Read `README.md` and `.github/copilot-instructions.md` first; they define the project scope, required detections, and documentation language.
- The stack lives at the repository root; do not assume a nested `siem-elk/` directory.
- `develop` is the working branch; `main` is stable and PR-only.

## Run The Stack
- For Podman on Linux, use `bash setup/run-podman-sentinel-stack.sh --build --force-recreate --no-sentinel` for the local stack, or remove `--no-sentinel` when Sentinel is enabled.
- Stop the matching stack with `podman-compose -f docker-compose.yml -f docker-compose-podman.yml down`, or add `-f docker-compose-sentinel.yml -f docker-compose-sentinel-podman.yml` for Sentinel.
- `docker-compose.yml` is the universal base; `docker-compose-podman.yml` is the Linux + Podman override and owns SELinux relabeling.
- The Podman override maps `$XDG_RUNTIME_DIR/podman/podman.sock` to `/var/run/docker.sock` for Filebeat metadata enrichment.
- The cluster bootstrap takes about 90 seconds before health checks are meaningful.

## Verify
- From the repository root: `bash setup/verify-cluster.sh`, `python3 setup/check-cluster-health.py`, `python3 setup/verify-ecs-mapping.py`.
- From the repository root: `bash logstash/test-pipeline.sh`.
- Run Filebeat config checks with Podman, not Docker:
`podman run --rm --user 0 -e ELASTIC_PASSWORD=test -v "$PWD/filebeat/filebeat.yml:/usr/share/filebeat/filebeat.yml:ro,Z" docker.elastic.co/beats/filebeat:9.3.3 filebeat test config -e -c /usr/share/filebeat/filebeat.yml`

## Filebeat And Logs
- Filebeat configs should stay on `filestream`; use `parsers` for multiline and container parsing.
- Do not reintroduce `type: log` or `type: container` in Filebeat configs.
- `logs/` contains live sample inputs for Filebeat.
- `setup/generate-test-logs.py` writes directly to log files; its defaults point at `/var/log/...`, so override paths before using it on the host.

## Scripts With Relative Paths
- `setup/import-rules.py` has brittle relative-path defaults; check `RULES_DIR` and `CACERT` before invoking it.
- Several helper scripts read `.env` from the current directory, so run them from the directory shown above.

## Coding Approach
- Think before coding: state assumptions explicitly. If something is unclear or has multiple plausible interpretations, ask instead of guessing.
- Keep solutions minimal: implement only what was requested. Avoid speculative abstractions, configurability, or error handling for impossible scenarios.
- Make surgical changes: touch only what is necessary, match existing style, and do not refactor unrelated code or comments. Remove only unused imports, variables, or functions created by your own changes.
- Be goal-driven: for multi-step work, outline a brief plan and define how each step will be verified.

## Content Rules
- Keep docs in Spanish and code/config identifiers in English.
- Do not use emojis under any circumstance.
- Preserve the 5 required detections, 3 attack simulations, and 2 response playbooks when editing project scope docs.
10 changes: 8 additions & 2 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ applyTo: "**"
# CloudSIEM Copilot Instructions

## Project scope
- This repository is an academic SIEM project built around `Elasticsearch`, `Logstash`, `Kibana`, `Filebeat` and/or `Metricbeat`, `Docker Compose`, Python log generators, and optional `Wazuh`.
- This repository is an academic SIEM project built around `Elasticsearch`, `Logstash`, `Kibana`, `Filebeat`, `Docker Compose` and Python log generators.
- Follow the project scope, requirements, deliverables, and acceptance criteria defined in [`README.md`](../README.md).

## Source of truth
Expand All @@ -14,10 +14,16 @@ applyTo: "**"

## Communication style
- Use clear, direct, and concise language.
- Do not use emojis.
- Do not use emojis under any circumstance.
- Prefer factual statements over promotional or decorative wording.
- Keep explanations practical and focused on what exists, what is missing, and what should be implemented next.

## Coding approach
- Think before coding: state assumptions explicitly. If something is unclear or multiple interpretations exist, ask instead of guessing.
- Keep solutions minimal: implement only what was requested. Avoid speculative abstractions, configurability, or error handling for impossible scenarios.
- Make surgical changes: touch only what is necessary, match existing style, and do not refactor unrelated code or comments. Clean up only unused imports, variables, or functions created by your own changes.
- Be goal-driven: for multi-step work, outline a brief plan and define how each step will be verified.

## Documentation rules
- Write project documentation in Spanish.
- Keep code, configuration keys, service names, index names, and technical identifiers in English.
Expand Down
144 changes: 144 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,144 @@
name: CI

on:
pull_request:
branches:
- main
push:
branches:
- main
workflow_dispatch:

permissions:
contents: read

jobs:
lint:
name: Lint
runs-on: ubuntu-latest
steps:
- name: Check out repository
uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.11"

- name: Install lint tools
run: |
python -m pip install --upgrade pip
python -m pip install ruff yamllint
sudo apt-get update
sudo apt-get install -y shellcheck

- name: Lint Python
run: ruff check --select E9,F63,F7,F82 setup

- name: Lint shell scripts
run: |
shellcheck -S error \
logstash/test-pipeline.sh \
setup/run-podman-sentinel-stack.sh \
setup/provisioning/*.sh \
setup/verify-cluster.sh

- name: Lint YAML
run: |
yamllint -d '{
extends: relaxed,
rules: {
colons: disable,
document-start: disable,
indentation: disable,
line-length: disable,
truthy: disable
}
}' \
.github/workflows/ci.yml \
docker-compose.yml \
docker-compose-podman.yml \
docker-compose-sentinel.yml \
docker-compose-sentinel-podman.yml \
filebeat/filebeat.yml \
filebeat/filebeat-k8s.yml \
filebeat/filebeat-daemonset.yaml \
kibana/kibana.yml \
logstash/config/logstash.yml \
setup/provisioning/cert-instances.yml

- name: Check line endings
run: |
bad=0
while IFS=$'\t' read -r meta path; do
case "$meta" in
*"w/crlf"*|*"i/crlf"*)
printf 'CRLF detected: %s\t%s\n' "$meta" "$path" >&2
bad=1
;;
esac
done < <(git ls-files --eol)
exit "$bad"

- name: Validate JSON files
run: |
python - <<'PY'
import json
from pathlib import Path

for path in sorted(Path('.').rglob('*.json')):
with path.open(encoding='utf-8') as handle:
json.load(handle)
print(path)
PY

static-tests:
name: Static Tests
runs-on: ubuntu-latest
steps:
- name: Check out repository
uses: actions/checkout@v4

- name: Set up Python
uses: actions/setup-python@v5
with:
python-version: "3.11"

- name: Compile Python scripts
run: python -m compileall setup

- name: Validate dashboard generation
run: |
python setup/create-dashboards-complete.py
git diff --exit-code setup/dashboards/executive-operational-dashboards.ndjson

- name: Validate dashboard references
run: |
python - <<'PY'
import json
from pathlib import Path

path = Path('setup/dashboards/executive-operational-dashboards.ndjson')
objects = [json.loads(line) for line in path.read_text(encoding='utf-8').splitlines() if line.strip()]
ids = {(item['type'], item['id']) for item in objects}
missing = []

for item in objects:
if item.get('type') != 'dashboard':
continue
panels = json.loads(item['attributes']['panelsJSON'])
references = {ref['name']: (ref['type'], ref['id']) for ref in item.get('references', [])}
for panel in panels:
ref = references.get(panel.get('panelRefName'))
if ref not in ids:
missing.append((item['id'], panel.get('panelRefName'), ref))

if missing:
raise SystemExit(f'Missing dashboard references: {missing}')
print(f'Validated {len(objects)} saved objects')
PY

- name: Validate Compose files
run: |
docker compose -f docker-compose.yml config --quiet
docker compose -f docker-compose.yml -f docker-compose-podman.yml config --quiet
10 changes: 10 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# Directorios y archivos a ignorar
setup/__pycache__/
.ruff_cache/

# Ignorar variables de entorno
.env
.env.sentinel

# Logs
logs
Loading
Loading