Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added aws-secret-manager-strategies.docx
Binary file not shown.
197 changes: 197 additions & 0 deletions parameters/PENDING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,197 @@
# Parameters Package — Pending Work

Status snapshot del estado actual del paquete `parameters/` y trabajo pendiente. Para una vista de la arquitectura completa ver `parameters/docs/architecture.md`.

---

## Estado actual

| Componente | Estado |
|---|---|
| Skeleton (entrypoint, build_context, dispatch, utils, workflows) | ✅ Implementado |
| Provider `hashicorp_vault` | ✅ Implementado (migrado del `parameters/vault/` original) |
| Provider `secret_manager` | ✅ Implementado |
| Provider `parameter_store` | ✅ Implementado (nuevo) |
| Provider `azure_key_vault` | ✅ Implementado (nuevo) |
| Error handling (not_found → idempotent, otros → fail loud) | ✅ Aplicado a deletes y retrieves de los 4 providers |
| Tests BATS | ✅ 150 tests pasando |
| Docs globales | ✅ architecture.md, configuration.md, adding_a_provider.md |
| Docs por provider | ✅ architecture.md (4 providers), iam-policy.md (SM + PS) |
| Decision doc para equipo | ✅ `aws-secret-manager-strategies.docx` (en root del repo) |
| Naming NRN+slug-based | ⏳ Pendiente — ver "1. Refactor de naming" |
| `fetch_configuration` por provider | ⏳ Pendiente — ver "2. Placeholders" |

---

## Decisiones tomadas

| Decisión | Valor | Origen |
|---|---|---|
| Estrategia de granularidad | 1:1 mapping (un secret por parámetro) | Review del equipo sobre el decision doc |
| Naming convention | NRN entities con slugs+ids + dimensiones + parameter_id | Conversación de diseño |
| Provider AWS Secrets Manager | Nombre futuro: `aws_secret_manager` (rename pendiente de `secret_manager`) | Conversación de diseño |
| Selector resolution | Env-only (`SECRET_PROVIDER`, `PARAMETER_PROVIDER`) | Limitación del provider-categories de nullplatform |
| Workflow YAMLs | 4 workflows unificados (store, retrieve, delete, notify), sin discriminación por kind | Cleanup arquitectónico |
| Discriminación secret/param | En `build_context` desde `$CONTEXT.secret`, no en entrypoint | Mismo cleanup |
| Logging | Todos los niveles routean a stderr (stdout reservado para JSON) | Bug encontrado durante tests |
| Delete failure semantics | "not found" → success idempotente, todo lo demás → exit 1 con troubleshooting | Feedback de revisión |
| Retrieve failure semantics | Idem delete: "not found" → `{value: "value not found"}`, otros errores → exit 1 | Idem |

---

## Pendiente

### 1. Refactor de naming a NRN+slugs+ids

**Bloqueado por:** falta confirmar la syntax exacta del `np` CLI para obtener slugs de entities por ID.

**Hipótesis (a confirmar antes de implementar):**

```bash
np organization get --id 1255165411 --query slug --output text
```

#### Diseño aprobado

El `external_id` retornado a nullplatform (y por tanto el nombre del secret en cada provider) se compone así:

```
<entity_type>=<slug>-<id>/<entity_type>=<slug>-<id>/.../<dim_key>=<dim_value>/<parameter_id>
```

- Entities iteradas en orden NRN canónico: `organization → account → namespace → application → scope`. Solo se incluyen las presentes.
- Dimensiones ordenadas alfabéticamente por key para garantizar determinismo.
- `parameter_id` al final como identificador único.
- Slugs son inmutables en nullplatform (garantía del contrato), por lo que el external_id no sufre deriva en el tiempo.

#### Ejemplo

Con `entities = {organization: "1255165411", account: "95118862", namespace: "37094320", application: "321402625"}`, `dimensions = {env: "prod"}`, `parameter_id = 42`:

```
organization=acme-1255165411/account=prod-95118862/namespace=billing-37094320/application=api-321402625/env=prod/42
```

#### Pasos

1. Crear `parameters/utils/build_external_id` con fetch paralelo de slugs vía `np` CLI (usando `mktemp` + `&` + `wait`).
2. Refactorizar `store` de los 4 providers:
- `hashicorp_vault/store`: nombre `secret/data/parameters/<external_id>`
- `secret_manager/store`: nombre `<SM_NAME_PREFIX><external_id>`
- `parameter_store/store`: nombre `<PS_NAME_PREFIX><external_id>`
- `azure_key_vault/store`: nombre `<AZ_SECRET_PREFIX><external_id transformado>`. AKV solo permite alfanumérico + `-`, así que transformamos `/` → `-` y removemos `=`.
3. `retrieve`/`delete`/`notify` NO cambian: usan el `EXTERNAL_ID` que llega de nullplatform.
4. Tests: mock de `np` CLI en `$BATS_TEST_TMPDIR/bin/`, expected paths actualizados en cada provider.
5. Update de `parameters/providers/<name>/docs/architecture.md` con el nuevo naming.

#### Edge cases (todos confirmados)

- Entities siempre vienen (parte del contrato de nullplatform) — no hay caso de "entities vacío".
- `np` CLI siempre está disponible (instalado en la imagen Docker base del agente).
- Slugs inmutables — no hay riesgo de deriva o reconstrucción incorrecta.

---

### 2. Placeholders `fetch_configuration` por provider

Cada provider necesita un placeholder `fetch_configuration` (opcional según el contrato pero útil) que populate `PROVIDER_CONFIG` con su config específica desde donde corresponda (np CLI, REST, file, etc.).

Hoy todos los providers funcionan vía env vars (`VAULT_ADDR`, `AWS_REGION`, etc.). El placeholder permite wirear el fetch real cuando el platform team defina el mecanismo.

Estructura sugerida:

```bash
#!/bin/bash
# parameters/providers/<name>/fetch_configuration
#
# TODO(platform-team): wire la lógica de fetch real (np CLI, REST, file montado, etc.)
# Mientras tanto, PROVIDER_CONFIG default a '{}' y todo cae a env vars.

: "${PROVIDER_CONFIG:=}"
if [ -z "$PROVIDER_CONFIG" ]; then
PROVIDER_CONFIG='{}'
fi
export PROVIDER_CONFIG
```

A duplicar en los 4 providers. Build_context ya sourcea `$PROVIDER_DIR/fetch_configuration` si existe.

---

### 3. Rename `secret_manager` → `aws_secret_manager` (opcional)

Decisión tomada en conversación pero no aplicada todavía. No bloqueante para nada. Cuando se haga:

- Mover `parameters/providers/secret_manager/` → `parameters/providers/aws_secret_manager/`
- Update referencias en docs (architecture.md, configuration.md, adding_a_provider.md, iam-policy.md)
- Update tests en `parameters/tests/providers/secret_manager/` (mover y renombrar)
- Actualizar valores aceptables de `SECRET_PROVIDER` / `PARAMETER_PROVIDER` en docs

---

## Contrato del payload — para referencia rápida

Notification de nullplatform tiene estos campos en `$CONTEXT` (después de que el entrypoint extrae `.notification`):

| Campo | Tipo | Acciones | Notas |
|---|---|---|---|
| `parameter_id` | number | store, notify | nullplatform parameter ID |
| `value` | string | store | el valor a persistir |
| `external_id` | string | retrieve, delete, notify | handle generado en store (NRN+slugs+ids+dims+id) |
| `secret` | bool | todas | discriminador secret/parameter (sigue derivando PARAMETER_KIND pero no afecta routing en 1:1) |
| `parameter_name` | string | todas | display name del parámetro |
| `encoding` | string | todas | `plain`, `base64`, etc. |
| `entities` | object | todas | IDs only — slugs se fetchean por separado vía np CLI |
| `value_entities` | object | retrieve (opcional) | Mismo formato que entities, solo presente si el value tiene NRN distinto al parámetro |
| `dimensions` | object | opcional | key-value pairs (env, country, etc.) — ordenarse alfabéticamente |

Las entities siempre vienen como IDs strings:

```json
{
"organization": "1255165411",
"account": "95118862",
"namespace": "37094320",
"application": "321402625"
}
```

---

## Cómo correr los tests

```bash
bats $(find parameters/tests -name "*.bats")
```

Distribución actual (150 tests):

- Skeleton (entrypoint, build_context, dispatch, utils): 55 tests
- hashicorp_vault: 27 tests
- secret_manager: 17 tests
- parameter_store: 23 tests
- azure_key_vault: 15 tests
- utils/log + utils/get_config_value: 13 tests

---

## Estructura del paquete

```
parameters/
├── PENDING.md # este archivo
├── entrypoint, build_context # router + provider resolution
├── store, retrieve, delete, notify # dispatch one-liners
├── workflows/ # 4 YAMLs (acción-only, kind se deriva)
├── utils/
│ ├── get_config_value # priority: provider config > env > default
│ └── log # todos los niveles a stderr
├── providers/
│ ├── README.md # contrato del provider
│ ├── hashicorp_vault/
│ ├── secret_manager/
│ ├── parameter_store/
│ └── azure_key_vault/
├── tests/ # 150 BATS tests
└── docs/ # docs globales del paquete
```
100 changes: 100 additions & 0 deletions parameters/build_context
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
#!/bin/bash
set -euo pipefail

# Resolves which provider implementation handles this workflow run.
#
# Inputs:
# CONTEXT — JSON of the notification body (set by entrypoint)
# PARAMETER_KIND — "secret" | "parameter" (set by workflow `configuration` block;
# falls back to deriving from $CONTEXT.secret if absent — e.g. notify)
# SECRET_PROVIDER — env var: name of provider to use when kind=secret
# PARAMETER_PROVIDER — env var: name of provider to use when kind=parameter
#
# Per-provider config fetching is delegated to providers/<name>/fetch_configuration
# (optional). Each provider owns its own fetching mechanism — no global config
# fetcher in this layer.
#
# Outputs (exported for subsequent workflow steps):
# PARAMETER_KIND, ACTIVE_PROVIDER, PROVIDER_DIR, PARAMETERS_ROOT
# EXTERNAL_ID, PARAMETER_ID, PARAMETER_VALUE, PARAMETER_NAME, PARAMETER_ENCODING
# Plus any vars the provider's fetch_configuration and setup scripts export.

SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
export PARAMETERS_ROOT="$SCRIPT_DIR"

source "$SCRIPT_DIR/utils/log"
source "$SCRIPT_DIR/utils/get_config_value"

export EXTERNAL_ID=$(echo "$CONTEXT" | jq -r '.external_id // empty')
export PARAMETER_ID=$(echo "$CONTEXT" | jq -r '.parameter_id // empty')
export PARAMETER_VALUE=$(echo "$CONTEXT" | jq -r '.value // empty')
export PARAMETER_NAME=$(echo "$CONTEXT" | jq -r '.parameter_name // empty')
export PARAMETER_ENCODING=$(echo "$CONTEXT" | jq -r '.encoding // empty')

if [ -z "${PARAMETER_KIND:-}" ]; then
# `// empty` would swallow `false` (jq's // treats false as missing), so use tostring.
case "$(echo "$CONTEXT" | jq -r '.secret | tostring')" in
true) PARAMETER_KIND="secret" ;;
false) PARAMETER_KIND="parameter" ;;
*) PARAMETER_KIND="" ;;
esac
fi
export PARAMETER_KIND

# Selector resolution is env-only at this layer.
# If the platform wants to derive selectors from provider config, it must
# populate these env vars BEFORE invoking the entrypoint.
case "$PARAMETER_KIND" in
secret)
ACTIVE_PROVIDER="${SECRET_PROVIDER:-}"
selector_env="SECRET_PROVIDER"
;;
parameter)
ACTIVE_PROVIDER="${PARAMETER_PROVIDER:-}"
selector_env="PARAMETER_PROVIDER"
;;
*)
ACTIVE_PROVIDER="${SECRET_PROVIDER:-${PARAMETER_PROVIDER:-}}"
selector_env="SECRET_PROVIDER or PARAMETER_PROVIDER"
;;
esac

if [ -z "$ACTIVE_PROVIDER" ]; then
log error "❌ No provider configured for kind '$PARAMETER_KIND'"
log error ""
log error "💡 Possible causes:"
log error " • $selector_env env var is not set in the workflow runtime"
log error ""
log error "🔧 How to fix:"
log error " • Set $selector_env=<provider_name> in the agent/runner environment"
log error " • Available providers: $(ls "$SCRIPT_DIR/providers" 2>/dev/null | grep -v '^README' | tr '\n' ' ' || true)"
exit 1
fi

PROVIDER_DIR="$SCRIPT_DIR/providers/$ACTIVE_PROVIDER"
if [ ! -d "$PROVIDER_DIR" ]; then
available=$(ls "$SCRIPT_DIR/providers" 2>/dev/null | grep -v '^README' | tr '\n' ' ' || true)
log error "❌ Provider implementation not found: '$ACTIVE_PROVIDER'"
log error ""
log error "🔧 How to fix:"
log error " • Available providers: ${available:-(none installed)}"
log error " • Set $selector_env to one of the above, or add a provider at parameters/providers/$ACTIVE_PROVIDER/"
exit 1
fi
export ACTIVE_PROVIDER
export PROVIDER_DIR

log debug "📦 active_provider=$ACTIVE_PROVIDER kind=$PARAMETER_KIND"

# Each provider owns its config fetching (np CLI, REST call, file, env vars, etc.)
# Optional: if absent, the provider relies on whatever's already in the environment.
if [ -f "$PROVIDER_DIR/fetch_configuration" ]; then
log debug "📡 Sourcing $ACTIVE_PROVIDER/fetch_configuration"
source "$PROVIDER_DIR/fetch_configuration"
fi

# Validation + connection handles. Operations downstream assume invariants hold.
if [ -f "$PROVIDER_DIR/setup" ]; then
log debug "📡 Sourcing $ACTIVE_PROVIDER/setup"
source "$PROVIDER_DIR/setup"
fi
6 changes: 6 additions & 0 deletions parameters/delete
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
#!/bin/bash
set -euo pipefail

# Dispatch: delegate to the active provider's delete implementation.
# PROVIDER_DIR is exported by build_context.
source "$PROVIDER_DIR/delete"
Loading
Loading