Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 46 additions & 0 deletions design/vaults.md
Original file line number Diff line number Diff line change
Expand Up @@ -194,6 +194,52 @@ keyData: ${{ vault.get('aws', 'machineKeyData') }}

The expression evaluation system resolves these at runtime.

## Sensitive Method Arguments

The same `z.meta({ sensitive: true })` annotation applies to **method input
argument schemas**, not just output resource schemas. When a method argument field
is marked sensitive, the framework:

1. Registers all resolved values from that field with `SecretRedactor` before
executing the method — scrubbing them from the per-run log file automatically.
2. Applies the redactor when writing result resource attributes — scrubbing
sensitive values even if the extension model writes them into result attributes.
3. Redacts the field to `"***"` in the auto-generated method summary reports
(both Markdown and JSON variants).

### Marking an Argument Field as Sensitive

```typescript
methods: {
exec: {
description: "Run a command in the container",
arguments: z.object({
// command may contain credentials — mark it sensitive
command: z.array(z.string()).meta({ sensitive: true }),
workdir: z.string().optional(),
}),
execute: async (args, context) => { ... },
},
},
```

String and string-array values are both supported. For array fields, each element
is individually registered with the redactor so any occurrence of any element in
log output is scrubbed.

### Behavior Comparison

| Location | Output schema `sensitive: true` | Argument schema `sensitive: true` |
| -------- | ------------------------------- | --------------------------------- |
| Result resource attributes | Stored in vault, replaced with vault ref | Scrubbed by redactor at write time |
| Per-run log file | Vault secrets scrubbed | Argument values scrubbed |
| Method summary report | Rendered as vault ref | Rendered as `***` |
| Audit log | Not covered | Not covered (see follow-up #244) |

Use output-schema sensitive marking when the value must be retrievable later via
`vault.get()`. Use argument-schema sensitive marking when the value is a
short-lived credential that should never be stored anywhere.

## AWS Secrets Manager Provider

The AWS Secrets Manager provider is the initial implementation supporting:
Expand Down
1 change: 1 addition & 0 deletions src/domain/drivers/raw_execution_driver.ts
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,7 @@ export class RawExecutionDriver implements ExecutionDriver {
this.context.vaultService,
this.methodName,
this.context.onEvent,
this.context.redactor,
);

const {
Expand Down
6 changes: 5 additions & 1 deletion src/domain/models/data_writer.ts
Original file line number Diff line number Diff line change
Expand Up @@ -433,6 +433,7 @@ export function createResourceWriter(
vaultService?: VaultService,
methodName?: string,
onEvent?: (event: MethodExecutionEvent) => void,
redactor?: SecretRedactor,
): {
writeResource: (
specName: string,
Expand Down Expand Up @@ -597,7 +598,10 @@ export function createResourceWriter(
resolvedOptions,
);

const handle = await writer.writeText(JSON.stringify(data));
const serialized = redactor
? redactor.redact(JSON.stringify(data))
: JSON.stringify(data);
const handle = await writer.writeText(serialized);
handles.push(handle);
return handle;
};
Expand Down
4 changes: 4 additions & 0 deletions src/domain/models/method_execution_service.ts
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,7 @@ interface PersistContext extends
| "tagOverrides"
| "runtimeTags"
| "vaultService"
| "redactor"
> {
resources: Record<string, ResourceOutputSpec>;
files: Record<string, FileOutputSpec>;
Expand Down Expand Up @@ -125,6 +126,8 @@ async function processDriverOutputs(
persistContext.definitionName,
persistContext.vaultService,
persistContext.methodName,
undefined, // onEvent
persistContext.redactor,
);
// Shape the raw content into resource data.
// If content is valid JSON, use it directly.
Expand Down Expand Up @@ -747,6 +750,7 @@ export class DefaultMethodExecutionService implements MethodExecutionService {
definitionName: currentDefinition.name,
vaultService: context.vaultService,
methodName,
redactor: context.redactor,
});
result = { dataHandles: currentHandles };
}
Expand Down
43 changes: 43 additions & 0 deletions src/domain/models/sensitive_field_extractor.ts
Original file line number Diff line number Diff line change
Expand Up @@ -179,6 +179,49 @@ export function extractSensitiveFields(
return results;
}

/**
* Extracts the runtime secret values from a data object based on its Zod schema.
*
* For each field marked `{ sensitive: true }` in the schema:
* - String values are collected directly.
* - Array values have each string element collected individually.
* - Undefined or null values are skipped.
* - Object values are skipped (nested object fields are found via recursion).
*
* Used to register sensitive argument values with SecretRedactor before
* method execution so they are scrubbed from log files and result resources.
*
* @param schema - A Zod schema (typically a method argument schema)
* @param data - The resolved data object to extract values from
* @returns Array of secret string values to register with SecretRedactor
*/
export function extractSensitiveFieldValues(
schema: z.ZodTypeAny,
data: Record<string, unknown>,
): string[] {
const fields = extractSensitiveFields(schema);
const secrets: string[] = [];

for (const field of fields) {
const value = getNestedValue(data, field.path);
if (value === undefined || value === null) {
continue;
}
if (typeof value === "string") {
secrets.push(value);
} else if (Array.isArray(value)) {
for (const element of value) {
if (typeof element === "string") {
secrets.push(element);
}
}
}
// Object values are skipped — nested fields are found by extractSensitiveFields recursion
}

return secrets;
}

/**
* Gets a nested value from an object by dot-separated path.
*/
Expand Down
83 changes: 83 additions & 0 deletions src/domain/models/sensitive_field_extractor_test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ import { assertEquals } from "@std/assert";
import { z } from "zod";
import {
extractSensitiveFields,
extractSensitiveFieldValues,
getNestedValue,
setNestedValue,
} from "./sensitive_field_extractor.ts";
Expand Down Expand Up @@ -146,6 +147,88 @@ Deno.test("extractSensitiveFields: deeply nested", () => {
assertEquals(fields[0].path, "level1.level2.secret");
});

Deno.test("extractSensitiveFieldValues: string field", () => {
const schema = z.object({
apiKey: z.string().meta({ sensitive: true }),
name: z.string(),
});
const data = { apiKey: "s3cr3t", name: "test" };
assertEquals(extractSensitiveFieldValues(schema, data), ["s3cr3t"]);
});

Deno.test("extractSensitiveFieldValues: array field collects each element", () => {
const schema = z.object({
command: z.array(z.string()).meta({ sensitive: true }),
name: z.string(),
});
const data = { command: ["sh", "-c", "echo TOKEN_HERE"], name: "test" };
assertEquals(extractSensitiveFieldValues(schema, data), [
"sh",
"-c",
"echo TOKEN_HERE",
]);
});

Deno.test("extractSensitiveFieldValues: undefined field is skipped", () => {
const schema = z.object({
apiKey: z.string().meta({ sensitive: true }).optional(),
});
const data: Record<string, unknown> = {};
assertEquals(extractSensitiveFieldValues(schema, data), []);
});

Deno.test("extractSensitiveFieldValues: null field is skipped", () => {
const schema = z.object({
apiKey: z.string().nullable().meta({ sensitive: true }),
});
const data = { apiKey: null };
assertEquals(extractSensitiveFieldValues(schema, data), []);
});

Deno.test("extractSensitiveFieldValues: non-sensitive fields ignored", () => {
const schema = z.object({
apiKey: z.string().meta({ sensitive: true }),
region: z.string(),
});
const data = { apiKey: "s3cr3t", region: "us-east-1" };
assertEquals(extractSensitiveFieldValues(schema, data), ["s3cr3t"]);
});

Deno.test("extractSensitiveFieldValues: nested sensitive field", () => {
const schema = z.object({
credentials: z.object({
token: z.string().meta({ sensitive: true }),
}),
});
const data = { credentials: { token: "tok_abc" } };
assertEquals(extractSensitiveFieldValues(schema, data), ["tok_abc"]);
});

Deno.test("extractSensitiveFieldValues: multiple sensitive fields", () => {
const schema = z.object({
apiKey: z.string().meta({ sensitive: true }),
secret: z.string().meta({ sensitive: true }),
name: z.string(),
});
const data = { apiKey: "key123", secret: "sec456", name: "test" };
const values = extractSensitiveFieldValues(schema, data);
assertEquals(values.sort(), ["key123", "sec456"]);
});

Deno.test("extractSensitiveFieldValues: array with non-string elements skips them", () => {
const schema = z.object({
items: z.array(z.unknown()).meta({ sensitive: true }),
});
const data = { items: ["str", 42, null, "other"] };
assertEquals(extractSensitiveFieldValues(schema, data), ["str", "other"]);
});

Deno.test("extractSensitiveFieldValues: empty schema returns empty", () => {
const schema = z.object({});
const data = {};
assertEquals(extractSensitiveFieldValues(schema, data), []);
});

Deno.test("getNestedValue: simple path", () => {
const obj = { apiKey: "secret-value" };
assertEquals(getNestedValue(obj, "apiKey"), "secret-value");
Expand Down
77 changes: 77 additions & 0 deletions src/domain/reports/report_execution_service_test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1316,3 +1316,80 @@ Deno.test("executeReports: already-loaded reports are not re-promoted", async ()
assertEquals(summary.results.length, 1);
assertEquals(summary.results[0].success, true);
});

Deno.test("buildRedactSensitiveArgs: redacts array-typed sensitive method args to ***", async () => {
const typeName = "@test-redact/sensitive-array";
const modelType = ModelType.create(typeName);
if (!modelRegistry.has(modelType)) {
modelRegistry.register({
type: modelType,
version: "2026.01.01.1",
globalArguments: z.object({}),
methods: {
exec: {
description: "test exec",
arguments: z.object({
command: z.array(z.string()).meta({ sensitive: true }),
name: z.string(),
}),
execute: () => Promise.resolve({ dataHandles: [] }),
},
},
});
}

const methodArgs = {
command: ["sh", "-c", "echo TOKEN_HERE | base64 -d"],
name: "test-step",
};
const { report, getResults } = makeRedactionCapturingReport({}, methodArgs);

const registry = new ReportRegistry();
registry.register("redaction-test-array", report);
const { repo } = createInMemoryDataRepo();

const context: MethodReportContext = {
scope: "method",
repoDir: "/tmp/test",
logger: {
debug: () => {},
info: () => {},
warn: () => {},
error: () => {},
fatal: () => {},
} as unknown as MethodReportContext["logger"],
// deno-lint-ignore no-explicit-any
dataRepository: repo as any,
// deno-lint-ignore no-explicit-any
definitionRepository: {} as any,
modelType,
modelId: "test-id",
definition: { id: "test-id", name: "test", version: 1, tags: {} },
globalArgs: {},
methodArgs,
methodName: "exec",
executionStatus: "succeeded",
dataHandles: [],
extensionFile: () => {
throw new Error("extensionFile not stubbed in this test");
},
};

await executeReports(
registry,
context,
modelType,
"test-id",
{ require: ["redaction-test-array"] },
{},
undefined,
"exec",
);

const results = getResults();
assertEquals(results !== null, true);
// The entire array is replaced with "***"
assertEquals(results!.redactedMethod.command, "***");
// Non-sensitive fields are preserved
assertEquals(results!.redactedMethod.name, "test-step");
});
29 changes: 29 additions & 0 deletions src/domain/workflows/execution_service.ts
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,7 @@ import {
RepoMarkerRepository,
} from "../../infrastructure/persistence/repo_marker_repository.ts";
import { createRepoMarkerLoader } from "../../infrastructure/persistence/repo_marker_loader.ts";
import { extractSensitiveFieldValues } from "../models/sensitive_field_extractor.ts";

/**
* Context for step execution.
Expand Down Expand Up @@ -662,6 +663,34 @@ export class DefaultStepExecutor implements StepExecutor {
driverConfig: evaluatedDefinition.driverConfig,
});

// Register sensitive argument values with the workflow redactor so they
// are scrubbed from the workflow log file. Must use post-vault-resolution
// values from evaluatedDefinition.
if (ctx.secretRedactor) {
const globalArgSchema = modelDef.globalArguments;
if (globalArgSchema) {
for (
const secret of extractSensitiveFieldValues(
globalArgSchema,
evaluatedDefinition.globalArguments,
)
) {
ctx.secretRedactor.addSecret(secret);
}
}
const methodArgSchema = modelDef.methods[task.methodName]?.arguments;
if (methodArgSchema) {
for (
const secret of extractSensitiveFieldValues(
methodArgSchema,
evaluatedDefinition.getMethodArguments(task.methodName),
)
) {
ctx.secretRedactor.addSecret(secret);
}
}
}

// Execute the method with the EVALUATED definition. The logger
// handles both console and file persistence via RunFileSink. Data
// is persisted by DataWriter during execution — no double-save.
Expand Down
Loading
Loading