Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 20 additions & 6 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

### Changed

- Remove `SkipReason` from RSOP test cases due to resolved merge logic bug.

## [0.41.0] - 2026-02-03

### Added

- Add configurable `default_json_depth` setting in `Datum.yml` to
Expand Down Expand Up @@ -40,15 +34,34 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
RSOP.md.
- Added AllNodes iteration examples for both flat and nested directory
layouts in README.md, RSOP.md, and AboutDatum.md.
- Support conditional `ResolutionPrecedence` entries using
`Datum.InvokeCommand` expressions (`[x= ... =]`). Entries that
evaluate to `$null` or empty strings are now silently skipped
instead of causing lookup errors.
- Added knockout support for hashtable array items.
- Added integration tests for environment variable resolution in
`ResolutionPrecedence` paths
([#126](https://github.com/gaelcolas/datum/pull/126)).

### Changed

- Rewrote README.md with structured sections, table of contents,
installation guide, merge strategy reference, handler documentation,
and public function catalogue.
- Remove `SkipReason` from RSOP test cases due to resolved merge logic bug.
- Adjusted integration tests for knockout of hashtable array items.
- Adjusted integration tests for hashtable array merge behauvior 'Sum'.
- Add integration test for conditional `ResolutionPrecedence` entry
using an InvokeCommand expression that returns a path for some nodes
and nothing for others.

### Fixed

- Fixed `Join-Path` throwing `DriveNotFoundException` when
`ResolutionPrecedence` entries contain environment variable
references such as `$($env:VarName)`, by using
`[System.IO.Path]::Combine` instead
([#126](https://github.com/gaelcolas/datum/pull/126)).
- Fix `ConvertTo-Json` truncation warnings for deep data structures
in `Merge-Datum`, `Merge-Hashtable`, and
`Invoke-TestHandlerAction`
Expand All @@ -61,6 +74,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
`$rsop.SomeKey.__source` pattern.
- Documented `-IncludeSource`/`-RemoveSource` mutual exclusivity in
RSOP.md and CmdletReference.md.
- Fixed merging of hashtable array items using merge behaviour 'Sum'.

## [0.41.0] - 2026-02-03

Expand Down
5 changes: 5 additions & 0 deletions docs/CmdletReference.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,11 @@ Resolve-Datum -PropertyPath 'Configurations' -Variable $Node -DatumTree $Datum
| `PathPrefixes` | `[string[]]` | No | `$DatumTree.__Definition.ResolutionPrecedence` | Resolution paths. Alias: `SearchPaths` |
| `MaxDepth` | `[int]` | No | From definition or `-1` | Maximum merge recursion depth |

> **Note:** `PathPrefixes` are processed through configured datum
> handlers before lookup begins. Entries that resolve to `$null` or
> empty/whitespace strings (e.g. conditional `[x= ... =]` expressions)
> are automatically removed from the search list.

**Output:** `[System.Array]` — The resolved value(s).

---
Expand Down
34 changes: 34 additions & 0 deletions docs/DatumYml.md
Original file line number Diff line number Diff line change
Expand Up @@ -156,6 +156,40 @@ Any property of the node hashtable can be referenced.

> **Tip:** `$Node.Name` is the file name of the node data file (set automatically by the FileProvider). `$Node.NodeName` is typically set inside the data file itself, often using `'[x={ $Node.Name }=]'` via InvokeCommand to derive it from the file name.

#### Conditional Entries with InvokeCommand

Resolution paths are processed through configured datum handlers
**before** lookup begins. This means you can use
`Datum.InvokeCommand` expressions (`[x= ... =]`) to include a path
only when a condition is met:

```yaml
ResolutionPrecedence:
- AllNodes\$($Node.Environment)\$($Node.NodeName)
- Environment\$($Node.Environment)
- Roles\$($Node.Role)
- Baselines\Security
- |
'[x= {
if ($Node.Name -notlike "*file*")
{
"Somevalue"
}
else
{
# returns $null — entry is skipped
}
} =]'
```

When the expression returns `$null` or an empty string, the entry
is silently removed from the search list for that node. This keeps
the lookup clean without requiring separate precedence lists per
node type.

> **Note:** The multi-line YAML block-scalar syntax (`|`) is required
> so the `[x= ... =]` marker spans multiple lines correctly.

#### Path Format

- Use **backslash** (`\`) as the path separator
Expand Down
26 changes: 10 additions & 16 deletions memory-bank/activeContext.md
Original file line number Diff line number Diff line change
@@ -1,28 +1,22 @@
# Active Context

## Current State (as of 2026-02-25)
The project is on the `main` branch. Recent work has focused on **PR #154 improvements** — making the `ConvertFrom-Yaml` consolidation for JSON files more comprehensive with proper CHANGELOG categorization and integration tests for JSON/YAML equivalence.
## Current State (as of 2026-03-07)
The project is on the `feature/docs` branch. Recent work has focused on adding support for **conditional ResolutionPrecedence entries** using `Datum.InvokeCommand` expressions.

## Current Work Focus
The project has **unreleased changes** (tracked in CHANGELOG.md under [Unreleased]) that include:

### Unreleased Features & Fixes
1. **Knockout support for basetype arrays** — New feature allowing items to be removed from inherited arrays
2. **Cleanup of knockout items** — Post-merge cleanup of knockout markers
3. **Pester tests for credential handling** — New test coverage
4. **ConvertTo-Datum fix** — Fixed returning `` when DatumHandler returns `False` (#139)
5. **Merge-DatumArray fix** — Fixed not returning array when merged array contains single hashtable
6. **Hashtable array merge fix** — Fixed items not merging when using datum handler for tuple keys (#155)
7. **Copy-Object fixes** — Fixed and extended tests, PowerShell 7 compatibility
8. **Pester 5 migration** — All integration tests migrated to Pester 5 syntax
9. **Build system update** — Updated to Sampler 0.119.0-preview0005
10. **Merge-DatumArray improvement** — Convert tuple key values to datum before merging
1. **Conditional ResolutionPrecedence entries** — `Resolve-Datum` now filters out null/empty/whitespace path prefixes after datum handler invocation, enabling conditional `[x= ... =]` expressions in `ResolutionPrecedence` that can return nothing for certain nodes.
2. **Integration test for conditional precedence** — Added InvokeCommand expression to test `Datum.yml` that conditionally includes a path only for non-file-server nodes.
3. **SkipReason removal** — Removed `SkipReason` from RSOP test cases due to resolved merge logic bug.

## Recent Changes (Last Session)
- Moved CHANGELOG entry for PR #154 from `Added` to `Changed` with expanded description and PR link
- Created `tests/Integration/GetFileProviderData.tests.ps1` — 33 new tests for JSON/YAML equivalence
- Created test data in `tests/Integration/assets/JsonYamlEquivalence/` (JSON + YAML pairs)
- **Build result**: 191 passed, 0 failed, 3 skipped (pre-existing)
- Added `Where-Object { -not [string]::IsNullOrWhiteSpace($_) }` filter in `Resolve-Datum.ps1` after `ConvertTo-Datum` handler processing of `$PathPrefixes`
- Added conditional InvokeCommand expression to `tests/Integration/assets/DscWorkshopConfigData/Datum.yml`
- Updated CHANGELOG.md with Added and Changed entries under [Unreleased]
- Added "Conditional Entries with InvokeCommand" section to `docs/DatumYml.md`
- Added PathPrefixes filtering note to `docs/CmdletReference.md`

## Next Steps
- ~~**Fix Issue #136**~~: DONE — depth now configurable via `default_json_depth` in Datum.yml (default 4), 8 tests, zero truncation warnings. Build verified, all tests passing. Ready for PR.
Expand Down
16 changes: 16 additions & 0 deletions memory-bank/techContext.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,22 @@
- **Versioning**: GitVersion (Semantic Versioning)
- **Package Distribution**: PowerShell Gallery

## Running Tests
**CRITICAL**: Never use the `runTests` tool or run Pester inside VS Code's integrated PowerShell session — it will hang VS Code. Always run tests in a **separate `pwsh` process**:

```powershell
# Full build + test (Sampler)
pwsh -NoProfile -NonInteractive -Command "./build.ps1 -Tasks test"

# Single test file
pwsh -NoProfile -NonInteractive -Command "Invoke-Pester -Path './tests/Integration/Resolve-Datum.Tests.ps1' -Output Detailed"

# With module pre-loaded (for integration tests that need the built module)
pwsh -NoProfile -NonInteractive -Command "Import-Module ./output/datum/0.0.1/datum.psd1 -Force; Invoke-Pester -Path './tests/Integration/' -Output Detailed"
```

Use `run_in_terminal` with `isBackground = false` and a generous timeout (180000+ ms).

## Dependencies
### Runtime
- `powershell-yaml` — YAML parsing (required module)
Expand Down
40 changes: 16 additions & 24 deletions source/Private/Clear-DatumKnockout.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -117,28 +117,20 @@ function Clear-DatumKnockout
{
if ($strategy.merge_hash_array -match '^MostSpecific$|^First')
{
return $ReferenceDatum
return , $ReferenceDatum
}

if ($knockoutPrefixMatcher = $strategy.merge_options.knockout_prefix)
{
if ($tupleKeyNames = [string[]]$Strategy.merge_options.tuple_keys)
$knockoutPrefixMatcher = [regex]::Escape($Strategy.merge_options.knockout_prefix).Insert(0, '^')

if ($tupleKeyNames = [string[]]$strategy.merge_options.tuple_keys)
{
$tupleKeyNames += $tupleKeyNames.ForEach{ $strategy.merge_options.knockout_prefix + $_ }
$knockoutItems = foreach ($refItem in $ReferenceDatum)
{
$refItemTupleKeys = $refItem.Keys.Where{ $_ -in $tupleKeyNames }
if ($refItemTupleKeys -match $knockoutPrefixMatcher)
if ($refItem.Keys.Where{ $_ -in $tupleKeyNames -and $refItem[$_] -match $knockoutPrefixMatcher })
{
$refItem
$filterScript = @()
foreach ($refItemTupleKey in $refItemTupleKeys)
{
$refItemTupleKeyWithoutKnockoutPrefix = $refItemTupleKey -replace $knockoutPrefixMatcher, ''
$filterScript += "`$_.$refItemTupleKeyWithoutKnockoutPrefix -eq '$($refItem."$refItemTupleKey")'"
}
$filterScript = [scriptblock]::Create($filterScript -join ' -and ')
$ReferenceDatum.Where{ &$filterScript }
}
}
foreach ($knockoutItem in $knockoutItems)
Expand All @@ -148,19 +140,19 @@ function Clear-DatumKnockout
}
}

$cleanedArray = [System.Collections.ArrayList]::new()

foreach ($currentItem in $ReferenceDatum)
{
$cleanupItemParams = @{
StartingPath = $StartingPath
ReferenceDatum = $currentItem
Strategies = $Strategies
$cleanedArray = @(
foreach ($currentItem in $ReferenceDatum)
{
$cleanupItemParams = @{
StartingPath = $StartingPath
ReferenceDatum = $currentItem
Strategies = $Strategies
}
Clear-DatumKnockout @cleanupItemParams
}
$null = $cleanedArray.Add((Clear-DatumKnockout @cleanupItemParams))
}
)

return (, $cleanedArray)
return , $cleanedArray
}

default
Expand Down
Loading