Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion docs/Pennington.Docs/CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,14 +99,21 @@ src/Pennington/Pipeline/ContentPipeline.cs > ContentPipeline.ParseAsync
### Diff two members — `<lang>:symbol-diff`
Body must contain exactly 2 references (before/after), one per line. Supports the same `,bodyonly` suffix.

Two more `:symbol` flags, comma-combinable with `,bodyonly`:

- **`,imports`** — prepends the file's top-of-file `using` / `import` / `require` statements above the snippet. Prepends *all* of them, not only the ones the member uses; no-op for whole-file embeds and import-less languages (Ruby).
- **`,signatures`** — elides member bodies to `{ … }` (or `…` for non-brace bodies) for an outline of a type's shape. Inverse of `,bodyonly`, so the two don't combine.

### When to use which
- **`:symbol`** — the default. A bare path embeds the whole file; `path > Type.Member` embeds one member. Language-agnostic.
- **`:symbol,bodyonly`** — when the declaration is noise (showing what's inside a method, or a type's members without its header).
- **`:symbol,imports`** — when a member snippet needs its import context to make sense on its own.
- **`:symbol,signatures`** — when the point is a type's surface (its members) rather than how each one works.
- **`:symbol-diff`** — before/after comparisons in explanation pages.

### Caveats
- **Overloads** resolve to the first declaration of that name in the file — name-path addressing can't distinguish signatures. Point at an unambiguous member, or hand-write the snippet.
- **No `using` prepend.** Tree-sitter is syntactic; there is no `,usings` equivalent. Snippets render without import directives.
- **`,imports` is unfiltered.** It prepends every top-of-file import, not just the ones the snippet references — trim by hand when a member needs only a few.

Requires `Pennington.TreeSitter` wired (`AddPenningtonTreeSitter`) with `ContentRoot` set.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,24 @@ examples/FocusedCodeSamplesExample/MonolithWordCounter.cs > MonolithWordCounter.

`,bodyonly` also works on types (members between the braces, skipping the type header) and properties (the `get`/`set` accessors).

## Carry the imports with `,imports`

A member-scoped fence drops the file's `using` / `import` / `require` lines, so a reader can't see where the types resolve from. Append `,imports` to prepend the file's top-of-file imports above the snippet, separated by a blank line.

````markdown
```csharp:symbol,imports
examples/FocusedCodeSamplesExample/MonolithWordCounter.cs > MonolithWordCounter.CountWords
```
````

Which renders as:

```csharp:symbol,imports
examples/FocusedCodeSamplesExample/MonolithWordCounter.cs > MonolithWordCounter.CountWords
```

`,imports` prepends every import at the top of the file, not only the ones the member references. It composes with `,bodyonly` — the imports lead, the body follows — and is a no-op for whole-file embeds and for languages with no syntactic import statement (Ruby's `require` is a method call, not a declaration).

## Walk a multi-phase method through named helpers

When the target method runs 25+ lines across distinct phases, fence each phase as its own helper instead of fencing the monolith. `ModularWordCounter` is the same logic as `MonolithWordCounter` split into three helpers — `Tokenize`, `Tally`, and `Format` — orchestrated by a short `CountWords`. A whole-type fence gives the reader the full picture in one place:
Expand Down Expand Up @@ -125,6 +143,24 @@ examples/FocusedCodeSamplesExample/ModularWordCounter.cs > ModularWordCounter.Fo

`:symbol` resolves a member by name path within the file, so give each helper a distinct name — overloads resolve to the first declaration and can't be told apart.

## Show a type's shape with `,signatures`

When the point is a type's surface — what it exposes, not how each member works — append `,signatures`. Every member body collapses to `{ … }`, leaving the declarations, signatures, doc comments, and member order intact for an at-a-glance outline.

````markdown
```csharp:symbol,signatures
examples/FocusedCodeSamplesExample/ModularWordCounter.cs > ModularWordCounter
```
````

Which renders as:

```csharp:symbol,signatures
examples/FocusedCodeSamplesExample/ModularWordCounter.cs > ModularWordCounter
```

`,signatures` works on a single member too, rendering just its signature over an elided body. It targets brace-delimited languages (C#, Java, TypeScript, Go, Rust); Python and Ruby suites collapse to a best-effort `…`. As the inverse of `,bodyonly`, the two don't combine — `,signatures` wins when both are set.

## Show a delta with `symbol-diff`

When the article's point is that one version replaces another — a small refactor, a migration, a perf tweak — fence both versions with `symbol-diff`. The preprocessor emits a unified diff so the reader sees the two or three lines that moved rather than comparing two fences by eye. The form works best when the delta is small; whole-method rewrites render every line as changed and bury the point.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,9 @@ The fence info-string grammar: the opening-fence text after the three backticks,
```text
info-string := language [ ":" suffix ] ( WS attribute )*
language := IDENT
suffix := "symbol" [ ",bodyonly" ]
suffix := "symbol" symbol-flag*
| "symbol-diff" [ ",bodyonly" ]
symbol-flag := ",bodyonly" | ",imports" | ",signatures"
attribute := key "=" value
key := IDENT
value := bare-value | "'" quoted-value "'" | '"' quoted-value '"'
Expand All @@ -38,9 +39,11 @@ quoted-value := any chars up to the matching quote
|---|---|---|
| `<lang>:symbol` | one `<file>` path, optionally followed by `> Member.Path`, per line | Embeds the whole file, or the named member's declaration and body. Concatenated in order. |
| `<lang>:symbol,bodyonly` | same as `:symbol` | Embeds only the member body, stripping the declaration line and enclosing braces. |
| `<lang>:symbol,imports` | same as `:symbol` | Prepends the file's top-of-file import/using/require statements above the snippet. Composes with `,bodyonly`; a no-op for whole-file embeds and import-less languages. |
| `<lang>:symbol,signatures` | same as `:symbol` | Replaces member bodies with an elision marker (`{ … }`, or `…` for non-brace bodies) for an outline view. Mutually exclusive with `,bodyonly`. |
| `<lang>:symbol-diff` | exactly two references, before then after | Emits a unified diff between the two members' source text. Accepts the `,bodyonly` suffix. |

Suffix forms are resolved by an `ICodeBlockPreprocessor`; `Pennington.TreeSitter` ships the implementations for `symbol` and `symbol-diff`.
The `symbol` flags (`,bodyonly`, `,imports`, `,signatures`) are comma-separated and order-independent. Suffix forms are resolved by an `ICodeBlockPreprocessor`; `Pennington.TreeSitter` ships the implementations for `symbol` and `symbol-diff`.

## `[!code …]` directives

Expand Down
47 changes: 43 additions & 4 deletions src/Pennington.TreeSitter/Fragments/FragmentExtractor.cs
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,18 @@ namespace Pennington.TreeSitter.Fragments;
internal static class FragmentExtractor
{
/// <summary>
/// Returns the declaration's full source text, or its body when <paramref name="bodyOnly"/> is set, dedented
/// so the fragment renders at column zero.
/// Returns the declaration's source text, dedented so the fragment renders at column zero. Honors
/// <paramref name="options"/>: <see cref="FragmentOptions.SignaturesOnly"/> elides member bodies (outline
/// view), otherwise <see cref="FragmentOptions.BodyOnly"/> returns just the body.
/// </summary>
public static string Extract(TsNode node, LanguageDeclarationConfig config, bool bodyOnly)
public static string Extract(TsNode node, LanguageDeclarationConfig config, FragmentOptions options)
{
if (!bodyOnly)
if (options.SignaturesOnly)
{
return ExtractSignatures(node, config);
}

if (!options.BodyOnly)
{
return DedentByColumn(node.Text, node.StartPosition.Column);
}
Expand All @@ -35,6 +41,39 @@ public static string Extract(TsNode node, LanguageDeclarationConfig config, bool
return DedentByMin(text);
}

/// <summary>
/// Renders the node as an outline: each member's body (or, for a single member, its own body) is replaced
/// with an elision marker — <c>{ … }</c> for brace-delimited bodies, <c>…</c> otherwise — leaving signatures,
/// member order, and the enclosing declaration intact.
/// </summary>
private static string ExtractSignatures(TsNode node, LanguageDeclarationConfig config)
{
var members = TreeWalker
.ChildrenMatching(node, config.TransparentNodeTypes, config.DeclarationNodeTypes)
.ToList();

// A type elides its members' bodies; a lone member (no nested declarations) elides its own body.
var targets = members.Count > 0 ? members : [node];

var bodies = targets
.Select(target => target.GetChildForField(config.BodyFieldName))
.Where(body => body is not null)
.Select(body => body!.Text.Replace("\r\n", "\n"))
.Where(text => text.Length > 0)
.Distinct()
.OrderByDescending(text => text.Length)
.ToList();

var outline = node.Text.Replace("\r\n", "\n");
foreach (var body in bodies)
{
var marker = body.TrimStart().StartsWith('{') ? "{ … }" : "…";
outline = outline.Replace(body, marker);
}

return DedentByColumn(outline, node.StartPosition.Column);
}

/// <summary>
/// Removes up to <paramref name="column"/> leading whitespace characters from every line after the first.
/// The node's own first line already starts at the declaration, so only continuation lines carry the
Expand Down
17 changes: 17 additions & 0 deletions src/Pennington.TreeSitter/Fragments/FragmentOptions.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
namespace Pennington.TreeSitter.Fragments;

/// <summary>Per-reference extraction flags parsed from a <c>:symbol</c> info-string.</summary>
public sealed record FragmentOptions
{
/// <summary>Emit only the declaration's body, stripping the signature and enclosing braces.</summary>
public bool BodyOnly { get; init; }

/// <summary>Prepend the file's top-of-file import/using/require statements to the fragment.</summary>
public bool IncludeImports { get; init; }

/// <summary>Render the node with member bodies replaced by an elision marker (outline view).</summary>
public bool SignaturesOnly { get; init; }

/// <summary>The default: full declaration text, no imports, no elision.</summary>
public static FragmentOptions Default { get; } = new();
}
6 changes: 3 additions & 3 deletions src/Pennington.TreeSitter/Fragments/ISourceFragmentService.cs
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,10 @@ public interface ISourceFragmentService
{
/// <summary>
/// Returns the source text of <paramref name="namePath"/> within <paramref name="relativeFilePath"/>, or a
/// failure. An empty <paramref name="namePath"/> returns the whole file. When <paramref name="bodyOnly"/> is
/// set, only the matched declaration's body is returned.
/// failure. An empty <paramref name="namePath"/> returns the whole file. <paramref name="options"/> select
/// body-only extraction, an elided-body outline, and whether the file's imports are prepended.
/// </summary>
FragmentResult GetFragment(string languageId, string relativeFilePath, string namePath, bool bodyOnly);
FragmentResult GetFragment(string languageId, string relativeFilePath, string namePath, FragmentOptions options);
}

/// <summary>Outcome of a fragment lookup: either the extracted <see cref="Text"/> or an <see cref="Error"/> message.</summary>
Expand Down
26 changes: 26 additions & 0 deletions src/Pennington.TreeSitter/Fragments/ImportCollector.cs
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
namespace Pennington.TreeSitter.Fragments;

using Resolution;
using TsNode = global::TreeSitter.Node;

/// <summary>Collects a file's top-of-file import/using/require statements for prepending to a fragment.</summary>
internal static class ImportCollector
{
/// <summary>
/// Returns the source text of every import node under <paramref name="root"/> in document order, joined by
/// newlines, or an empty string when the language declares no import node types or the file has none.
/// </summary>
public static string Collect(TsNode root, LanguageDeclarationConfig config)
{
if (config.ImportNodeTypes.Count == 0)
{
return string.Empty;
}

var imports = TreeWalker
.ChildrenMatching(root, config.TransparentNodeTypes, config.ImportNodeTypes)
.Select(node => node.Text.Replace("\r\n", "\n"));

return string.Join("\n", imports);
}
}
14 changes: 12 additions & 2 deletions src/Pennington.TreeSitter/Fragments/SourceFragmentService.cs
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ public SourceFragmentService(TreeSitterOptions options, ITreeSitterParserPool po
_resolver = resolver;
}

public FragmentResult GetFragment(string languageId, string relativeFilePath, string namePath, bool bodyOnly)
public FragmentResult GetFragment(string languageId, string relativeFilePath, string namePath, FragmentOptions options)
{
if (string.IsNullOrEmpty(_options.ContentRoot))
{
Expand Down Expand Up @@ -63,6 +63,16 @@ public FragmentResult GetFragment(string languageId, string relativeFilePath, st
return FragmentResult.Fail($"Member '{namePath}' not found in {relativeFilePath}.");
}

return FragmentResult.Ok(FragmentExtractor.Extract(node, config, bodyOnly));
var fragment = FragmentExtractor.Extract(node, config, options);
if (options.IncludeImports)
{
var imports = ImportCollector.Collect(tree.RootNode, config);
if (imports.Length > 0)
{
fragment = imports + "\n\n" + fragment;
}
}

return FragmentResult.Ok(fragment);
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -39,24 +39,25 @@ public TreeSitterCodeBlockPreprocessor(
/// <inheritdoc />
public CodeBlockPreprocessResult? TryProcess(string code, string languageId)
{
var (baseLanguage, modifier, bodyOnly) = ParseLanguageId(languageId);
var (baseLanguage, modifier, options) = ParseLanguageId(languageId);
return modifier switch
{
"symbol" => ProcessSymbol(baseLanguage, code, bodyOnly),
"symbol-diff" => ProcessSymbolDiff(baseLanguage, code, bodyOnly),
"symbol" => ProcessSymbol(baseLanguage, code, options),
// Imports and outline elision make no sense across a diff; honor only the body-only flag.
"symbol-diff" => ProcessSymbolDiff(baseLanguage, code, new FragmentOptions { BodyOnly = options.BodyOnly }),
_ => null,
};
}

private CodeBlockPreprocessResult ProcessSymbol(string baseLanguage, string code, bool bodyOnly)
private CodeBlockPreprocessResult ProcessSymbol(string baseLanguage, string code, FragmentOptions options)
{
try
{
var fragments = new List<string>();
foreach (var line in code.Split('\n', StringSplitOptions.RemoveEmptyEntries | StringSplitOptions.TrimEntries))
{
var (filePath, namePath) = ParseReference(line);
var result = _fragmentService.GetFragment(baseLanguage, filePath, namePath, bodyOnly);
var result = _fragmentService.GetFragment(baseLanguage, filePath, namePath, options);
if (!result.Succeeded)
{
Diagnostics?.AddWarning($"tree-sitter :symbol — {result.Error}");
Expand All @@ -80,7 +81,7 @@ private CodeBlockPreprocessResult ProcessSymbol(string baseLanguage, string code
}
}

private CodeBlockPreprocessResult ProcessSymbolDiff(string baseLanguage, string code, bool bodyOnly)
private CodeBlockPreprocessResult ProcessSymbolDiff(string baseLanguage, string code, FragmentOptions options)
{
try
{
Expand All @@ -92,8 +93,8 @@ private CodeBlockPreprocessResult ProcessSymbolDiff(string baseLanguage, string
return new CodeBlockPreprocessResult(errorHtml, baseLanguage, SkipTransform: true);
}

var fragment1 = ResolveFragment(baseLanguage, references[0], bodyOnly);
var fragment2 = ResolveFragment(baseLanguage, references[1], bodyOnly);
var fragment1 = ResolveFragment(baseLanguage, references[0], options);
var fragment2 = ResolveFragment(baseLanguage, references[1], options);

if (!fragment1.Succeeded || !fragment2.Succeeded)
{
Expand Down Expand Up @@ -136,18 +137,18 @@ private CodeBlockPreprocessResult ProcessSymbolDiff(string baseLanguage, string
}
}

private FragmentResult ResolveFragment(string baseLanguage, string reference, bool bodyOnly)
private FragmentResult ResolveFragment(string baseLanguage, string reference, FragmentOptions options)
{
var (filePath, namePath) = ParseReference(reference);
return _fragmentService.GetFragment(baseLanguage, filePath, namePath, bodyOnly);
return _fragmentService.GetFragment(baseLanguage, filePath, namePath, options);
}

/// <summary>
/// Splits an info-string such as <c>python:symbol,bodyonly</c> or <c>python:symbol-diff</c> into its base
/// language, the modifier (<c>symbol</c>, <c>symbol-diff</c>, or null when absent), and whether the
/// <c>bodyonly</c> flag is present.
/// language, the modifier (<c>symbol</c>, <c>symbol-diff</c>, or null when absent), and the
/// <see cref="FragmentOptions"/> parsed from the comma-separated flag tail.
/// </summary>
internal static (string baseLanguage, string? modifier, bool bodyOnly) ParseLanguageId(string languageId)
internal static (string baseLanguage, string? modifier, FragmentOptions options) ParseLanguageId(string languageId)
{
var trimmed = languageId.Trim();
const string diffMarker = ":symbol-diff";
Expand All @@ -157,22 +158,29 @@ internal static (string baseLanguage, string? modifier, bool bodyOnly) ParseLang
var diffIndex = trimmed.IndexOf(diffMarker, StringComparison.OrdinalIgnoreCase);
if (diffIndex >= 0)
{
return (trimmed[..diffIndex], "symbol-diff", HasBodyOnly(trimmed[(diffIndex + diffMarker.Length)..]));
return (trimmed[..diffIndex], "symbol-diff", ParseFlags(trimmed[(diffIndex + diffMarker.Length)..]));
}

var index = trimmed.IndexOf(marker, StringComparison.OrdinalIgnoreCase);
if (index < 0)
{
return (trimmed, null, false);
return (trimmed, null, FragmentOptions.Default);
}

return (trimmed[..index], "symbol", HasBodyOnly(trimmed[(index + marker.Length)..]));
return (trimmed[..index], "symbol", ParseFlags(trimmed[(index + marker.Length)..]));
}

private static bool HasBodyOnly(string afterMarker) =>
afterMarker
.Split(',', StringSplitOptions.RemoveEmptyEntries | StringSplitOptions.TrimEntries)
.Any(flag => flag.Equals("bodyonly", StringComparison.OrdinalIgnoreCase));
/// <summary>Reads the comma-separated flag tail after a <c>:symbol</c> marker into a <see cref="FragmentOptions"/>; unknown flags are ignored.</summary>
private static FragmentOptions ParseFlags(string afterMarker)
{
var flags = afterMarker.Split(',', StringSplitOptions.RemoveEmptyEntries | StringSplitOptions.TrimEntries);
return new FragmentOptions
{
BodyOnly = flags.Any(flag => flag.Equals("bodyonly", StringComparison.OrdinalIgnoreCase)),
IncludeImports = flags.Any(flag => flag.Equals("imports", StringComparison.OrdinalIgnoreCase)),
SignaturesOnly = flags.Any(flag => flag.Equals("signatures", StringComparison.OrdinalIgnoreCase)),
};
}

/// <summary>Splits a body line into a file path and member name path on the <c>" &gt; "</c> separator; a bare path yields an empty name path.</summary>
internal static (string filePath, string namePath) ParseReference(string line)
Expand Down
Loading
Loading