From 034755fd86976426e4c2b9c32d6e7548a52c4310 Mon Sep 17 00:00:00 2001 From: harehare Date: Wed, 24 Jun 2026 23:44:30 +0900 Subject: [PATCH 1/2] =?UTF-8?q?=F0=9F=93=9D=20docs(skills):=20consolidate?= =?UTF-8?q?=20processing-markdown=20skill=20reference=20into=20SKILL.md?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit REFERENCE.md's CLI options and function list fully duplicated mq --help and mq --doc output, which drift out of sync as the binary evolves. Point to those live commands instead, and fold the remaining markdown-specific content (node attributes, MDX nodes, env vars) into SKILL.md, dropping REFERENCE.md. --- skills/processing-markdown/REFERENCE.md | 184 ------------------------ skills/processing-markdown/SKILL.md | 80 +++++++---- 2 files changed, 56 insertions(+), 208 deletions(-) delete mode 100644 skills/processing-markdown/REFERENCE.md diff --git a/skills/processing-markdown/REFERENCE.md b/skills/processing-markdown/REFERENCE.md deleted file mode 100644 index b12f17507..000000000 --- a/skills/processing-markdown/REFERENCE.md +++ /dev/null @@ -1,184 +0,0 @@ -# mq Reference - -- All function calls require parentheses `()`. -- If a function is called with missing arguments, the piped value (`|`) is used as the first argument. - -## Full CLI Options - -``` -mq [OPTIONS] [QUERY OR FILE] [FILES]... [COMMAND] - -Commands: - repl Start interactive REPL session -``` - -| Flag | Purpose | -| --------------------------------- | ------------------------------------------------------------------------------- | -| `-A, --aggregate` | Combine all inputs into single array | -| `-I, --input-format` | `markdown`, `mdx`, `html`, `text`, `null`, `raw`, `json`, `toml`, `yaml`, `xml`, `csv`, `tsv`, `psv`, `bytes`, `cbor` | -| `-F, --output-format` | `markdown` (default), `html`, `text`, `json`, `table`, `grep`, `raw`, `none` | -| `-U, --update` | Update file in place (aliases: `-i`, `--in-place`, `--inplace`) | -| `-f, --from-file` | Load query from `.mq` file | -| `-o, --output FILE` | Write to output file | -| `-S, --separator QUERY` | Insert query result between files | -| `--args NAME VALUE` | Set runtime variable (aliases: `--arg`, `--define`) | -| `--rawfile NAME FILE` | Load file contents into variable | -| `--stream` | Process line by line (streaming mode) | -| `--unbuffered` | Unbuffered output | -| `-C, --color-output` | Colorize output (supports JSON format too) | -| `-P THRESHOLD` | Parallel processing threshold (default: 10) | -| `-B, --before-context NUM` | Show NUM nodes before each match (grep mode only) | -| `--after-context NUM` | Show NUM nodes after each match (grep mode only) | -| `--context NUM` | Show NUM nodes before and after each match (grep mode only) | -| `--list-style` | List style: `dash` (default), `plus`, `star` | -| `--link-title-style` | Link title style: `double` (default), `single`, `paren` | -| `--link-url-style` | Link URL style: `none` (default), `angle` | -| `-L, --directory DIR` | Search modules from directory | -| `-M, --module-names NAMES` | Load additional modules from files | -| `-m, --import-module-names NAMES` | Import modules as `name::fn()` in queries | -| `--csv`, `--yaml`, `--toml`, `--xml` | Load built-in modules | -| `--list` | List all available subcommands (built-in and external) | - - -## Auto-Parsing by File Extension - -When no `-I` flag is given, mq automatically imports based on file extension: - -| Extension | Behavior | -| ---------------- | ------------------------------------------ | -| `.json` | `import "json" \| json::json_parse()` | -| `.yaml` / `.yml` | `import "yaml" \| yaml::yaml_parse()` | -| `.toml` | `import "toml" \| toml::toml_parse()` | -| `.xml` | `import "xml" \| xml::xml_parse()` | -| `.csv` | `import "csv" \| csv::csv_parse(true)` | -| `.tsv` | `import "csv" \| csv::tsv_parse(true)` | -| `.psv` | `import "csv" \| csv::psv_parse(true)` | - -Use `-I raw` to disable auto-parsing and receive the raw string. - -## Full Attribute Reference - -### Heading (`.h`) - - -| Attribute | Type | Description | -| ---------------- | ------- | -------------------------- | -| `level`, `depth` | Integer | Heading level (1–6) | -| `value` | String | Text content | - - -### Code Block (`.code`) - - -| Attribute | Type | Description | -| ------------------ | ------- | --------------------------- | -| `lang`, `language` | String | Language identifier | -| `value` | String | Code content | -| `meta` | String | Metadata string | -| `fence` | Boolean | Whether fenced | - - -### Link (`.link`) / Image (`.image`) - - -| Attribute | Type | `.link` | `.image` | -| --------- | ------ | ------------ | ------------- | -| `url` | String | Link URL | Image URL | -| `title` | String | Link title | Image title | -| `value` | String | Link text | — | -| `alt` | String | — | Alt text | - - -### List (`.list`) - - -| Attribute | Type | Description | -| --------- | ------- | ---------------------- | -| `index` | Integer | Item index | -| `level` | Integer | Nesting level | -| `ordered` | Boolean | Whether ordered | -| `checked` | Boolean | Checkbox state | -| `value` | String | Text content | - - -### Table Cell (`.[row][col]`) - - -| Attribute | Type | Description | -| ----------------------- | ------- | ---------------------------- | -| `row` | Integer | Row number | -| `column` | Integer | Column number | -| `last_cell_in_row` | Boolean | Last cell in row | -| `last_cell_of_in_table` | Boolean | Last cell in table | -| `value` | String | Text content | - - -### Reference Nodes - - -| Node | Attributes | -| --------------- | -------------------------------- | -| `.link_ref` | `ident`, `label` | -| `.image_ref` | `ident`, `label`, `alt` | -| `.footnote_ref` | `ident`, `label` | -| `.footnote` | `ident`, `text` | -| `.definition` | `ident`, `url`, `title`, `label` | - - -### MDX Nodes - - -| Node | Attribute | Description | -| ------------------------- | --------- | ---------------- | -| `.mdx_jsx_flow_element` | `name` | Element name | -| `.mdx_flow_expression` | `value` | Expression value | - - -## Function Reference - -### String - -`upcase()`, `downcase()`, `split(s, sep)`, `join(arr, sep)`, `trim()`, `ltrimstr(s, prefix)`, `rtrimstr(s, suffix)`, `starts_with(s, prefix)`, `ends_with(s, suffix)`, `contains(haystack, needle)`, `index(s, sub)`, `rindex(s, sub)`, `slice(s, start, end)`, `replace(s, old, new)`, `gsub(s, pattern, rep)`, `regex_match(s, pat)`, `capture(s, pat)`, `repeat(s, n)`, `explode(s)`, `implode(arr)`, `url_encode(s)`, `base64(s)`, `base64d(s)` - -### Array & Collection - -`len`, `reverse`, `sort`, `sort_by(arr, fn)`, `uniq`, `unique_by(arr, fn)`, `compact`, `flatten`, `first`, `last`, `min`, `max`, `group_by(arr, fn)`, `pluck(arr, key)`, `any(arr, fn)`, `all(arr, fn)`, `map(arr, fn)`, `filter(arr, fn)`, `fold(arr, init, fn)`, `select(condition)`, `range(start, end, step)` - -### Numeric - -`add`, `sub`, `mul`, `div`, `mod`, `pow`, `abs`, `round`, `ceil`, `floor`, `trunc`, `negate`, `to_number` - -### Bytes - -`len`, `type`, `is_empty`, `==`, `base64(b)`, `base64d(s)` — byte sequences created with `b"..."` literals - -### Dictionary - -`dict()`, `get(d, key)`, `set(d, key, val)`, `keys`, `values`, `entries`, `update(d1, d2)` - -### Markdown Creation - -`to_h(text, depth)`, `to_code(text, lang)`, `to_code_inline(text)`, `to_link(url, text, title)`, `to_image(url, alt, title)`, `to_strong(text)`, `to_em(text)`, `to_hr()`, `to_math(text)`, `to_math_inline(text)`, `to_md_text(text)`, `to_md_list(val, level)`, `to_md_table_row(cells...)`, `to_md_table_cell(val, row, col)` - -### Markdown Manipulation - -`set_attr(node, attr, val)`, `attr(node, attr)`, `set_children(node, children)`, `set_check(list, checked)`, `set_ref(node, ref_id)`, `set_code_block_lang(code, lang)`, `set_list_ordered(list, ordered)`, `increase_header_level(h)`, `decrease_header_level(h)`, `to_text(node)`, `to_markdown_string(node)`, `to_html(node)`, `to_md_name(node)` - -### Type, I/O & Utility - -**Type**: `type`, `to_string()`, `to_number()`, `to_array()`, `is_none()`, `is_empty()`, `coalesce(a, b)` - -**I/O**: `print`, `stderr`, `input`, `read_file(path)`, `collection(dir)` (file-io only; reads a directory of Markdown files into an array of `{path, title, frontmatter, content}` dicts for cross-file queries) - -**Utility**: `identity()`, `error(msg)`, `halt(code)`, `assert(a, b)`, `now`, `from_date(str)`, `to_date(ts, fmt)`, `all_symbols` - -**Comparison**: `eq`, `ne`, `lt`, `lte`, `gt`, `gte`, `and`, `or`, `not` - -**Modules**: `include "csv"`, `include "yaml"`, `include "fuzzy"`, `include "test"` - -## Environment Variables - -- `__FILE__` — full path to the file being processed -- `__FILE_NAME__` — filename without path -- `__FILE_STEM__` — filename without extension - diff --git a/skills/processing-markdown/SKILL.md b/skills/processing-markdown/SKILL.md index be4784e65..c6a130ebe 100644 --- a/skills/processing-markdown/SKILL.md +++ b/skills/processing-markdown/SKILL.md @@ -74,7 +74,7 @@ mq '.h' file.md # All headings mq '.h(2)' file.md # h2 only mq '.code("rust")' file.md # Rust code blocks mq '.link.url' file.md # All URLs -mq '.yaml | to_text()' post.md # Frontmatter +mq '.yaml | to_text' post.md # Frontmatter # Filter mq 'select(.code)' file.md # Only code blocks @@ -83,17 +83,17 @@ mq 'select(.h.level <= 2)' file.md # h1 and h2 only mq 'select(contains("TODO"))' file.md # Nodes with "TODO" # Transform -mq '.h | to_text()' file.md # Headings as plain text +mq '.h | to_text' file.md # Headings as plain text mq -U '.code.lang |= "rust"' file.md # Update in place # Multi-file mq -A 'pluck(.code.value)' *.md # Collect all code values -mq -S 's"\n---\n"' 'identity()' *.md # Merge with separator +mq -S 's"\n---\n"' 'identity' *.md # Merge with separator # Format conversion -mq -F html 'identity()' file.md # Markdown → HTML -mq -F json '.h | to_text()' file.md # Headings → JSON -mq -I html 'identity()' page.html # HTML → Markdown +mq -F html 'identity' file.md # Markdown → HTML +mq -F json '.h | to_text' file.md # Headings → JSON +mq -I html 'identity' page.html # HTML → Markdown # Streaming large files mq --stream 'select(contains("ERROR"))' large.md @@ -105,32 +105,64 @@ When using `-I html`, mq converts HTML to Markdown first — use Markdown select ```bash # WRONG -curl -s https://example.com | mq -I html '.p | to_text()' +curl -s https://example.com | mq -I html '.p | to_text' # CORRECT -curl -s https://example.com | mq -I html '.text | to_text()' +curl -s https://example.com | mq -I html '.text | to_text' curl -s https://example.com | mq -I html '.link.url' -curl -s https://example.com | mq -I html '.h | to_text()' +curl -s https://example.com | mq -I html '.h | to_text' ``` ## Essential CLI Flags +A small, stable cheat sheet — not exhaustive. See below for everything else. + +| Flag | Purpose | +| ----------------------- | ------------------------------ | +| `-A, --aggregate` | Combine inputs into one array | +| `-F, --output-format` | Set output format | +| `-I, --input-format` | Set input format | +| `-U, --update` | Update file in place | +| `-S, --separator` | Insert separator between files | +| `--stream` | Process line by line | +| `mq repl` | Interactive REPL session | + +For the full CLI option list (all flags, possible format values, auto-parsing by file extension, `ARGS` handling), run `mq --help`. +For the full built-in function reference (300+ functions with descriptions), run `mq --doc`. + +Note: `--args` also accepts the hidden aliases `--arg` and `--define` (not shown in `mq --help`). + +## Node Attribute Reference + +These attributes are Markdown-selector-specific and are not covered by `mq --doc` / `mq --help`. + +| Node | Attributes | +| ------------------------------------------ | ------------------------------------------------------------------ | +| `.h` | `level`/`depth` (1–6), `value` | +| `.code` | `lang`/`language`, `value`, `meta`, `fence` (bool) | +| `.link` | `url`, `title`, `value` | +| `.image` | `url`, `title`, `alt` | +| `.list` | `index`, `level`, `ordered` (bool), `checked` (bool), `value` | +| `.[row][col]` (table cell) | `row`, `column`, `last_cell_in_row` (bool), `last_cell_of_in_table` (bool), `value` | +| `.link_ref` | `ident`, `label` | +| `.image_ref` | `ident`, `label`, `alt` | +| `.footnote_ref` | `ident`, `label` | +| `.footnote` | `ident`, `text` | +| `.definition` | `ident`, `url`, `title`, `label` | +| `.mdx_jsx_flow_element` | `name` | +| `.mdx_flow_expression` | `value` | + +## Function Call Syntax + +- All function calls require parentheses `()`. +- If a function is called with missing arguments, the piped value (`|`) is used as the first argument. + +## Environment Variables + +- `__FILE__` — full path to the file being processed +- `__FILE_NAME__` — filename without path +- `__FILE_STEM__` — filename without extension -| Flag | Purpose | -| --------------------- | ----------------------------------------------------- | -| `-A, --aggregate` | Combine all inputs into single array | -| `-I, --input-format` | Input format (see REFERENCE.md for full list) | -| `-F, --output-format` | Output format: `markdown`, `html`, `text`, `json`, `table`, `grep`, `raw`, `none` | -| `-U, --update` | Update file in place | -| `-f, --from-file` | Load query from `.mq` file | -| `-S, --separator` | Insert query result between files | -| `--args NAME VALUE` | Set runtime variable (aliases: `--arg`, `--define`) | -| `--stream` | Process line by line | -| `-C, --color-output` | Colorize output | -| `-P THRESHOLD` | Parallel processing threshold (default: 10) | -| `mq repl` | Start interactive REPL session | - -For full CLI options, attribute reference, and function list, see [REFERENCE.md](REFERENCE.md). For advanced examples, see [EXAMPLES.md](EXAMPLES.md). ## When NOT to Use mq From 8c26665fa3f2fc1468b2b12ac63387c2b8c4a309 Mon Sep 17 00:00:00 2001 From: harehare Date: Thu, 25 Jun 2026 00:01:44 +0900 Subject: [PATCH 2/2] =?UTF-8?q?=F0=9F=93=9D=20docs(skills):=20note=20that?= =?UTF-8?q?=20mq=20accepts=20multiple=20file=20args=20via=20shell=20glob?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- skills/processing-markdown/SKILL.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/skills/processing-markdown/SKILL.md b/skills/processing-markdown/SKILL.md index c6a130ebe..408faadfc 100644 --- a/skills/processing-markdown/SKILL.md +++ b/skills/processing-markdown/SKILL.md @@ -90,6 +90,10 @@ mq -U '.code.lang |= "rust"' file.md # Update in place mq -A 'pluck(.code.value)' *.md # Collect all code values mq -S 's"\n---\n"' 'identity' *.md # Merge with separator +# mq accepts multiple file args directly (shell glob expansion) — +# no need to loop over files in bash: +mq '.h | to_text' *.md work/*.md docs/*.md + # Format conversion mq -F html 'identity' file.md # Markdown → HTML mq -F json '.h | to_text' file.md # Headings → JSON