Skip to content

Support adding complex-morphology languages (nspell SFX/PFX conflict check) #24

Description

@SteadfastKnight

Background

nspell (our client-side spell check engine) stores SFX and PFX affix rules in the same lookup table, keyed only by flag letter. If a dictionary reuses the same letter for both an SFX block and a PFX block, the PFX block silently overwrites the SFX block. This causes entire verb conjugation rule sets to disappear, so correctly-spelled inflected forms get flagged as errors.

All six current languages (en, fr, de, es, it, la) have zero conflicts and are unaffected. Romanian had 11 conflicting flags and required remediation. Full write-up: docs/explanations/WKP-16-Spell-checking/2026-02-19-spell-check-architecture-and-choices.txt.


Checklist for adding a new language

Step 1 — Install and wire up

  • npm install dictionary-xx
  • Add .aff and .dic asset entries to angular.json (follow existing pattern)
  • Add lang code to SUPPORTED_LANGS in src/app/services/spell-check.service.ts

Step 2 — Run the conflict check (from wordkeep-client/):

node -e "
const fs = require('fs');
const aff = fs.readFileSync('./node_modules/dictionary-xx/index.aff', 'utf8');
const sfx = new Set(), pfx = new Set();
for (const line of aff.split('\n')) {
  const p = line.trim().split(/\s+/);
  if (p[0]==='SFX' && (p[2]==='Y'||p[2]==='N')) sfx.add(p[1]);
  if (p[0]==='PFX' && (p[2]==='Y'||p[2]==='N')) pfx.add(p[1]);
}
const c = [...sfx].filter(f => pfx.has(f));
console.log('Conflicts:', c.length ? c.join(' ') : 'none');
"

Step 3 — If no conflicts: done. Test in browser and move on.

Step 4 — If conflicts found: remediate the dictionary:

  • Copy the .aff and .dic files from node_modules/dictionary-xx/ into
    public/assets/dictionaries/xx/ (so they are committed to the repo and editable)
  • Update the angular.json asset entry to point to public/assets/dictionaries/xx/
    instead of node_modules/
  • Remove the conflicting PFX blocks from index.aff — each block is exactly
    2 lines (a header line and a single rule line). Identify them by running:
    grep -n "^PFX [<conflicting_flags>]" index.aff
  • Create wordkeep-client/scripts/expand-xx-dictionary.mjs modelled on
    scripts/expand-ro-dictionary.mjs. The script must:
    - Identify which flags were re- vs ne- prefixes (or whatever prefix applies)
    - Collect all distinct flag strings per base word — a word may appear
    multiple times in the .dic with different flag sets (different grammatical
    paradigms). Emit one prefix+word/flags entry per combination, not just one
    per word. Missing a flag set means some inflections of the prefixed word will
    still be unrecognised even though the bare form passes.
    - Skip prefix+word/flags combinations already present in the .dic (exact match)
    - Verify a set of known words (base forms AND inflected prefix forms) at the end
  • Run the script: node scripts/expand-xx-dictionary.mjs
  • Confirm all verification words pass, including inflected forms (e.g. feminine,
    plural) of prefixed words — not just the bare masculine singular
  • Commit the modified index.aff and index.dic together with the script

Step 5 — Test in browser

  • Open a document in the new language, enter edit mode
  • Confirm common inflected forms are not underlined
  • Confirm gibberish is underlined

Notes

  • Languages most likely to have conflicts: Polish, Hungarian, Turkish, Finnish, Czech,
    Slovak — any language with rich verb or noun morphology.
  • The conflicting PFX blocks are almost always trivial single-rule prefixes (like
    "re-" or "ne-"). Removing them has no practical impact on spell checking quality,
    provided the prefix forms are pre-generated in the .dic.
  • Copying the base word's flags to the prefixed entry is essential — without this,
    only the exact bare form is recognised and all its inflections remain flagged.
  • Some words appear multiple times in the .dic with different flag sets. The expansion
    script must handle this by emitting one entry per (word, flags) pair, not one per
    word. Deduplicating by word causes silent gaps in inflection coverage.
  • If a language's PFX blocks are complex (multi-rule, combineable with other rules),
    the dictionary patch approach may be insufficient. In that case, revisit hunspell-asm
    with a proper Vite/esbuild plugin that copies the WASM binary correctly.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions