diff --git a/.agents/skills/cut-release/SKILL.md b/.agents/skills/cut-release/SKILL.md index 7b74cc99..14ed8692 100644 --- a/.agents/skills/cut-release/SKILL.md +++ b/.agents/skills/cut-release/SKILL.md @@ -40,7 +40,7 @@ From [release.md](../../../docs/standards/release.md): | Stable library symbol added to `forgelm.__all__` (e.g. new `verify_*` function) | MINOR | MINOR | | Stable library symbol removed or signature changed | MAJOR | MAJOR | -`__version__` (single source of truth: [`pyproject.toml`](../../../pyproject.toml) line 7) tracks the *CLI / YAML / behavioural* contract. `__api_version__` (single source of truth: [`forgelm/_version.py`](../../../forgelm/_version.py)) tracks the *Python library* contract — operators who pin against it for feature detection rely on this signal. The two version numbers are decoupled by design: a release that adds a new training paradigm bumps `__version__` MINOR while leaving `__api_version__` untouched (no new public symbol), and a release that adds a new lazy-import target bumps both. +`__version__` (single source of truth: the `version =` line in [`pyproject.toml`](../../../pyproject.toml)) tracks the *CLI / YAML / behavioural* contract. `__api_version__` (single source of truth: [`forgelm/_version.py`](../../../forgelm/_version.py)) tracks the *Python library* contract — operators who pin against it for feature detection rely on this signal. The two version numbers are decoupled by design: a release that adds a new training paradigm bumps `__version__` MINOR while leaving `__api_version__` untouched (no new public symbol), and a release that adds a new lazy-import target bumps both. Library consumers should compare `__api_version__` via `packaging.version.Version` rather than string `>=`, because lexicographic comparison breaks for two-digit minor / patch components (e.g. `"1.10.0" < "1.2.0"` as strings): @@ -92,7 +92,7 @@ Rules: ### 3. Bump version -Edit [`pyproject.toml`](../../../pyproject.toml) line 7: +Edit the `version =` line in [`pyproject.toml`](../../../pyproject.toml): ```toml version = "0.4.0" # was "0.3.1rc1" or "0.4.0rc1" diff --git a/.claude/skills/cut-release/SKILL.md b/.claude/skills/cut-release/SKILL.md index 7b74cc99..14ed8692 100644 --- a/.claude/skills/cut-release/SKILL.md +++ b/.claude/skills/cut-release/SKILL.md @@ -40,7 +40,7 @@ From [release.md](../../../docs/standards/release.md): | Stable library symbol added to `forgelm.__all__` (e.g. new `verify_*` function) | MINOR | MINOR | | Stable library symbol removed or signature changed | MAJOR | MAJOR | -`__version__` (single source of truth: [`pyproject.toml`](../../../pyproject.toml) line 7) tracks the *CLI / YAML / behavioural* contract. `__api_version__` (single source of truth: [`forgelm/_version.py`](../../../forgelm/_version.py)) tracks the *Python library* contract — operators who pin against it for feature detection rely on this signal. The two version numbers are decoupled by design: a release that adds a new training paradigm bumps `__version__` MINOR while leaving `__api_version__` untouched (no new public symbol), and a release that adds a new lazy-import target bumps both. +`__version__` (single source of truth: the `version =` line in [`pyproject.toml`](../../../pyproject.toml)) tracks the *CLI / YAML / behavioural* contract. `__api_version__` (single source of truth: [`forgelm/_version.py`](../../../forgelm/_version.py)) tracks the *Python library* contract — operators who pin against it for feature detection rely on this signal. The two version numbers are decoupled by design: a release that adds a new training paradigm bumps `__version__` MINOR while leaving `__api_version__` untouched (no new public symbol), and a release that adds a new lazy-import target bumps both. Library consumers should compare `__api_version__` via `packaging.version.Version` rather than string `>=`, because lexicographic comparison breaks for two-digit minor / patch components (e.g. `"1.10.0" < "1.2.0"` as strings): @@ -92,7 +92,7 @@ Rules: ### 3. Bump version -Edit [`pyproject.toml`](../../../pyproject.toml) line 7: +Edit the `version =` line in [`pyproject.toml`](../../../pyproject.toml): ```toml version = "0.4.0" # was "0.3.1rc1" or "0.4.0rc1" diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 8ed18dfc..5fd93029 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -40,28 +40,20 @@ jobs: # safe_get). This grep guard fails CI if any new module reaches # for requests / urllib / httpx directly. # Phase 16: every Pydantic field in forgelm/config.py must carry a - # description= argument so the autogenerated configuration reference - # stays in lockstep with the schema. Strict mode exits 1 on any + # description= argument so the hand-maintained configuration reference + # (docs/reference/configuration.md + -tr.md mirror) always has + # authoritative field text to mirror. Strict mode exits 1 on any # undocumented field — a new contributor adding a field forgets the # description here, not silently drifting from the operator-facing docs. - name: Pydantic description= guard run: python tools/check_field_descriptions.py --strict forgelm/config.py + # F-P8-C-19: promoted from an inline grep (which missed + # requests.Session()/aliased-import/whitespace-before-paren forms + # and had no own test) to a tested tool with its own + # tests/test_check_http_discipline.py. - name: HTTP discipline guard - run: | - set +e - # Match ACTUAL CALLS only (the function name followed by an - # opening paren) so docstring prose mentioning the same names - # is not flagged. - violations=$(grep -rn -E '(requests\.(get|post|put|delete|patch|request|head)\(|urllib\.request\.urlopen\(|httpx\.[a-z]+\()' forgelm/ --include='*.py' | grep -v 'forgelm/_http.py') - if [ -n "$violations" ]; then - echo "::error::Found undisciplined HTTP call(s) outside forgelm/_http.py:" - echo "$violations" - echo "" - echo "Route through forgelm._http.safe_post / safe_get instead." - echo "See docs/standards/architecture.md 'HTTP discipline' section." - exit 1 - fi + run: python tools/check_http_discipline.py # --- Job 2: Test (matrix across Python versions) --- test: @@ -195,6 +187,15 @@ jobs: # tools/check_site_claims.py for what's checked and why. run: python3 tools/check_site_claims.py --strict + - name: Doc numerical-claims drift check (strict) + # F-P8-C-06 (W1/H5): re-derives the canonical counts (secret + # families, trainer types, quickstart templates, webhook events) + # from the Python sources of truth and fails the build when a doc + # asserts a stale number — e.g. the "five webhook events" prose that + # drifted after the vocabulary grew to eight. Wired once H5's 5->8 + # doc-drift fix made the guard green at HEAD. + run: python3 tools/check_doc_numerical_claims.py --strict + - name: Bilingual doc H2/H3/H4 parity check (strict) # Phase 24: extended structural parity guard. Where the prior # inline check counted H2 only and missed reordered / demoted @@ -251,6 +252,71 @@ jobs: # tools/check_audit_event_catalog.py. run: python3 tools/check_audit_event_catalog.py --strict + - name: TR cross-links prefer the TR mirror (strict) + # W1/H11 (F-P8-C-04): a docs/**/*-tr.md page must route its in-prose + # cross-references to the Turkish sibling when a -tr.md mirror + # exists — a Turkish operator following a 'Bkz.'/'See also' link must + # stay in Turkish, not silently land on the English page. Neither + # check_anchor_resolution (link resolves) nor check_bilingual_parity + # (heading spine) catches this. The **Ayna:** backlink line is exempt. + # 62 leaks across 19 files at HEAD were swept to zero in H11. + run: python3 tools/check_tr_links_prefer_mirror.py --strict + + - name: Library-API doc ↔ __all__ drift check (strict) + # W1/H11 (F-P8-C-07): forgelm.__all__ must match the symbol roster in + # docs/reference/library_api_reference.md in both directions. Previously + # unwired — a renamed/removed public symbol could drift from the doc. + run: python3 tools/check_library_api_doc.py --strict + + - name: Bilingual fenced-code-block parity (strict) + # W1/H11 (F-P8-C-13): the Wave 6 fenced-block-count + per-ordinal + # YAML-key parity guard was an orphan (wired nowhere, no own test). + # Catches TR code-block / YAML-key drift the spine guard cannot see. + run: python3 tools/check_bilingual_code_blocks.py --strict + + - name: ForgeConfig YAML snippet validation (strict) + # W1/H11 (F-P8-C-07): every ForgeConfig-shaped YAML snippet in the docs + # must pass Pydantic validation. Previously unwired despite catching + # real doc-vs-schema drift. + run: python3 tools/check_yaml_snippets.py --strict + + - name: Site chrome (EN/TR translation) parity + # W1/H11 (F-P8-C-07): the active-tier (EN<->TR) translation-key sets in + # site/js/translations.js must stay in lockstep. Run WITHOUT --strict: + # --strict additionally gates the deferred de/fr/es/zh tiers, which are + # a known v0.6.x backlog the guard's own help text says is "NOT wired + # into CI". Default mode enforces only the active-tier parity rule. + run: python3 tools/check_site_chrome_parity.py + + - name: Module-size ceiling guard (strict) + # W1/H11 (F-P8-C-07): no forgelm/ module may drift past the + # architecture-doc ~1000-LOC sub-package-split ceiling. Previously + # unwired — module-size regressions could merge with green CI. + run: python3 tools/check_module_size.py --strict + + - name: Wizard defaults schema-sync (strict) + # W1/H11 (F-P8-C-07): the wizard's shipped defaults JSON must match the + # Pydantic schema source of truth. Was gauntlet-only (human discipline). + run: python3 tools/check_wizard_defaults_sync.py + + - name: No public-tree refs into gitignored working-memory + # W1/H11 (F-P8-C-07): public-tree files must not cite docs/analysis/ or + # docs/marketing/. Was gauntlet-only (human discipline) until now. + run: python3 tools/check_no_analysis_refs.py + + - name: Marketing-site version sync (strict) + # W1/H11 (F-P8-C-07): the marketing site's displayed version must match + # CHANGELOG's latest released header. Was gauntlet-only. + run: python3 tools/update_site_version.py --check + + - name: Notebook forgelm pin lockstep (strict) + # W2/M7 (F-P8-C-09): every notebooks/*.ipynb !pip install forgelm pin + # must target a shipping wheel — the exact pyproject version or the + # latest released CHANGELOG version (so onboarding notebooks never + # point users at a pre-release rc). Previously unwired; the pins had + # drifted two minors (0.5.7 vs released 0.7.0) undetected. + run: python3 tools/check_notebook_pins.py --strict + - name: User-manual self-contained link guard (strict) # Post-v0.7.0 cycle: the docs/usermanuals/ tree is the source of # truth for the static-site SPA viewer (site/usermanual.html + diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml index b7fbc18a..32a6392a 100644 --- a/.github/workflows/publish.yml +++ b/.github/workflows/publish.yml @@ -84,19 +84,15 @@ jobs: shell: bash run: python -m pip install pytest pytest-cov - - name: Verify --ignore target exists - shell: bash - # Guard against silent rename: if test_cost_estimation.py is renamed - # or removed without updating the --ignore flag below, the publish - # would silently start running it (or no-op the ignore). Fail fast. - run: test -f tests/test_cost_estimation.py - - name: Test (CPU-only, no-network) shell: bash - # test_cost_estimation is excluded per Faz 31 closure plan: it - # depends on a snapshot pricing fixture that drifts on a different - # cadence than the release matrix. - run: pytest tests/ -q --ignore=tests/test_cost_estimation.py + # Drift-sensitive tests (test_cost_estimation: snapshot pricing + # fixture on a different cadence than the release matrix) are + # excluded via the @pytest.mark.fixture_drift marker — F-P8-C-20 + # made the marker the single source of truth, so no filename guard + # is needed: a renamed/removed test simply keeps (or drops) the + # marker, and the exclusion follows the marker, not a brittle path. + run: pytest tests/ -q -m 'not fixture_drift' - name: Generate SBOM (CycloneDX 1.5) shell: bash diff --git a/AGENTS.md b/AGENTS.md index 90ada9a9..4b71eb2c 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -57,7 +57,7 @@ ForgeLM/ │ │ # _aggregator, _streaming, _simhash, _minhash, │ │ # _pii_regex, _pii_ml, _secrets, _quality, │ │ # _croissant, _summary, _splits -│ ├── config.py # Pydantic schemas (19 models) +│ ├── config.py # Pydantic schemas (23 models) │ ├── trainer.py # TRL wrapper (SFT/DPO/SimPO/KTO/ORPO/GRPO) │ ├── model.py # HF + PEFT model loading │ ├── data.py # Dataset loading + format detection @@ -158,11 +158,13 @@ Default workflow for a non-trivial change: python3 tools/check_no_analysis_refs.py && \ python3 tools/check_no_unguarded_sys_modules_pop.py && \ python3 tools/check_audit_event_catalog.py --strict && \ + python3 tools/check_tr_links_prefer_mirror.py --strict && \ python3 tools/check_usermanual_self_contained.py --strict && \ + python3 tools/check_notebook_pins.py --strict && \ python3 tools/update_site_version.py --check ``` - All thirteen must pass. The first four are the historical gauntlet; + All fifteen must pass. The first four are the historical gauntlet; the three doc guards (Wave 3 / Wave 4 / Wave 5 additions) catch bilingual structural drift, broken markdown anchors, and CLI ↔ docs help-text drift before the PR opens. The wizard-defaults guard @@ -180,7 +182,14 @@ Default workflow for a non-trivial change: table in `docs/reference/audit_event_catalog.md` in both directions — the append-only audit log is an EU AI Act Art. 12 contract, and this guard was previously unwired while six `pipeline.*` stage events - drifted into the code uncatalogued. The + drifted into the code uncatalogued. The TR-links-prefer-mirror + guard (full-project-review W1/H11, F-P8-C-04) fails when a + `docs/**/*-tr.md` page links the un-suffixed English sibling even + though a `-tr.md` mirror exists — a Turkish reader following + an in-prose link must stay in Turkish; the `**Ayna:**` backlink + line is the one exempt case. 62 leaks across 19 files were swept + to zero when this guard landed — see `docs/standards/localization.md` + "Structural mirror rule". The usermanual self-contained guard (post-v0.7.0 cycle) blocks any link inside `docs/usermanuals/` that would 404 in the static SPA viewer: every link must be either a `#/
/` SPA @@ -192,7 +201,11 @@ Default workflow for a non-trivial change: latest released header and fails the PR if any of the 15+ literals across `site/*.html` and `site/js/translations.js` has drifted; the v0.5.5 → v0.6.0 release shipped with the hero badge still reading - `v0.5.5`, which this guard now prevents. + `v0.5.5`, which this guard now prevents. The notebook-pin guard + (v0.7.0 / F-P8-C-09) verifies that every `*.ipynb` in the repo + pins its kernel and package versions so that example notebooks + remain reproducible; it was wired into CI at the v0.7.0 pin bump + but was absent from the local gauntlet until this update. ## Etiquette when communicating with the user diff --git a/CHANGELOG.md b/CHANGELOG.md index b060a8f6..b91786f3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,66 @@ All notable changes to ForgeLM are documented here. _(v0.7.1 dev cycle — entries will land here as PRs merge.)_ +### Added + +- **Config-driven merge hyperparameters.** `merge.ties_trim_fraction`, + `merge.dare_drop_rate`, and `merge.dare_seed` expose the TIES/DARE knobs + that were previously fixed module constants (defaults unchanged: `0.2`, + `0.3`, `42`). Operators can now reach paper-faithful sparsity from YAML. +- **Config-driven synthetic sanity bound.** `synthetic.sanity_failure_rate` + (default `0.2`) replaces the hardcoded warn-only failure-rate threshold in + `forgelm --generate-data`; it is independent of `min_success_rate`, which + still gates the exit code. + +### Changed + +- **Config validation hardened.** `distributed.strategy` is now a + `Literal["deepspeed", "fsdp"]` (an unsupported value such as `horovod` + used to validate and then silently run single-GPU). `data.mix_ratio` + now rejects non-finite weights (NaN / inf) and must carry exactly one + weight per dataset (`1 primary + len(extra_datasets)`); a length + mismatch raised no config error and silently fell back to uniform + mixing at runtime. Both now fail fast at config time (exit 1). +- **Deprecation removals deferred to v0.8.0.** `evaluation.staging_ttl_days` + and the `--data-audit` CLI flag were originally scheduled for removal in + v0.7.0, but v0.7.0 shipped with both still present to preserve the full + one-minor deprecation window. Every `DeprecationWarning`, `--help` text, + reference-doc note, and the release-standard worked example now name + **v0.8.0** consistently as the removal target. The `### Removed` section + lands in the v0.8.0 CHANGELOG. + +### Deprecated + +- **`training.sample_packing`** is now a deprecated alias for + `training.packing`. It was previously a documented-but-unconsumed field + (a silent no-op); it now forwards to `packing` with a + `DeprecationWarning` so the documented behaviour actually fires. Use + `packing` instead — `sample_packing` is removed in **v0.9.0**. See + [docs/standards/release.md](docs/standards/release.md#deprecation-cadence). + +### Fixed + +- **Eval artefact privacy-redaction documented.** `safety_results.json` and + `judge_results.json` have been privacy-redacted by default since v0.7.0; + this entry adds the previously missing CHANGELOG documentation. Raw + `prompt` / `response` / judge `reason` strings are not persisted unless the + opt-in flags `evaluation.safety.include_eval_samples` and + `evaluation.llm_judge.include_eval_samples` (both default `false`) are set. + This honours GDPR / EU AI Act Article 10 privacy-by-default — adversarial + prompts and judge reasoning can quote sensitive content. Set the flag to + `true` only for debugging. +- **Nightly pip-audit gate — transformers PYSEC-2025-217 / CVE-2025-14929.** + Advisory records X-CLIP checkpoint-conversion deserialization RCE + (CVSS AV:L/UI:R — local + user-interaction required). No fixed version in + the `transformers<5.0.0` range. Codebase check 2026-05-24: no X-CLIP usage + in `forgelm/`, no direct `torch.load` calls. Risk accepted in + `tools/pip_audit_ignores.yaml`; re-evaluate each release cycle. +- **Pipeline configs with `pipeline:` + `retention.staging_ttl_days` + + any `evaluation:` block** no longer raise a false + `ConfigError ("Conflicting staging_ttl_days values")`. The stage-merge + round-trip re-materialised the deprecated `staging_ttl_days=7` default as + if the operator had written it; the dump now excludes unset defaults. + ## [0.7.0] — 2026-05-14 Phase 14 (Multi-Stage Pipeline Chains) closes the "operators have to write @@ -97,10 +157,11 @@ block is present. validator lives in private helper `_verify_manifest_payload` so the in-memory and disk-bound entry points cannot drift. -### Fixed (Phase 14 review-response — PR #53) +### Fixed -Addresses the consolidated reviewer-pass findings (3 blocking + -4 significant + Gemini's 3 inline comments) against the initial Phase +Addresses the consolidated reviewer-pass findings from Phase 14 review-response +(PR [#53](https://github.com/HodeTech/ForgeLM/pull/53)) — 3 blocking + +4 significant + Gemini's 3 inline comments — against the initial Phase 14 merge candidate: - **F-B-1** — `forgelm --config pipeline.yaml --dry-run` now reaches @@ -872,7 +933,7 @@ collectors / BYOD / IO concerns. bash/shell `forgelm` invocations; reports drift classes (subcommand not in parser; flag not in parser; flag value not in parser's `choices` list). -- **`tools/check_field_descriptions.py --strict`** — AST-based +- **`tools/check_field_descriptions.py --strict forgelm/config.py`** — AST-based scanner of Pydantic `BaseModel` subclasses; exits 1 on any field missing a `description=`. - **`tools/check_no_analysis_refs.py`** — prohibits citations to diff --git a/CLAUDE.md b/CLAUDE.md index 98917e16..1a20dac3 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -57,7 +57,7 @@ ForgeLM/ │ │ # _aggregator, _streaming, _simhash, _minhash, │ │ # _pii_regex, _pii_ml, _secrets, _quality, │ │ # _croissant, _summary, _splits -│ ├── config.py # Pydantic schemas (19 models) +│ ├── config.py # Pydantic schemas (23 models) │ ├── trainer.py # TRL wrapper (SFT/DPO/SimPO/KTO/ORPO/GRPO) │ ├── model.py # HF + PEFT model loading │ ├── data.py # Dataset loading + format detection @@ -158,11 +158,13 @@ Default workflow for a non-trivial change: python3 tools/check_no_analysis_refs.py && \ python3 tools/check_no_unguarded_sys_modules_pop.py && \ python3 tools/check_audit_event_catalog.py --strict && \ + python3 tools/check_tr_links_prefer_mirror.py --strict && \ python3 tools/check_usermanual_self_contained.py --strict && \ + python3 tools/check_notebook_pins.py --strict && \ python3 tools/update_site_version.py --check ``` - All thirteen must pass. The first four are the historical gauntlet; + All fifteen must pass. The first four are the historical gauntlet; the three doc guards (Wave 3 / Wave 4 / Wave 5 additions) catch bilingual structural drift, broken markdown anchors, and CLI ↔ docs help-text drift before the PR opens. The wizard-defaults guard @@ -180,7 +182,14 @@ Default workflow for a non-trivial change: table in `docs/reference/audit_event_catalog.md` in both directions — the append-only audit log is an EU AI Act Art. 12 contract, and this guard was previously unwired while six `pipeline.*` stage events - drifted into the code uncatalogued. The + drifted into the code uncatalogued. The TR-links-prefer-mirror + guard (full-project-review W1/H11, F-P8-C-04) fails when a + `docs/**/*-tr.md` page links the un-suffixed English sibling even + though a `-tr.md` mirror exists — a Turkish reader following + an in-prose link must stay in Turkish; the `**Ayna:**` backlink + line is the one exempt case. 62 leaks across 19 files were swept + to zero when this guard landed — see `docs/standards/localization.md` + "Structural mirror rule". The usermanual self-contained guard (post-v0.7.0 cycle) blocks any link inside `docs/usermanuals/` that would 404 in the static SPA viewer: every link must be either a `#/
/` SPA @@ -192,7 +201,11 @@ Default workflow for a non-trivial change: latest released header and fails the PR if any of the 15+ literals across `site/*.html` and `site/js/translations.js` has drifted; the v0.5.5 → v0.6.0 release shipped with the hero badge still reading - `v0.5.5`, which this guard now prevents. + `v0.5.5`, which this guard now prevents. The notebook-pin guard + (v0.7.0 / F-P8-C-09) verifies that every `*.ipynb` in the repo + pins its kernel and package versions so that example notebooks + remain reproducible; it was wired into CI at the v0.7.0 pin bump + but was absent from the local gauntlet until this update. ## Etiquette when communicating with the user diff --git a/config_template.yaml b/config_template.yaml index fe35b495..b7b2d918 100644 --- a/config_template.yaml +++ b/config_template.yaml @@ -28,7 +28,8 @@ training: final_model_dir: "final_model" merge_adapters: false trainer_type: "sft" # "sft", "dpo", "simpo", "kto", "orpo", "grpo" - num_train_epochs: 3 + max_steps: -1 # -1 = use num_train_epochs; a positive value overrides epochs + num_train_epochs: 3 # Only consulted when max_steps == -1 per_device_train_batch_size: 4 gradient_accumulation_steps: 2 learning_rate: 2.0e-5 @@ -37,6 +38,7 @@ training: eval_steps: 200 save_steps: 200 save_total_limit: 3 + early_stopping_patience: 3 # Stop after N evals without validation-loss improvement packing: false # --- Alignment trainer parameters (used by respective trainer_type) --- # dpo_beta: 0.1 # DPO temperature @@ -73,7 +75,7 @@ data: # factor: 4.0 # Context length multiplier (4.0 = 4x original) # neftune_noise_alpha: 5.0 # NEFTune: add noise to embeddings (improves quality) # sliding_window_attention: 4096 # Override model's sliding window (e.g. Mistral) -# sample_packing: true # Pack multiple short sequences into one +# packing: true # Pack multiple short sequences into one (sample_packing is a deprecated alias) # Synthetic data generation (teacher→student distillation) # Generate training data from a larger teacher model. @@ -89,7 +91,9 @@ data: # max_new_tokens: 1024 # temperature: 0.7 # output_file: "synthetic_data.jsonl" -# output_format: "messages" # "messages", "instruction", "chatml" +# output_format: "messages" # "messages", "instruction", "chatml", "prompt_response" +# min_success_rate: 0.0 # min fraction of seeds that must yield (0.0 = any yield succeeds) +# sanity_failure_rate: 0.2 # failure rate above which a WARNING is logged (warn-only) # GaLore — optimizer-level memory optimization (alternative to LoRA) # Enables full-parameter training with low-rank gradient projection. @@ -124,6 +128,9 @@ data: # - path: "./checkpoints/run2/final_model" # weight: 0.3 # output_dir: "./merged_model" +# ties_trim_fraction: 0.2 # TIES: fraction of smallest deltas trimmed (0.0-1.0) +# dare_drop_rate: 0.3 # DARE: probability each delta is dropped (0.0-1.0) +# dare_seed: 42 # DARE: RNG seed for the random drop mask # Distributed training (multi-GPU). Requires: pip install forgelm[distributed] # Launch with: torchrun --nproc_per_node=N forgelm --config my_config.yaml @@ -181,6 +188,7 @@ data: # notify_on_start: true # notify_on_success: true # notify_on_failure: true +# # require_https: true # refuse plaintext http:// delivery (regulated estates) # EU AI Act Compliance (Art. 11 + Annex IV) # compliance: diff --git a/docs/design/blackwell_optimized.md b/docs/design/blackwell_optimized.md index d2e7b293..f1dd3051 100644 --- a/docs/design/blackwell_optimized.md +++ b/docs/design/blackwell_optimized.md @@ -8,7 +8,7 @@ > auto-enable flow are all pending. Operators on Blackwell hardware > currently configure `expandable_segments` via the upstream > `PYTORCH_ALLOC_CONF` env var directly and set `training.bf16: true` / -> `training.sample_packing: true` by hand. +> `training.packing: true` by hand. ## Overview Specific optimizations for the latest Nvidia Blackwell architecture (notably GB10 with 128GB unified RAM) to maximize throughput and stability. diff --git a/docs/design/iso27001_soc2_alignment.md b/docs/design/iso27001_soc2_alignment.md index d49ab6df..96a4668c 100644 --- a/docs/design/iso27001_soc2_alignment.md +++ b/docs/design/iso27001_soc2_alignment.md @@ -629,8 +629,11 @@ A.6.8, A.8.15, A.8.16. mapping. 3. **Rollback procedure** — `auto_revert` for in-pipeline; manual redeploy of previous model SHA for post-deployment. -4. **Configuration drift detection** — `tools/regenerate_config_doc.py` - diff-guard catches Pydantic-schema drift between code and docs. +4. **Configuration drift detection** — `tools/check_field_descriptions.py --strict` + (CI) fails the build when a Pydantic field lacks a `description=`, and + `tools/check_bilingual_parity.py --strict` enforces EN↔TR doc parity; + the operator-facing configuration reference is maintained by hand and + reviewed against the schema. 5. **SBOM drift detection** — `tools/generate_sbom.py` deterministic contract: a release's SBOM is reproducible from the corresponding `git tag`. diff --git a/docs/design/library_api.md b/docs/design/library_api.md index 35ae0101..aad392b7 100644 --- a/docs/design/library_api.md +++ b/docs/design/library_api.md @@ -322,17 +322,11 @@ __api_version__ = "MAJOR.MINOR" # e.g. "0.5" The original rationale was that patch-level changes to the library are by definition non-breaking, so no consumer needs to detect them. `release.md` later argued the opposite: keeping a `PATCH` slot lets consumers `packaging.version.Version`-compare reliably and gives implementation tweaks a parking spot without burning a MINOR. The 3-segment shape won. -### 5.2 Breaking-change detection (CI guard, optional) +### 5.2 Breaking-change detection (in-repo snapshot test) -Phase 19 ships a `tools/check_api_compat.py` script that compares the **stable** symbols of the current branch against the previously released `__api_version__` and flags any signature change. Implementation sketch: +Breaking-change detection runs as an **in-repo snapshot test** rather than the venv-diffing `tools/check_api_compat.py` script that earlier drafts of this section sketched (that standalone tool was never shipped). `tests/test_library_api.py::TestApiSignatureSnapshot` records every public symbol's signature (callables), field roster (dataclasses), or `model_fields` keys (Pydantic models) in `tests/_data/api_signatures_<__api_version__>.json`, keyed to the current `__api_version__`. The test fails whenever the live surface differs from the snapshot for that version. -1. Pip install the previous release into a temp venv. -2. `python -c "import forgelm; print(json.dumps({n: inspect.signature(getattr(forgelm, n)).__str__() for n in }))"`. -3. Diff against the same dump from the working tree. - -CI trigger: **`pull_request` job that runs whenever the PR targets `main` AND modifies `forgelm/__init__.py` or `forgelm/_version.py`**, plus a manual `workflow_dispatch` step the `cut-release` skill invokes before tagging. We do **not** use `release-*` branches as the trigger: `docs/standards/release.md:259` explicitly forbids release branches ("**No release branches** — if a hotfix is needed for an old version that has diverged, create a branch at that tag and cherry-pick — rare"). A `release-*` trigger would ship a CI step that never fires because no `release-*` branch ever exists. No-op for the v0.5.5 release itself: there is no prior `__api_version__`-bearing release to diff against, so the script's first useful run is at v0.5.6 cut time. Phase 19 ships the script + the `pull_request` + `workflow_dispatch` workflow steps; Phase 33 (v0.5.5 release) treats the script as a documented future contract rather than an immediate gate. - -This is not a blocking gate; it is a notification. A genuinely-needed signature change still merges, but the release notes get an automatic "BREAKING:" line. +Workflow: a genuinely-needed signature change makes the test red; the contributor then bumps `__api_version__` per the §5.1 rules (MAJOR for a removed/signature-changed **stable** symbol, MINOR for an addition) and regenerates the snapshot file under the new version key, so the bump and the surface change land in the same commit. This is a blocking gate (the test is part of the suite) — stronger than the "notification only" posture the original sketch proposed, and it needs no temp-venv install of a prior release. ### 5.3 Deprecation cadence @@ -438,7 +432,7 @@ The rule: **library code does not call `sys.exit`**. Every exit-code mapping li |---|---|---| | `ForgeTrainer.train()` | No — TRL trainer holds GPU state. | No. | | `audit_dataset()` | Yes (multiple corpora in parallel threads); each call is self-contained. | Yes. | -| `AuditLogger.log_event()` | Yes — uses `fcntl.flock` on POSIX (Wave 1). Windows uses `msvcrt.locking`. | Each forked child should construct its own `AuditLogger`; sharing the file handle across forks is unsupported. | +| `AuditLogger.log_event()` | Yes — uses `fcntl.flock` on POSIX (Wave 1). On Windows there is no cross-process lock (the advisory flock helper is a no-op); do not share an `output_dir` across concurrent processes on Windows. | Each forked child should construct its own `AuditLogger`; sharing the file handle across forks is unsupported. | | `verify_audit_log()` | Yes — read-only. | Yes. | | `WebhookNotifier.notify_*()` | Yes — each call opens its own `requests` session. | Yes. | @@ -530,4 +524,4 @@ The design above shipped end-to-end in v0.5.5: - Two version strings (§5): `__version__` (runtime, `importlib.metadata`-derived) + `__api_version__` (semver `1.0.0`); both live in `forgelm/_version.py`. - Integration tests (§6): `tests/test_library_api.py` covers public-surface invariants — stable-symbol roster matches `forgelm.__all__`, lazy-import discipline (subprocess assertion that `import forgelm` does not pull `torch` / `transformers` / `trl`), `__getattr__` resolution + caching, `dir(forgelm)` shape, `py.typed` marker presence, and `__api_version__` contract — paired with `tests/test_lazy_imports.py` for the per-submodule lazy-load regression corpus. End-to-end train-and-chat user journeys are exercised by the per-trainer test modules (`tests/test_trainer*.py`, `tests/test_chat.py`), not by `test_library_api.py`. - Docs (§7): `docs/reference/library_api_reference{,-tr}.md`, `docs/guides/library_api{,-tr}.md`, `docs/usermanuals/{en,tr}/reference/library-api.md`, README "Library API" section. -- Errors + concurrency (§8): `ConfigError`, `OptionalDependencyError`, `HttpSafetyError`; multiprocess-safe `AuditLogger.log_event` via `fcntl.flock` / `msvcrt.locking`. +- Errors + concurrency (§8): `ConfigError`, `OptionalDependencyError`, `HttpSafetyError`; `AuditLogger.log_event` is multiprocess-safe on POSIX via `fcntl.flock` (no cross-process lock on Windows — no-op). diff --git a/docs/guides/air_gap_deployment-tr.md b/docs/guides/air_gap_deployment-tr.md index 845a6a54..4e1f57c1 100644 --- a/docs/guides/air_gap_deployment-tr.md +++ b/docs/guides/air_gap_deployment-tr.md @@ -2,7 +2,7 @@ > **Hedef kitle:** ForgeLM'i kısıtlı-egress host'a deploy eden operatörler — savunma, sağlık, bazı finans sektörleri, sınıflandırılmış-ağ müşteri ortamları. Eğitim sırasında outbound HTTPS'i reddeden güvenlik politikası olan herkes. > -> Bu derinlemesine bir deployer pişirme kitabıdır. [GDPR erasure rehberi](gdpr_erasure.md) ve [ISO/SOC 2 deployer rehberi](iso_soc2_deployer_guide.md) derinliğini yansıtır — gösterilen her komut [`forgelm/cli/subcommands/_cache.py`](../../forgelm/cli/subcommands/_cache.py) ve [`forgelm/cli/subcommands/_doctor.py`](../../forgelm/cli/subcommands/_doctor.py) implementasyonuna karşı doğrulanmıştır. +> Bu derinlemesine bir deployer pişirme kitabıdır. [GDPR erasure rehberi](gdpr_erasure-tr.md) ve [ISO/SOC 2 deployer rehberi](iso_soc2_deployer_guide-tr.md) derinliğini yansıtır — gösterilen her komut [`forgelm/cli/subcommands/_cache.py`](../../forgelm/cli/subcommands/_cache.py) ve [`forgelm/cli/subcommands/_doctor.py`](../../forgelm/cli/subcommands/_doctor.py) implementasyonuna karşı doğrulanmıştır. ## Bu rehber neyi çözer diff --git a/docs/guides/alignment-tr.md b/docs/guides/alignment-tr.md index b6c8a6a1..44a3f067 100644 --- a/docs/guides/alignment-tr.md +++ b/docs/guides/alignment-tr.md @@ -202,11 +202,13 @@ skalere toplar): döner; bu sayede erken eğitimde format uyumu başlamadan da non-flat gradient olur. - **`_math_reward_fn`** (`forgelm/trainer.py`) — yalnız dataset'in - `gold_answer` alanı varsa eklenir. `Answer:`'tan sonraki değeri - yakalar, yaygın birimleri (`$`, `%`, `km/h`, `m²`, `liters`, …) - soyar ve `gold_answer` ile önce exact-string, sonra numerik - tolerans (1e-6) ile karşılaştırır. Doğru yanıt için `1.0`, - aksi halde `0.0`. + `gold_answer` alanı varsa eklenir. **Son** `Answer:` işaretinden + sonraki değeri yakalar (böylece kendini düzelten bir üretim, + gerçekten sonuçlandırdığı yanıt üzerinden — sona sabitlenmiş + format ödülüyle tutarlı olarak — puanlanır), yaygın birimleri + (`$`, `%`, `km/h`, `m²`, `liters`, …) soyar ve `gold_answer` ile + önce exact-string, sonra numerik tolerans (1e-6) ile + karşılaştırır. Doğru yanıt için `1.0`, aksi halde `0.0`. Bundled `forgelm quickstart grpo-math` şablonu `gold_answer` doldurulmuş olarak ship olur, böylece model kutudan çıktığı gibi hem format diff --git a/docs/guides/alignment.md b/docs/guides/alignment.md index ff4d42cd..36096960 100644 --- a/docs/guides/alignment.md +++ b/docs/guides/alignment.md @@ -179,7 +179,7 @@ GRPO needs a reward signal. ForgeLM wires reward callables additively (TRL sums 1. **`grpo_reward_model` set** — Loads the HF sequence-classification model at that path and uses its scalar output as the only reward signal. The built-in rewards below are bypassed; the operator opted into a learned reward. 2. **No `grpo_reward_model`** — A baseline reward is always wired: - **`combined_format_length_reward`** (`forgelm/grpo_rewards.py`) — `0.8 × format_match + 0.2 × length_shaping`. The format component returns 1.0 when the generation ends with `Answer: ` (case-insensitive, units allowed); the length component returns `min(len(completion) / 200, 1.0)` so early training has a non-flat gradient even before format compliance kicks in. - - **`_math_reward_fn`** (`forgelm/trainer.py`) — appended only when the dataset has a `gold_answer` field. Captures the value after `Answer:`, strips common units (`$`, `%`, `km/h`, `m²`, `liters`, …), and compares to `gold_answer` with exact-string match first, then numeric tolerance (1e-6). Returns `1.0` for a correct answer, `0.0` otherwise. + - **`_math_reward_fn`** (`forgelm/trainer.py`) — appended only when the dataset has a `gold_answer` field. Captures the value after the **last** `Answer:` marker (so a self-correcting generation is graded on the answer it actually concludes with, consistent with the end-anchored format reward), strips common units (`$`, `%`, `km/h`, `m²`, `liters`, …), and compares to `gold_answer` with exact-string match first, then numeric tolerance (1e-6). Returns `1.0` for a correct answer, `0.0` otherwise. The bundled `forgelm quickstart grpo-math` template ships with `gold_answer` populated, so the model gets both format teaching AND correctness teaching out of the box. To use a real reward model on top of grpo-math, set `grpo_reward_model` and the built-in rewards are bypassed. diff --git a/docs/guides/gdpr_erasure-tr.md b/docs/guides/gdpr_erasure-tr.md index dfabf93d..7ba344fc 100644 --- a/docs/guides/gdpr_erasure-tr.md +++ b/docs/guides/gdpr_erasure-tr.md @@ -38,7 +38,7 @@ $ forgelm purge \ Tool şunları yapar: -1. Per-output-dir salt'ı `/.forgelm_audit_salt`'ta çözer (ilk kullanımda oluşturur; mode `0600`; `FORGELM_AUDIT_SECRET[:16]` set'liyse XOR'lar). **Not:** bu XOR yalnızca **identifier hashing**'i besler — audit-chain HMAC anahtarı bağımsız olarak `SHA-256(FORGELM_AUDIT_SECRET ‖ run_id)` şeklinde türetilir (bkz. [`docs/qms/access_control.md`](../qms/access_control.md) §3.4 / `forgelm/compliance.py:104-114`). İki primitif kasıtlı olarak ayrıdır. +1. Per-output-dir salt'ı `/.forgelm_audit_salt`'ta çözer (ilk kullanımda oluşturur; mode `0600`; `FORGELM_AUDIT_SECRET[:16]` set'liyse XOR'lar). **Not:** bu XOR yalnızca **identifier hashing**'i besler — audit-chain HMAC anahtarı bağımsız olarak `SHA-256(FORGELM_AUDIT_SECRET ‖ run_id)` şeklinde türetilir (bkz. [`docs/qms/access_control.md`](../qms/access_control-tr.md) §3.4 / `forgelm/compliance.py:104-114`). İki primitif kasıtlı olarak ayrıdır. 2. Audit log'a `data.erasure_requested` yazar — **hash'lenmiş** target_id (`SHA-256(salt + value)`) ve operatör-sağlanan justification ile. 3. Eşleşen JSONL satırını `id` (veya `row_id`) field'ı ile bulur — re-order edilmiş bir dosyada sessiz yanlış-satır silmesini engellemek için satır-numarası fallback'i **reddedilir**. ForgeLM şu an bir id-doldurma yardımcısı sunmuyor (Phase 28 backlog'unda `forgelm audit --add-row-ids` flag'i bekliyor); id'siz corpus'lara sahip operatörler purge'den önce id'leri operatör-tarafı bir script ile (örn. `jq -c 'to_entries | with_entries(...)'` ya da tek-seferlik bir Python döngüsü) pre-populate etmeli. 4. Corpus'u atomic olarak yeniden yazar (kardeş bir temp dosyaya yazar + `os.replace`); operatörler ya tam silme-öncesi dosyayı ya da tam silme-sonrası dosyayı görür, asla kısmi state'i değil. diff --git a/docs/guides/getting-started-tr.md b/docs/guides/getting-started-tr.md index 3b83e6d4..e4241101 100644 --- a/docs/guides/getting-started-tr.md +++ b/docs/guides/getting-started-tr.md @@ -79,7 +79,7 @@ $ export FORGELM_OPERATOR="gha:Acme/repo:training:run-${GITHUB_RUN_ID}" $ forgelm doctor # şimdi [+ pass] operator.identity ``` -Bu kimlik trainer'ın ürettiği her audit-log girişine stamp'lenir — bir EU AI Act Madde 12 incelemecisinin model provenance'ı atfetmek için okuduğu şeydir. Önerilen namespace şeması için [`docs/qms/access_control.md`](../qms/access_control.md)'a bakın. +Bu kimlik trainer'ın ürettiği her audit-log girişine stamp'lenir — bir EU AI Act Madde 12 incelemecisinin model provenance'ı atfetmek için okuduğu şeydir. Önerilen namespace şeması için [`docs/qms/access_control.md`](../qms/access_control-tr.md)'a bakın. ### Adım 4 — Gated-model kimlik doğrulamayı devredin diff --git a/docs/guides/human_approval_gate-tr.md b/docs/guides/human_approval_gate-tr.md index ded1b5c2..8501fe62 100644 --- a/docs/guides/human_approval_gate-tr.md +++ b/docs/guides/human_approval_gate-tr.md @@ -62,7 +62,7 @@ O tek flag, bu config'i tüketen her koşum için kapıyı açmaya yeter. Kapı ## 2. Trainer'ın CI runner'ını yapılandır -`forgelm`'i çalıştıran CI runner `FORGELM_OPERATOR`'ı makineokunabilir bir kimliğe SET ETMELİDİR (per [`../qms/access_control.md`](../qms/access_control.md) §3.2): +`forgelm`'i çalıştıran CI runner `FORGELM_OPERATOR`'ı makineokunabilir bir kimliğe SET ETMELİDİR (per [`../qms/access_control.md`](../qms/access_control-tr.md) §3.2): ```yaml # .github/workflows/train.yml @@ -90,7 +90,7 @@ steps: İki tamamlayıcı mekanizma: -**Webhook (push).** Run config'inde `webhook.url_env: SLACK_WEBHOOK_URL` set et. Trainer `notify_awaiting_approval`'ı run id ve metric özetiyle fırlatır; Slack/Teams on-call rotasyonuna yönlendirir. Webhook secret hijyeni için bkz. [`../qms/access_control.md`](../qms/access_control.md) §7. +**Webhook (push).** Run config'inde `webhook.url_env: SLACK_WEBHOOK_URL` set et. Trainer `notify_awaiting_approval`'ı run id ve metric özetiyle fırlatır; Slack/Teams on-call rotasyonuna yönlendirir. Webhook secret hijyeni için bkz. [`../qms/access_control.md`](../qms/access_control-tr.md) §7. **Keşif (pull).** Zamanlanmış bir CI job'ı: @@ -149,7 +149,7 @@ Her satır önceki satıra bağlayan bir `prev_hash` (SHA-256) ve `FORGELM_AUDIT Onaylayanın `FORGELM_OPERATOR`'ı trainer'ınkinden FARKLI OLMALIDIR. ForgeLM bunu zorunlu kılmaz — bu deployer-tarafı IdP kontrolüdür — ancak audit zinciri her ikisini kaydeder; ihlal post-hoc tespit edilebilir. -Kanonik tespit cookbook'u [`../qms/access_control.md`](../qms/access_control.md) §6'da bulunur. Kolaylık için burada da: +Kanonik tespit cookbook'u [`../qms/access_control.md`](../qms/access_control-tr.md) §6'da bulunur. Kolaylık için burada da: ```bash # 1. Önce zincir bütünlüğünü doğrula (positional log_path; bu subcommand'da @@ -228,6 +228,6 @@ jq -rs ' - [`../usermanuals/tr/compliance/human-approval-gate.md`](../usermanuals/tr/compliance/human-approval-gate.md) — bu rehberle eşleşen deployer-yüzlü kullanıcı kılavuzu sayfası. - [`../reference/approve_subcommand-tr.md`](../reference/approve_subcommand-tr.md) — `approve` / `reject` için flag-başına, event-başına referans. - [`../reference/approvals_subcommand-tr.md`](../reference/approvals_subcommand-tr.md) — `approvals` için flag-başına, event-başına referans. -- [`../qms/access_control.md`](../qms/access_control.md) §6 — kanonik segregation-of-duties cookbook. +- [`../qms/access_control.md`](../qms/access_control-tr.md) §6 — kanonik segregation-of-duties cookbook. - [`iso_soc2_deployer_guide-tr.md`](iso_soc2_deployer_guide-tr.md) §"Q5" — kapının kanıtını tüketen auditor walkthrough. - [`gdpr_erasure-tr.md`](gdpr_erasure-tr.md) — GDPR Madde 15 + 17 için kardeş akış. diff --git a/docs/guides/ingestion-tr.md b/docs/guides/ingestion-tr.md index 6bef76f1..bc7dc664 100644 --- a/docs/guides/ingestion-tr.md +++ b/docs/guides/ingestion-tr.md @@ -490,7 +490,7 @@ Wave 3 backlog'undadır. > (akademik posterler, geniş oluklu regülasyon yayınları) güvenilirdir > ama yayın-kalite iki-kolonlu makaleleri kaçırır. Histogram-tabanlı > bimodal-mode refactor Wave 3 takipi olarak izlenmektedir (bkz. -> [Faz 15 arşivi](../roadmap/completed-phases.md#phase-15-ingestion-pipeline-reliability-v060) +> [Faz 15 arşivi](../roadmap/completed-phases.md#phase-15--ingestion-pipeline-reliability-v060) > Wave 3 — multi-column layout extraction). ### Markdown-bilen splitter — `--strategy markdown` (Faz 12) diff --git a/docs/guides/ingestion.md b/docs/guides/ingestion.md index e3d469f5..2cdd3440 100644 --- a/docs/guides/ingestion.md +++ b/docs/guides/ingestion.md @@ -481,7 +481,7 @@ is on the Wave 3 backlog. > wide-gutter regulatory publications) but misses publication-grade > two-column papers. A histogram-based bimodal-mode refactor is tracked > as a Wave 3 follow-up (see -> [Phase 15 archive](../roadmap/completed-phases.md#phase-15-ingestion-pipeline-reliability-v060) +> [Phase 15 archive](../roadmap/completed-phases.md#phase-15--ingestion-pipeline-reliability-v060) > Wave 3 — multi-column layout extraction). ### Markdown-aware splitter — `--strategy markdown` (Phase 12) diff --git a/docs/guides/library_api-tr.md b/docs/guides/library_api-tr.md index d1762c59..af1cf8e3 100644 --- a/docs/guides/library_api-tr.md +++ b/docs/guides/library_api-tr.md @@ -245,7 +245,7 @@ Kütüphane `logging.basicConfig()` **çağırmaz**. Uygulamanız çağırır. B ### `AuditLogger`'ı fork'lar arasında paylaşmak -`AuditLogger` POSIX `fcntl.flock` (veya Windows'ta `msvcrt.locking`) kullanır. File handle'ı `os.fork()` çocukları arasında paylaşmak desteklenmez — her child aynı `output_dir`'i işaret eden kendi logger'ını kurmalıdır. Tüm yazımlar lock'u edindiği için zincir tutarlı kalır. +`AuditLogger` POSIX `fcntl.flock` kullanır; Windows'ta süreçler-arası kilit **yoktur** (advisory flock yardımcısı no-op'tur). File handle'ı `os.fork()` çocukları arasında paylaşmak desteklenmez — her child aynı `output_dir`'i işaret eden kendi logger'ını kurmalıdır. POSIX'te tüm yazımlar lock'u edindiği için zincir tutarlı kalır; Windows'ta aynı `output_dir`'e karşı eşzamanlı süreç çalıştırmayın (her koşu için ayrı bir `output_dir` kullanın). ### Hot path'te yeniden import @@ -268,6 +268,6 @@ Library API ve CLI aynı wheel'i, aynı audit log'u ve aynı lock semantik'ini p - [`../reference/library_api_reference-tr.md`](../reference/library_api_reference-tr.md) — tam sembol referansı. - [`../reference/audit_event_catalog-tr.md`](../reference/audit_event_catalog-tr.md) — `AuditLogger.log_event`'in kabul ettiği her olay. - [`../reference/configuration-tr.md`](../reference/configuration-tr.md) — `ForgeConfig` alan-by-alan referans. -- [`cicd_pipeline.md`](cicd_pipeline.md) — bu rehberin CLI karşılığı. +- [`cicd_pipeline-tr.md`](cicd_pipeline-tr.md) — bu rehberin CLI karşılığı. - [`iso_soc2_deployer_guide-tr.md`](iso_soc2_deployer_guide-tr.md) — audit katı cookbook'u (kütüphane çağıranları aynı artefaktları görür). - [`../design/library_api.md`](../design/library_api.md) — Faz 18 tasarım. diff --git a/docs/guides/library_api.md b/docs/guides/library_api.md index caa7b4e3..cad4f768 100644 --- a/docs/guides/library_api.md +++ b/docs/guides/library_api.md @@ -244,7 +244,7 @@ The library does **not** call `logging.basicConfig()`. Your application does. If ### Sharing `AuditLogger` across forks -`AuditLogger` uses POSIX `fcntl.flock` (or `msvcrt.locking` on Windows). Sharing the file handle across `os.fork()` children is unsupported — each child must construct its own logger pointing at the same `output_dir`. The chain stays consistent because all writes acquire the lock. +`AuditLogger` uses POSIX `fcntl.flock`; on Windows there is **no** cross-process lock (the advisory flock helper is a no-op). Sharing the file handle across `os.fork()` children is unsupported — each child must construct its own logger pointing at the same `output_dir`. On POSIX the chain stays consistent because all writes acquire the lock; on Windows, do not run concurrent processes against the same `output_dir` (use a distinct `output_dir` per run). ### Re-importing on a hot path diff --git a/docs/guides/performance-tr.md b/docs/guides/performance-tr.md index 189c78d2..30c6346a 100644 --- a/docs/guides/performance-tr.md +++ b/docs/guides/performance-tr.md @@ -192,4 +192,4 @@ GaLore belleği compute ile takas eder. Projeksiyon matris çarpımı gerçek ov - [`library_api-tr.md`](library_api-tr.md) — bu düğmeleri Python'dan çağırma. - [`../standards/coding.md`](../standards/coding.md) — ForgeLM'in dayattığı lazy-import standardı. - [`ingestion-tr.md`](ingestion-tr.md) — chunker kullanıcı-bakışlı derinlik. -- [`alignment.md`](alignment.md) — GaLore + 4-bit + packing'in ne zaman uygun olduğu. +- [`alignment.md`](alignment-tr.md) — GaLore + 4-bit + packing'in ne zaman uygun olduğu. diff --git a/docs/guides/pipeline-tr.md b/docs/guides/pipeline-tr.md index 311f5d3b..fcfdc65e 100644 --- a/docs/guides/pipeline-tr.md +++ b/docs/guides/pipeline-tr.md @@ -309,7 +309,7 @@ Sıfır olmayan exit kodu ihlalleri keşfedildikleri sırayla listeler. ## Cross-references -- Faz 14 yayınlanan kapsam: [docs/roadmap/completed-phases.md](../roadmap/completed-phases.md#phase-14-multi-stage-pipeline-chains-v070) +- Faz 14 yayınlanan kapsam: [docs/roadmap/completed-phases.md](../roadmap/completed-phases.md#phase-14--multi-stage-pipeline-chains-v070) - Faz 14.5 takip planı: [docs/roadmap/phase-14-5-pipeline-hardening.md](../roadmap/phase-14-5-pipeline-hardening.md) - Roadmap girişi: [docs/roadmap-tr.md](../roadmap-tr.md) - Annex IV doğrulayıcısı: `forgelm verify-annex-iv --pipeline ` (CLI help'e bakın) diff --git a/docs/guides/pipeline.md b/docs/guides/pipeline.md index 0bccdf62..bd3cef3d 100644 --- a/docs/guides/pipeline.md +++ b/docs/guides/pipeline.md @@ -306,7 +306,7 @@ discovered. ## Cross-references -- Phase 14 shipped scope: [docs/roadmap/completed-phases.md](../roadmap/completed-phases.md#phase-14-multi-stage-pipeline-chains-v070) +- Phase 14 shipped scope: [docs/roadmap/completed-phases.md](../roadmap/completed-phases.md#phase-14--multi-stage-pipeline-chains-v070) - Phase 14.5 follow-up plan: [docs/roadmap/phase-14-5-pipeline-hardening.md](../roadmap/phase-14-5-pipeline-hardening.md) - Roadmap entry: [docs/roadmap.md](../roadmap.md) - Annex IV verifier: `forgelm verify-annex-iv --pipeline ` (see CLI help) diff --git a/docs/guides/troubleshooting-tr.md b/docs/guides/troubleshooting-tr.md index f664b60a..da290656 100644 --- a/docs/guides/troubleshooting-tr.md +++ b/docs/guides/troubleshooting-tr.md @@ -65,6 +65,8 @@ pip install forgelm[eval] Etkili batch size denemeler arasında korunur. Her deneme audit trail'e loglanır. + > **Yeniden deneme yörüngesi hakkında not.** Başarılı bir yeniden deneme, trainer'ı küçük batch size ile sıfırdan bir optimizer ve LR scheduler (adım 0) kurarak yeniden oluşturur; çalışmayı açıkça `--resume ` ile başlatmadıysanız OOM'un oluştuğu adımdan devam etmez (o durumda ilgili checkpoint'e geri sarılır). Bu nedenle optimizasyon yörüngesi kesintisiz bir çalışmadan biraz farklıdır. Bu beklenen bir durumdur — bit düzeyinde tekrar üretilebilirlik için otomatik kurtarmaya güvenmek yerine batch size'ı manuel olarak sabitleyin. + 3. **Batch size'ı manuel azalt**: ```yaml training: @@ -155,24 +157,28 @@ Long-context eğitimi (büyük `sliding_window_attention` ya da RoPE ölçekleme) VRAM kullanımını ciddi şekilde artırır. Hafifletmek için: 1. **Sliding window boyutunu azalt**: + ```yaml training: sliding_window_attention: 2048 # 4096'dan indir ``` 2. **Gradient checkpointing'i aç** (hız pahasına VRAM azaltır): + ```yaml training: gradient_checkpointing: true ``` -3. **Sample packing kullan** (padding israfını azalt): +3. **Sekans paketleme kullan** (padding israfını azalt): + ```yaml training: - sample_packing: true + packing: true ``` 4. **Ek bellek tasarrufu için GaLore ile birleştir**: + ```yaml training: galore_enabled: true diff --git a/docs/guides/troubleshooting.md b/docs/guides/troubleshooting.md index c75cd1f2..fc8df682 100644 --- a/docs/guides/troubleshooting.md +++ b/docs/guides/troubleshooting.md @@ -63,6 +63,8 @@ pip install forgelm[eval] ``` Effective batch size is preserved across retries. Each attempt is logged to the audit trail. + > **Note on the retry trajectory.** A successful retry rebuilds the trainer with a fresh optimizer and LR scheduler (step 0) at the smaller batch size; it does not continue from the exact step where the OOM fired unless you launched the run with an explicit `--resume ` (in which case it rewinds to that checkpoint). The optimization trajectory therefore differs slightly from an uninterrupted run. This is expected — for bit-for-bit reproducibility, fix the batch size manually instead of relying on auto-recovery. + 3. **Reduce batch size manually**: ```yaml training: @@ -148,24 +150,28 @@ If you need both multi-GPU and GaLore, use `galore_adamw` or `galore_adamw_8bit` Long-context training (large `sliding_window_attention` or RoPE scaling) significantly increases VRAM usage. To mitigate: 1. **Reduce sliding window size**: + ```yaml training: sliding_window_attention: 2048 # down from 4096 ``` 2. **Enable gradient checkpointing** (reduces VRAM at cost of speed): + ```yaml training: gradient_checkpointing: true ``` -3. **Use sample packing** to reduce padding waste: +3. **Use sequence packing** to reduce padding waste: + ```yaml training: - sample_packing: true + packing: true ``` 4. **Combine with GaLore** for additional memory savings: + ```yaml training: galore_enabled: true diff --git a/docs/qms/README-tr.md b/docs/qms/README-tr.md index 30b0f0ef..96630826 100644 --- a/docs/qms/README-tr.md +++ b/docs/qms/README-tr.md @@ -26,7 +26,7 @@ ## Bkz. -- [Uyumluluk özeti](../reference/compliance_summary.md) — özlü EU AI Act + ISO + SOC 2 kanıt haritası. +- [Uyumluluk özeti](../reference/compliance_summary-tr.md) — özlü EU AI Act + ISO + SOC 2 kanıt haritası. - [ISO / SOC 2 operatör (deployer) rehberi](../guides/iso_soc2_deployer_guide-tr.md) — Wave 4 denetim cookbook'u. - [Audit event kataloğu](../reference/audit_event_catalog-tr.md) — tam event vocabulary. diff --git a/docs/qms/sop_change_management-tr.md b/docs/qms/sop_change_management-tr.md index be1f2b11..c010fc36 100644 --- a/docs/qms/sop_change_management-tr.md +++ b/docs/qms/sop_change_management-tr.md @@ -114,11 +114,14 @@ attribute, çift kontrollü ve forensic olarak kayıtlıdır. ### 4.2 Yapılandırma drift tespiti -`tools/regenerate_config_doc.py` (Phase 16) -`docs/reference/configuration.md`'yi (ve `-tr.md` mirror'ını) -Pydantic schema'dan yeniden üretir. CI diff guard çalıştırır; -karşılık gelen doc güncellemesi olmadan bir config-schema değişimi -build'i fail eder. +`tools/check_field_descriptions.py --strict` (Phase 16) her CI build'inde +(`ci.yml`) çalışır ve `forgelm/config.py`'deki herhangi bir Pydantic +alanı `description=` argümanından yoksunsa non-zero çıkar. Operatöre dönük +`docs/reference/configuration.md` (ve `-tr.md` mirror'ı) elle bakımı +yapılır ve schema'ya karşı incelenir; `tools/check_bilingual_parity.py --strict` +EN↔TR yapısal pariteyi zorlar. Otomatik üretici yoktur — guard her schema +alanının yetkili açıklama metni taşıdığını garanti eder ve değişiklik +incelemesi referans doc'unun bunu yansıttığını kontrol eder. Bu, schema'nın evrildiği ama operatöre dönük doc'un geri kaldığı "doc drift" arıza modunu kapatır. ISO A.5.36 bunu zorunlu mekanizma @@ -149,4 +152,5 @@ Herhangi bir modeli tam eğitim yapılandırması ve verisine geri izlemek için | Versiyon | Tarih | Yazar | Değişiklikler | |---------|------|--------|---------| | 1.0 | [DATE] | [AUTHOR] | İlk versiyon | -| 1.1 | 2026-05-05 | Wave 4 / Faz 23 | §4 CI-gates-as-change-control tablosu eklendi (11 gate × ISO kontrolleri); §4.1 Madde 14 approval gate CAB substitute olarak; §4.2 `regenerate_config_doc.py` üzerinden config-drift tespiti; §4.3 `generate_sbom.py` + determinism testi üzerinden SBOM drift tespiti | +| 1.1 | 2026-05-05 | Wave 4 / Faz 23 | §4 CI-gates-as-change-control tablosu eklendi (11 gate × ISO kontrolleri); §4.1 Madde 14 approval gate CAB substitute olarak; §4.2 `check_field_descriptions.py` + elle doc incelemesi üzerinden config-drift tespiti; §4.3 `generate_sbom.py` + determinism testi üzerinden SBOM drift tespiti | +| 1.2 | 2026-06-14 | Wave 1 / H7 | §4.2 düzeltildi: config-drift kontrolü `check_field_descriptions.py --strict` + elle doc incelemesi + bilingual-parity guard'dır; önceden atıfta bulunulan `regenerate_config_doc.py` otomatik üreticisi hiç var olmadı | diff --git a/docs/qms/sop_change_management.md b/docs/qms/sop_change_management.md index 8162314b..79336c3d 100644 --- a/docs/qms/sop_change_management.md +++ b/docs/qms/sop_change_management.md @@ -109,10 +109,14 @@ attributed, dual-controlled, and forensically recorded. ### 4.2 Configuration drift detection -`tools/regenerate_config_doc.py` (Phase 16) regenerates -`docs/reference/configuration.md` (and `-tr.md` mirror) from the -Pydantic schema. CI runs the diff guard; a config-schema change -without a corresponding doc update fails the build. +`tools/check_field_descriptions.py --strict` (Phase 16) runs on every +CI build (`ci.yml`) and exits non-zero if any Pydantic field in +`forgelm/config.py` lacks a `description=` argument. The operator-facing +`docs/reference/configuration.md` (and its `-tr.md` mirror) are +maintained by hand and reviewed against the schema; `tools/check_bilingual_parity.py --strict` +enforces EN↔TR structural parity. There is no autogenerator — the guard +guarantees that every schema field carries authoritative description text, +and change review checks that the reference doc mirrors it. This closes the "doc drift" failure mode where the schema evolves but the operator-facing doc lags. ISO A.5.36 cites this as a @@ -143,4 +147,5 @@ Use these to trace any model back to its exact training configuration and data. | Version | Date | Author | Changes | |---------|------|--------|---------| | 1.0 | [DATE] | [AUTHOR] | Initial version | -| 1.1 | 2026-05-05 | Wave 4 / Faz 23 | Added §4 CI-gates-as-change-control table (11 gates × ISO controls); §4.1 Article 14 approval gate as CAB substitute; §4.2 config-drift detection via `regenerate_config_doc.py`; §4.3 SBOM drift detection via `generate_sbom.py` + determinism test | +| 1.1 | 2026-05-05 | Wave 4 / Faz 23 | Added §4 CI-gates-as-change-control table (11 gates × ISO controls); §4.1 Article 14 approval gate as CAB substitute; §4.2 config-drift detection via `check_field_descriptions.py` + manual doc review; §4.3 SBOM drift detection via `generate_sbom.py` + determinism test | +| 1.2 | 2026-06-14 | Wave 1 / H7 | Corrected §4.2: the config-drift control is `check_field_descriptions.py --strict` + manual doc review + bilingual-parity guard; the previously cited `regenerate_config_doc.py` autogenerator never existed | diff --git a/docs/qms/sop_data_management-tr.md b/docs/qms/sop_data_management-tr.md index 58b04f28..42f83b04 100644 --- a/docs/qms/sop_data_management-tr.md +++ b/docs/qms/sop_data_management-tr.md @@ -73,7 +73,7 @@ ForgeLM otomatik kontroller: - Trainer tipine göre format doğrulama (SFT, DPO, KTO, GRPO) - Metin temizleme (`clean_text: true`) - **ForgeLM audit pipeline (v0.5.0+, `forgelm audit `; legacy - `forgelm --data-audit` deprecated, v0.7.0'da kaldırılma planlandı)** — + `forgelm --data-audit` deprecated, v0.8.0'da kaldırılma planlandı)** — per-split sample sayıları, uzunluk dağılımı, top-3 dil tespiti, near-duplicate oranı (varsayılan 64-bit simhash, veya >50K-row corpora için `--dedup-method minhash` üzerinden **MinHash LSH**), diff --git a/docs/qms/sop_data_management.md b/docs/qms/sop_data_management.md index a680f401..ffba402e 100644 --- a/docs/qms/sop_data_management.md +++ b/docs/qms/sop_data_management.md @@ -72,7 +72,7 @@ ForgeLM automated checks: - Format validation per trainer type (SFT, DPO, KTO, GRPO) - Text cleaning (`clean_text: true`) - **ForgeLM audit pipeline (v0.5.0+, `forgelm audit `; legacy - `forgelm --data-audit` deprecated, removal scheduled v0.7.0)** — + `forgelm --data-audit` deprecated, removal scheduled v0.8.0)** — produces `data_audit_report.json` with per-split sample counts, length distribution, top-3 language detection, near-duplicate rate (default 64-bit simhash, or **MinHash LSH** via diff --git a/docs/reference/approvals_subcommand-tr.md b/docs/reference/approvals_subcommand-tr.md index 4b7c65a3..0a2e5a7a 100644 --- a/docs/reference/approvals_subcommand-tr.md +++ b/docs/reference/approvals_subcommand-tr.md @@ -3,7 +3,7 @@ > **Hedef kitle:** Madde 14 insan-onay kararı bekleyen koşumları keşfeden ForgeLM operatörleri ve tek bir koşumun audit zincirini uçtan uca okuyan denetçiler. > **Ayna:** [approvals_subcommand.md](approvals_subcommand.md) -`forgelm approvals`, [`forgelm approve` / `forgelm reject`](approve_subcommand.md)'in **keşif tamamlayıcısıdır** (Phase 37). `--output-dir` altındaki `audit_log.jsonl`'i tarar ve karar bekleyen tüm koşumları (`--pending`) veya tek bir koşum için tam audit zincirini (`--show RUN_ID`) raporlar. +`forgelm approvals`, [`forgelm approve` / `forgelm reject`](approve_subcommand-tr.md)'in **keşif tamamlayıcısıdır** (Phase 37). `--output-dir` altındaki `audit_log.jsonl`'i tarar ve karar bekleyen tüm koşumları (`--pending`) veya tek bir koşum için tam audit zincirini (`--show RUN_ID`) raporlar. Subcommand salt-okunurdur: audit log'u, staging dizinini veya başka herhangi bir on-disk artefact'ı asla değiştirmez. @@ -163,7 +163,7 @@ Daha zengin bir policy kuran operatörler (örn. "herhangi bir bekleyen karar N ## Bkz. -- [`approve_subcommand.md`](approve_subcommand.md) — terminal karar tamamlayıcısı (`approve` / `reject`). -- [`../guides/human_approval_gate.md`](../guides/human_approval_gate.md) — deployer akışı. -- [`audit_event_catalog.md`](audit_event_catalog.md) — bu subcommand tarafından okunan `human_approval.*` satırları dahil tam event sözlüğü. -- [`../qms/access_control.md`](../qms/access_control.md) §6 — segregation-of-duties cookbook (`--show`'un projelendirdiği `human_approval.granted` satırlarını kullanır). +- [`approve_subcommand.md`](approve_subcommand-tr.md) — terminal karar tamamlayıcısı (`approve` / `reject`). +- [`../guides/human_approval_gate.md`](../guides/human_approval_gate-tr.md) — deployer akışı. +- [`audit_event_catalog.md`](audit_event_catalog-tr.md) — bu subcommand tarafından okunan `human_approval.*` satırları dahil tam event sözlüğü. +- [`../qms/access_control.md`](../qms/access_control-tr.md) §6 — segregation-of-duties cookbook (`--show`'un projelendirdiği `human_approval.granted` satırlarını kullanır). diff --git a/docs/reference/approve_subcommand-tr.md b/docs/reference/approve_subcommand-tr.md index 72119d85..62e2ed0d 100644 --- a/docs/reference/approve_subcommand-tr.md +++ b/docs/reference/approve_subcommand-tr.md @@ -5,7 +5,7 @@ `forgelm approve` ve `forgelm reject`, **EU AI Act Madde 14** insan-gözetim terminal-karar subcommand'larıdır (Phase 9). Eğitim koşusu kod 4 (`EXIT_AWAITING_APPROVAL`) ile çıktığında, diskte `final_model.staging/` ve zincirde `human_approval.required` event'iyle duraklar; yetkili reviewer ardından `approve` (terfi için) veya `reject` (atmak için) çalıştırır. -Listeleme tamamlayıcısı için bkz. [`approvals_subcommand.md`](approvals_subcommand.md). Deployer akışı (CI koşusu 4 ile çıkar → reviewer'a haber gider → CLI çağrısı → audit) için bkz. [`../guides/human_approval_gate.md`](../guides/human_approval_gate.md). +Listeleme tamamlayıcısı için bkz. [`approvals_subcommand.md`](approvals_subcommand-tr.md). Deployer akışı (CI koşusu 4 ile çıkar → reviewer'a haber gider → CLI çağrısı → audit) için bkz. [`../guides/human_approval_gate.md`](../guides/human_approval_gate-tr.md). ## Synopsis @@ -17,7 +17,7 @@ forgelm reject run_id --output-dir DIR [--comment TEXT] [--output-format {text,json}] ``` -İki subcommand da **positional `run_id`** alır (`--run-id` flag'i YOK). Bu, `forgelm/cli/subcommands/_approve.py` içindeki CLI yüzeyiyle ve [`../qms/access_control.md`](../qms/access_control.md) §6'daki `forgelm approve ` cookbook'uyla eşleşir. +İki subcommand da **positional `run_id`** alır (`--run-id` flag'i YOK). Bu, `forgelm/cli/subcommands/_approve.py` içindeki CLI yüzeyiyle ve [`../qms/access_control.md`](../qms/access_control-tr.md) §6'daki `forgelm approve ` cookbook'uyla eşleşir. | Argüman / flag | Zorunlu | Açıklama | |---|---|---| @@ -58,7 +58,7 @@ Staging dizini **silinmez** — operatörler red kaydı zincire girdikten sonra 2. `getpass.getuser()` (OS-raporlu kullanıcı adı). 3. İkisi de başarısızsa `"anonymous"`. -**Madde 14 segregation of duties.** Onaylayanın `FORGELM_OPERATOR`'ı eğiticininkinden farklı OLMALIDIR (ISO 27001:2022 A.5.3, SOC 2 CC1.5). ForgeLM farkı zorunlu kılmaz — bu deployer-tarafı IdP kontrolüdür — ancak audit zinciri her ikisini de kaydeder; auditor [`../qms/access_control.md`](../qms/access_control.md) §6'daki `jq -rs` cookbook'uyla ihlalleri tespit edebilir: +**Madde 14 segregation of duties.** Onaylayanın `FORGELM_OPERATOR`'ı eğiticininkinden farklı OLMALIDIR (ISO 27001:2022 A.5.3, SOC 2 CC1.5). ForgeLM farkı zorunlu kılmaz — bu deployer-tarafı IdP kontrolüdür — ancak audit zinciri her ikisini de kaydeder; auditor [`../qms/access_control.md`](../qms/access_control-tr.md) §6'daki `jq -rs` cookbook'uyla ihlalleri tespit edebilir: ```bash jq -rs ' @@ -75,7 +75,7 @@ Yazdırılan herhangi bir satır segregation-of-duties ihlalidir. ## Yayılan audit event'leri -İki event de [`audit_event_catalog.md`](audit_event_catalog.md)'deki ortak zarfı taşır. Katalog satırları kolaylık için burada da listelenir. +İki event de [`audit_event_catalog.md`](audit_event_catalog-tr.md)'deki ortak zarfı taşır. Katalog satırları kolaylık için burada da listelenir. | Event | Ne zaman yayılır | Anahtar payload | |---|---|---| @@ -133,8 +133,8 @@ Hata (her ikisi, `_output_error_and_exit` tarafından): ## Bkz. -- [`approvals_subcommand.md`](approvals_subcommand.md) — keşif tamamlayıcısı (`--pending` / `--show RUN_ID`). -- [`../guides/human_approval_gate.md`](../guides/human_approval_gate.md) — deployer akışı. -- [`audit_event_catalog.md`](audit_event_catalog.md) — zarf spec'i ile birlikte tam event sözlüğü. -- [`../qms/access_control.md`](../qms/access_control.md) §6 — segregation-of-duties cookbook. +- [`approvals_subcommand.md`](approvals_subcommand-tr.md) — keşif tamamlayıcısı (`--pending` / `--show RUN_ID`). +- [`../guides/human_approval_gate.md`](../guides/human_approval_gate-tr.md) — deployer akışı. +- [`audit_event_catalog.md`](audit_event_catalog-tr.md) — zarf spec'i ile birlikte tam event sözlüğü. +- [`../qms/access_control.md`](../qms/access_control-tr.md) §6 — segregation-of-duties cookbook. - [`../usermanuals/tr/compliance/human-oversight.md`](../usermanuals/tr/compliance/human-oversight.md) — operatör-yüzlü kullanıcı kılavuzu sayfası. diff --git a/docs/reference/audit_event_catalog-tr.md b/docs/reference/audit_event_catalog-tr.md index 79befed8..521e7370 100644 --- a/docs/reference/audit_event_catalog-tr.md +++ b/docs/reference/audit_event_catalog-tr.md @@ -31,8 +31,9 @@ Hash zinciri, satır diske düştükten (`flush` + `fsync`) sonra ilerler; kirli | `training.started` | Trainer fine-tuning koşusunu başlatır. | _(payload yok — yalnız zarf)_ | 12 | | `training.oom_recovery` | OOM kurtarma yolu `per_device_train_batch_size`'i yarıya indirip yeniden denedi (eğitim-arası event). | `old_batch_size`, `new_batch_size`, `new_grad_accum` | 12 / 15 | | `benchmark.evaluation_completed` | `lm-eval-harness` yapılandırılmış benchmark suite'inin değerlendirmesini bitirdi. | `passed`, `average`, `scores` | 15 | -| `safety.evaluation_completed` | Güvenlik değerlendirmesi bitti (Llama Guard / ShieldGemma koşusu). | `passed`, `safe_ratio`, `safety_score`, `categories` | 15 | +| `safety.evaluation_completed` | Güvenlik değerlendirmesi bitti (Llama Guard / ShieldGemma koşusu). | `passed`, `safe_ratio`, `total_count`, `safety_score`, `categories` | 15 | | `judge.evaluation_completed` | LLM-as-judge skorlaması bitti. | `passed`, `average_score` | 15 | +| `evaluation.loss_gate_completed` | Kayıp/eval-loss otomatik geri-alma kapısı yapılandırılmış eşiklere göre karar verdi (geçti veya kaldı). | `passed`, `eval_loss`, `max_acceptable_loss`, `baseline_loss` | 15 | | `pipeline.completed` | Uçtan uca CLI koşusu (eğitim + değerlendirme + dışa aktarma) 0 koduyla biter. | `success`, `metrics_summary` | 12 | | `pipeline.failed` | Pipeline tamamlanmadan bir hata ile iptal olur. | `error` | 12 | | `pipeline.started` | Çok-aşamalı pipeline orchestrator yeni bir koşu başlattı (`--resume-from` değil). | `pipeline_run_id`, `config_hash`, `stage_count`, `stage_names` | 12 | @@ -64,8 +65,10 @@ Hash zinciri, satır diske düştükten (`flush` + `fsync`) sonra ilerler; kirli | Event | Ne zaman emit edilir | Payload | Madde | |----------------------------------|-------------------------------------------------------------------------------|--------------------------------------------------|---------------| | `compliance.governance_exported` | Madde 10 veri yönetişim raporu diske yazıldı. | `output_path`, `dataset_count` | 10 | +| `compliance.governance_section_missing` | Yönetişim raporu yazıldı ancak Madde 10 veri-kalitesi bölümü eksikti (`data_audit_report.json` yok). | `section`, `expected_path` | 10 | | `compliance.governance_failed` | Yönetişim raporu üretimi iptal edildi (örn. şema uyumsuzluğu). | `reason` | 10 | -| `compliance.artifacts_exported` | Ek IV teknik dokümantasyon paketi (manifest, model card, audit zip) yazıldı. | `output_dir`, `files` | 11, Ek IV | +| `compliance.artifacts_exported` | Ek IV teknik dokümantasyon paketi (manifest, model card, audit zip) yazıldı. | `output_dir`, `files`, `governance_ok` | 11, Ek IV | +| `compliance.artifacts_export_failed` | Ek IV / Madde 11 manifest export'u başarısız oldu veya yarım kaldı (disk dolu, SIGKILL, serileştirme hatası). | `reason` | 11, Ek IV | ### Madde 17 — GDPR Silinme Hakkı (Phase 21 — `forgelm purge`) @@ -111,7 +114,7 @@ Hash zinciri, satır diske düştükten (`flush` + `fsync`) sonra ilerler; kirli 1. Mevcut isim alanlarını (`training.*`, `compliance.*`, `audit.*`, `human_approval.*`, `model.*`, `cli.*`) takip eden noktalı bir ad seçin. 2. Yukarıdaki tabloya, payload anahtarları ve desteklediği Madde dahil olmak üzere bir satır ekleyin. -3. Satırı [audit_event_catalog.md](audit_event_catalog.md)'ye yansıtın. +3. Aynı satırı İngilizce kardeş katalog `audit_event_catalog.md`'ye de ekleyin (EN ↔ TR senkron kalmalı). 4. `AuditLogger.log_event(event, **payload)` üzerinden emit edin. `audit_log.jsonl`'a doğrudan `json.dump` çağırmayın; hash zinciri kanonik yazıcıya bağımlıdır. ## Tampering-evidence özeti @@ -128,8 +131,10 @@ Hash zinciri, satır diske düştükten (`flush` + `fsync`) sonra ilerler; kirli Webhook payload'ları (Slack / Teams / jenerik HTTP) operatör bildirimlerine kapsamlanmış ayrı bir sözlüktür, regülasyon kaydı değil. Webhook olayları `audit_log.jsonl`'a **eklenmez**; yan-kanal bildirim bus'ı üzerinde gider. Kanonik yaşam döngüsü sözlüğü ayrıca [logging-observability.md](../standards/logging-observability.md)'da da belgelenmiştir. -Bu beş yaşam döngüsü olayı, webhook alıcılarının `WebhookNotifier`'dan -beklemesi gereken **tek** olaylardır. Her biri, karşılık gelen bir +Bu sekiz olay, webhook alıcılarının `WebhookNotifier`'dan +beklemesi gereken **tek** olaylardır: beş tek-aşamalı yaşam döngüsü +olayı ve çok-aşamalı orkestratörün bunların yanı sıra emit ettiği +üç-olaylı `pipeline.*` ailesi. Her biri, karşılık gelen bir denetim günlüğü olayını yansıtır; böylece aşağı akıştaki bir operatör webhook ping → denetim girdisi korelasyonunu `run_name` + zaman damgasıyla kurabilir. Uygulama: `forgelm/webhook.py`. @@ -137,10 +142,13 @@ damgasıyla kurabilir. Uygulama: `forgelm/webhook.py`. | Webhook `event` | Denetim günlüğü karşılığı | Tetikleyici | Kapı (gate) | Zorunlu payload alanları | |---|---|---|---|---| | `training.start` | `training.started` | `train()` çağrıldı, model yüklenmeden önce. | `webhook.notify_on_start` | `run_name`, `status="started"` | -| `training.success` | `pipeline.completed` | Tüm kapılar geçildi, insan onayı gerekmiyor. | `webhook.notify_on_success` | `run_name`, `status="succeeded"`, `metrics` | +| `training.success` | `pipeline.completed` | Koşu revert veya bekleyen onay olmadan tamamlandı. `evaluation.auto_revert: true` ile tüm kapılar geçildi; varsayılan `auto_revert: false` ile bir kapı geçemediği halde yalnızca kaydedildiğinde de tetiklenir (model yine terfi eder). | `webhook.notify_on_success` | `run_name`, `status="succeeded"`, `metrics` | | `training.failure` | `pipeline.failed` | Eğitim sürecinin kendisi hata fırlattı (OOM, veri seti hatası, yakalanmayan istisna). | `webhook.notify_on_failure` | `run_name`, `status="failed"`, `reason` (maskelenmiş, ≤2048 karakter) | | `training.reverted` | `model.reverted` | Eğitim sonrası bir kapı (değerlendirme, güvenlik, hakem, benchmark) çalışmayı reddetti ve `_revert_model` adaptörleri sildi. | `webhook.notify_on_failure` | `run_name`, `status="reverted"`, `reason` (maskelenmiş, ≤2048 karakter) | | `approval.required` | `human_approval.required` | Çalışma başarılı oldu, `evaluation.require_human_approval=true`, model insan incelemesi için staging'de (EU AI Act Madde 14). | `webhook.notify_on_success` | `run_name`, `status="awaiting_approval"`, `model_path` | +| `pipeline.started` | `pipeline.started` | Çok-aşamalı bir pipeline koşusu başlar, herhangi bir aşama çalışmadan önce. | `webhook.notify_on_start` | `run_name`, `status="started"`, `stage_count` | +| `pipeline.completed` | `pipeline.completed` | Çok-aşamalı bir pipeline koşusu terminal durumuna ulaşır. Denetim olayıyla aynı adı paylaşır (bilinen wire/audit çakışması; payload alan-kümesiyle korele edin). | `webhook.notify_on_success` / `webhook.notify_on_failure` | `run_name`, `status`, `final_status`, `stopped_at` | +| `pipeline.stage_reverted` | `pipeline.stage_reverted` | Bir pipeline aşaması auto-revert olur, aşağı akış aşamaları skip işaretlenmeden önce. | `webhook.notify_on_failure` | `run_name`, `status="reverted"`, `stage_name`, `reason` (maskelenmiş, ≤2048 karakter) | ### Bu yaşam döngüsü durumlarından ikisinin neden ayrıldığı @@ -162,9 +170,9 @@ Her webhook olayı aynı zarfı taşır: ```json { - "event": "training.start | training.success | training.failure | training.reverted | approval.required", + "event": "training.start | training.success | training.failure | training.reverted | approval.required | pipeline.started | pipeline.completed | pipeline.stage_reverted", "run_name": "", - "status": "started | succeeded | failed | reverted | awaiting_approval", + "status": "started | succeeded | failed | reverted | awaiting_approval | completed | stopped_at_stage", "metrics": {"": , ...}, "reason": "", "model_path": "", diff --git a/docs/reference/audit_event_catalog.md b/docs/reference/audit_event_catalog.md index 6cbce070..dc63c10f 100644 --- a/docs/reference/audit_event_catalog.md +++ b/docs/reference/audit_event_catalog.md @@ -31,8 +31,9 @@ The hash chain advances after the line lands on disk (`flush` + `fsync`), so an | `training.started` | Trainer begins a fine-tuning run. | _(no payload — envelope only)_ | 12 | | `training.oom_recovery` | OOM recovery path halved `per_device_train_batch_size` and retried (mid-training event). | `old_batch_size`, `new_batch_size`, `new_grad_accum` | 12 / 15 | | `benchmark.evaluation_completed` | `lm-eval-harness` finished evaluating the configured benchmark suite. | `passed`, `average`, `scores` | 15 | -| `safety.evaluation_completed` | Safety evaluation finished (Llama Guard / ShieldGemma run). | `passed`, `safe_ratio`, `safety_score`, `categories` | 15 | +| `safety.evaluation_completed` | Safety evaluation finished (Llama Guard / ShieldGemma run). | `passed`, `safe_ratio`, `total_count`, `safety_score`, `categories` | 15 | | `judge.evaluation_completed` | LLM-as-judge scoring finished. | `passed`, `average_score` | 15 | +| `evaluation.loss_gate_completed` | Loss/eval-loss auto-revert gate decided (pass or fail) against the configured thresholds. | `passed`, `eval_loss`, `max_acceptable_loss`, `baseline_loss` | 15 | | `pipeline.completed` | End-to-end CLI run (training + evaluation + export) returned exit code 0. | `success`, `metrics_summary` | 12 | | `pipeline.failed` | Pipeline aborted with an error before completion. | `error` | 12 | | `pipeline.started` | Multi-stage pipeline orchestrator began a fresh run (not a `--resume-from`). | `pipeline_run_id`, `config_hash`, `stage_count`, `stage_names` | 12 | @@ -64,8 +65,10 @@ The hash chain advances after the line lands on disk (`flush` + `fsync`), so an | Event | When emitted | Payload | Article | |----------------------------------|-----------------------------------------------------------------------------|--------------------------------------------------|------------| | `compliance.governance_exported` | Article 10 data governance report written to disk. | `output_path`, `dataset_count` | 10 | +| `compliance.governance_section_missing` | Governance report exported but the Article 10 data-quality section was absent (no `data_audit_report.json`). | `section`, `expected_path` | 10 | | `compliance.governance_failed` | Governance report generation aborted (e.g., schema mismatch). | `reason` | 10 | -| `compliance.artifacts_exported` | Annex IV technical documentation bundle (manifest, model card, audit zip). | `output_dir`, `files` | 11, Annex IV | +| `compliance.artifacts_exported` | Annex IV technical documentation bundle (manifest, model card, audit zip). | `output_dir`, `files`, `governance_ok` | 11, Annex IV | +| `compliance.artifacts_export_failed` | Annex IV / Article 11 manifest export failed or was torn (disk full, SIGKILL, serialization error). | `reason` | 11, Annex IV | ### Article 17 — GDPR Right-to-Erasure (Phase 21 — `forgelm purge`) @@ -128,8 +131,10 @@ The hash chain advances after the line lands on disk (`flush` + `fsync`), so an Webhook payloads (Slack / Teams / generic HTTP) are a separate vocabulary scoped to operator notifications, not the regulatory record. Webhook events are **not** appended to `audit_log.jsonl`; they ride the side-channel notification bus. The canonical lifecycle vocabulary is also documented in [logging-observability.md](../standards/logging-observability.md). -These five lifecycle events are the **only** events that webhook -receivers should expect from `WebhookNotifier`. Each one mirrors a +These eight events are the **only** events that webhook +receivers should expect from `WebhookNotifier`: five single-stage +lifecycle events plus the three-event `pipeline.*` family the +multi-stage orchestrator emits alongside them. Each one mirrors a corresponding audit-log event so a downstream operator can correlate webhook ping → audit entry by `run_name` + timestamp. Implementation: `forgelm/webhook.py`. @@ -137,10 +142,13 @@ webhook ping → audit entry by `run_name` + timestamp. Implementation: | Webhook `event` | Audit-log mirror | Trigger | Gated by | Required payload fields | |---|---|---|---|---| | `training.start` | `training.started` | `train()` entered, before model load. | `webhook.notify_on_start` | `run_name`, `status="started"` | -| `training.success` | `pipeline.completed` | All gates passed, no human-approval requirement. | `webhook.notify_on_success` | `run_name`, `status="succeeded"`, `metrics` | +| `training.success` | `pipeline.completed` | Run completed without revert or pending approval. With `evaluation.auto_revert: true` all gates passed; with the default `auto_revert: false` it also fires when a gate failed but was only recorded (model still promoted). | `webhook.notify_on_success` | `run_name`, `status="succeeded"`, `metrics` | | `training.failure` | `pipeline.failed` | Training itself raised (OOM, dataset error, unhandled exception). | `webhook.notify_on_failure` | `run_name`, `status="failed"`, `reason` (masked, ≤2048 chars) | | `training.reverted` | `model.reverted` | A post-training gate (evaluation, safety, judge, benchmark) rejected the run and `_revert_model` deleted the adapters. | `webhook.notify_on_failure` | `run_name`, `status="reverted"`, `reason` (masked, ≤2048 chars) | | `approval.required` | `human_approval.required` | Run succeeded, `evaluation.require_human_approval=true`, model staged for review (EU AI Act Art. 14). | `webhook.notify_on_success` | `run_name`, `status="awaiting_approval"`, `model_path` | +| `pipeline.started` | `pipeline.started` | A multi-stage pipeline run begins, before any stage executes. | `webhook.notify_on_start` | `run_name`, `status="started"`, `stage_count` | +| `pipeline.completed` | `pipeline.completed` | A multi-stage pipeline run reaches its terminal state. Shares the audit event's name (a known wire/audit collision; correlate on the payload field-set). | `webhook.notify_on_success` / `webhook.notify_on_failure` | `run_name`, `status`, `final_status`, `stopped_at` | +| `pipeline.stage_reverted` | `pipeline.stage_reverted` | A pipeline stage auto-reverts, before downstream stages are skipped. | `webhook.notify_on_failure` | `run_name`, `status="reverted"`, `stage_name`, `reason` (masked, ≤2048 chars) | ### Why two of these split lifecycle states @@ -161,9 +169,9 @@ Every webhook event ships the same envelope: ```json { - "event": "training.start | training.success | training.failure | training.reverted | approval.required", + "event": "training.start | training.success | training.failure | training.reverted | approval.required | pipeline.started | pipeline.completed | pipeline.stage_reverted", "run_name": "", - "status": "started | succeeded | failed | reverted | awaiting_approval", + "status": "started | succeeded | failed | reverted | awaiting_approval | completed | stopped_at_stage", "metrics": {"": , ...}, "reason": "", "model_path": "", diff --git a/docs/reference/configuration-tr.md b/docs/reference/configuration-tr.md index a47b3a33..5a7221d2 100644 --- a/docs/reference/configuration-tr.md +++ b/docs/reference/configuration-tr.md @@ -52,9 +52,11 @@ Tam açıklamalı örnek için `config_template.yaml` dosyasına bakın. | Alan | Tip | Varsayılan | Açıklama | |------|-----|-----------|----------| | `trainer_type` | string | `"sft"` | `"sft"`, `"dpo"`, `"simpo"`, `"kto"`, `"orpo"`, `"grpo"` | -| `num_train_epochs` | int | `3` | Eğitim epoch sayısı | +| `max_steps` | int | `-1` | Sıkı adım üst sınırı. `-1` = `num_train_epochs` kullanılır; pozitif bir değer epoch'ları geçersiz kılar. | +| `num_train_epochs` | int | `3` | Eğitim epoch sayısı (yalnızca `max_steps == -1` iken dikkate alınır). | | `per_device_train_batch_size` | int | `4` | GPU başına batch boyutu | | `learning_rate` | float | `2e-5` | Öğrenme oranı | +| `early_stopping_patience` | int | `3` | Doğrulama kaybı iyileşmeden N değerlendirme sonra dur (yalnızca bir doğrulama bölünmesi varsa etkin). | | `report_to` | string | `"tensorboard"` | `"tensorboard"`, `"wandb"`, `"mlflow"`, `"none"` | #### OOM Recovery (Bellek Hatası Kurtarma) @@ -97,7 +99,7 @@ training: | `rope_scaling` | `Optional[Dict[str, Any]]` | `null` | RoPE ölçekleme yöntemi sözlüğü (`{"type": "linear", "factor": 2.0}` vs.). Desteklenen tipler: `"linear"`, `"dynamic"`, `"yarn"`, `"longrope"`. | | `neftune_noise_alpha` | float | `null` | NEFTune gürültü enjeksiyonu alpha değeri (ör. `5.0`) | | `sliding_window_attention` | int | `null` | Kayan pencere dikkat boyutu (token) | -| `sample_packing` | bool | `false` | Kısa örnekleri tam uzunluklu dizilere paketle | +| `sample_packing` | bool | `false` | **Kullanımdan kaldırıldı** — `packing` için takma ad (TRL tek bir packing düğmesi sunar). `true` ayarlamak `DeprecationWarning` ile `packing: true`'ya yönlendirir; v0.9.0'da kaldırılır. Bunun yerine `packing` kullanın. | #### GPU Maliyet Tahmini @@ -152,8 +154,11 @@ training: |------|-----|-----------|----------| | `enabled` | bool | `false` | lm-eval-harness benchmark'ları | | `tasks` | list | `[]` | Görev isimleri (ör. `["arc_easy", "hellaswag"]`) | +| `output_dir` | string | `null` | Benchmark sonuç JSON'unun yazılacağı yer. `null` = training `output_dir`. | | `min_score` | float | `null` | Minimum ortalama doğruluk | +> `enabled: true`, `tasks` içinde en az bir görev gerektirir — görevi olmayan etkin bir benchmark kapısı config yüklemesinde reddedilir. + #### `evaluation.safety` (İsteğe bağlı) | Alan | Tip | Varsayılan | Açıklama | @@ -168,6 +173,7 @@ training: | `track_categories` | bool | `false` | Llama Guard S1-S14 zarar kategorilerini ayrıştır | | `severity_thresholds` | dict | `null` | Ciddiyet bazlı sınırlar: `{"critical": 0, "high": 0.01}` | | `batch_size` | int | `8` | Güvenlik değerlendirmesi için batched generation boyutu. `1` batching'i devre dışı bırakır; geniş VRAM'de throughput için artırın, küçük VRAM'de OOM riskini azaltmak için düşürün. | +| `include_eval_samples` | bool | `false` | Ham `prompt` / `response` dizgelerini `safety_results.json`'a yazar. GDPR / EU AI Act Madde 10 gizliliği için **varsayılan olarak kapalı** — adversarial prompt'lar ve yanıtlar hassas içerik açığa çıkarabilir. Yalnızca hata ayıklama için açın. | #### `evaluation.llm_judge` (İsteğe bağlı) @@ -180,11 +186,21 @@ training: | `eval_dataset` | string | `"eval_prompts.jsonl"` | Değerlendirme prompt dosyası | | `min_score` | float | `5.0` | Minimum ortalama puan (1-10) | | `batch_size` | int | `8` | LLM-hakim turunda puanlanan (prompt, completion) çift sayısı. `1` batching'i devre dışı bırakır. | - +| `include_eval_samples` | bool | `false` | Ham eval `prompt`, `response` ve hakim `reason` dizgelerini `judge_results.json`'a yazar. GDPR / EU AI Act Madde 10 gizliliği için **varsayılan olarak kapalı** — hakim gerekçesi eval setinden PII alıntılayabilir. Yalnızca hata ayıklama için açın. | + +> **Hakim girdisi kırpma:** her puanlama prompt'u oluşturulurken hakim, +> eval prompt'unun en fazla ilk **500 karakterini** ve model yanıtının en fazla +> ilk **1000 karakterini** görür. Bu, hakim prompt'unu sınırlı tutar (ve API +> yolunu ucuz kılar); tipik bir `max_new_tokens` üretim bütçesinin altındadır, +> bu yüzden çok uzun yanıtlar yalnızca baştaki bir parça üzerinden değerlendirilir. +> Bir satır gerçekten kırpıldığında ForgeLM tek seferlik bir `WARNING` kaydeder. +> Limitler sabittir (henüz config ile ayarlanamaz) — uzun biçimli ince ayarlar +> için `min_score` ayarlarken bunu göz önünde bulundurun. +> > **Kullanımdan kaldırıldı:** `evaluation.staging_ttl_days`, -> [`retention.staging_ttl_days`](#retention-isteğe-bağlı-gdpr-madde-17-silme-ufukları) -> tarafından devralınmıştır. Eski anahtar v0.5.5 → v0.6.x penceresi boyunca -> `DeprecationWarning` ile alias-forward edilir ve v0.7.0'da kaldırılır. +> [`retention.staging_ttl_days`](#retention-isteğe-bağlı--gdpr-madde-17-silme-ufukları) +> tarafından devralınmıştır. Eski anahtar `DeprecationWarning` ile alias-forward +> edilir ve v0.8.0'da kaldırılır. > Bkz. [release.md](../standards/release.md#deprecation-cadence). --- @@ -201,15 +217,15 @@ uzatmasını engeller. | Alan | Tip | Varsayılan | Açıklama | |------|-----|-----------|----------| | `audit_log_retention_days` | int | `1825` (~5 yıl) | `audit_log.jsonl` dosyasının Madde 5(1)(e) kapsamında "geciken" olarak işaretlenmeden önce saklanacağı gün sayısı. `0` süresiz saklamayı belirtir (Madde 17(3)(b) savunması). | -| `staging_ttl_days` | int | `7` | `forgelm reject` kararından sonra `final_model.staging./` dizininin planlı temizlenmeden önce saklanacağı gün sayısı. `0` süresiz saklama anlamına gelir. Kullanımdan kaldırılan `evaluation.staging_ttl_days` yerine geçer; v0.5.5 → v0.6.x deprecation penceresinde her iki anahtar da aynı değerlerle kabul edilir. | +| `staging_ttl_days` | int | `7` | `forgelm reject` kararından sonra `final_model.staging./` dizininin planlı temizlenmeden önce saklanacağı gün sayısı. `0` süresiz saklama anlamına gelir. Kullanımdan kaldırılan `evaluation.staging_ttl_days` yerine geçer; deprecation penceresinde her iki anahtar da aynı değerlerle kabul edilir (eski anahtar v0.8.0'da kaldırılır). | | `ephemeral_artefact_retention_days` | int | `90` | Uyumluluk paketleri, veri denetim raporları ve diğer çalışma kapsamlı türetilmiş artefaktların saklanma süresi (gün). `0` süresiz saklama. | | `raw_documents_retention_days` | int | `90` | İngest edilmiş ham belgelerin (PDF / DOCX / EPUB / TXT / Markdown) operatörün ingestion-output dizininde saklanma süresi (gün). `0` süresiz saklama. | | `enforce` | string | `"log_only"` | Politika uygulama modu: `"log_only"` (yalnızca audit log), `"warn_on_excess"` (stderr'e yapılandırılmış uyarı), `"block_on_excess"` (`EXIT_EVAL_FAILURE` = 3 ile trainer ön-kontrolünü iptal eder). | > **Kullanımdan kaldırma:** `evaluation.staging_ttl_days`, v0.5.5 itibarıyla > `retention.staging_ttl_days` lehine kullanımdan kaldırılmıştır. Eski anahtar -> v0.7.0'a kadar `DeprecationWarning` ile alias-forward edilir. Tam -> deprecation politikası için +> v0.8.0'daki kaldırılışına kadar `DeprecationWarning` ile alias-forward edilir. +> Tam deprecation politikası için > [release.md](../standards/release.md#deprecation-cadence). --- @@ -269,7 +285,9 @@ uzatmasını engeller. | `max_new_tokens` | int | `1024` | Öğretmen yanıtı başına maksimum token. | | `temperature` | float | `0.7` | Öğretmene geçirilen örnekleme sıcaklığı. | | `output_file` | string | `"synthetic_data.jsonl"` | Çıktı JSONL dosya yolu. | -| `output_format` | string | `"messages"` | Şunlardan biri: `"messages"` (chat-style array), `"instruction"` (Alpaca-style), `"chatml"`, `"prompt_response"`. | +| `output_format` | string | `"messages"` | Şunlardan biri: `"messages"` (chat-style array), `"instruction"` (Alpaca-style), `"chatml"`, `"prompt_response"`. **`chatml`, ForgeLM'in eski `{User, Assistant}` anahtar düzenini üretir — OpenAI `<\|im_start\|>` ChatML işaretlemesini DEĞİL.** Taşınabilir bir sohbet formatı için `messages` kullanın. | +| `min_success_rate` | float | `0.0` | `forgelm --generate-data`'nin 0 çıkış kodu vermesi için seed prompt'ların başarılı olması gereken minimum oran (0.0–1.0). Varsayılan `0.0`, eski "sıfırdan farklı herhangi bir verim başarılıdır" davranışını korur; bir CI hattının neredeyse boş bir veri kümesiyle devam etmemesi için yükseltin. | +| `sanity_failure_rate` | float | `0.2` | `forgelm --generate-data`'nin, veri kümesinin küçük veya çarpık olabileceğine dair bir `WARNING` kaydettiği başarısızlık oranı eşiği (0.0–1.0) — çıkış kodunu belirleyen `min_success_rate`'ten bağımsızdır. Varsayılan `0.2`, prompt'ların %20'sinden fazlası başarısız olduğunda uyarır. | --- @@ -284,6 +302,7 @@ uzatmasını engeller. | `notify_on_failure` | bool | `true` | Hata durumunda bildir | | `timeout` | int | `10` | HTTP istek zaman aşımı (saniye). Notifier ≥ 1s'ye clamp'ler. v0.5.5'te varsayılan 10s'ye çıkarıldı (önceden 5s'di) — Slack/Teams gateway gecikme atışları production'da düzenli olarak 5s'yi aşıyor ve bir webhook zaman aşımı audit chain'i sessizce zayıflatıyor (webhook arızası best-effort). | | `allow_private_destinations` | bool | `false` | RFC1918 / loopback / link-local hedeflere webhook gönderimine izin verir (cluster içi Slack proxy, on-prem Teams gateway gibi). Varsayılan yalnızca genel internet — SSRF koruması | +| `require_https` | bool | `false` | TLS-only zorlama. `true`, plaintext bir `http://` URL'ini reddeder (SSRF chokepoint raise eder; POST atlanır), warn-and-send yerine. Varsayılan `false`, warn-then-send davranışını korur | | `tls_ca_bundle` | string | `null` | `requests`'e `verify=` olarak iletilen özel CA bundle yolu (örn. kurumsal MITM CA). Boşsa `certifi` paketinin gömülü deposu kullanılır | ## `merge` (İsteğe bağlı) @@ -294,6 +313,20 @@ uzatmasını engeller. | `method` | string | `"ties"` | `"ties"`, `"dare"`, `"slerp"`, `"linear"` | | `models` | list | `[]` | `{path, weight}` sözlük listesi | | `output_dir` | string | `"./merged_model"` | Çıktı dizini | +| `ties_trim_fraction` | float | `0.2` | TIES: görev başına kırpılan en küçük büyüklükteki delta'ların oranı (0.0–1.0). Yalnızca `method` `ties` olduğunda kullanılır. | +| `dare_drop_rate` | float | `0.3` | DARE: yeniden ölçeklemeden önce her delta'nın rastgele düşürülme olasılığı (0.0–1.0). Yalnızca `method` `dare` olduğunda kullanılır. | +| `dare_seed` | int | `42` | DARE: rastgele düşürme maskesi için RNG seed'i; bir birleştirme çalıştırmadan çalıştırmaya tekrarlanabilir olur. | + +> **TIES/DARE varsayılan hiperparametreleri kasıtlı olarak korumacıdır.** +> ForgeLM'in yerel `ties` birleştirmesi, ağırlıkların büyüklüğe göre alttaki +> **%20**'sini kırpar (üstteki %80'i tutar); `dare` birleştirmesi sabit bir +> seed ile `drop_rate=0.3` kullanır. Bu varsayılanlar, yayımlanmış TIES (üstteki +> ~%20'yi tut) ve DARE (`drop_rate` 0.9+) varsayılanlarından kasıtlı olarak daha +> korumacıdır — daha fazla sinyal tutarlar, böylece iki-adaptörlü bir birleştirme +> kutudan çıktığı haliyle daha az yıkıcıdır, ancak sonuç makaleye sadık bir +> birleştirmeden farklı olacaktır. Yayımlanmış seyreklik rejimlerine ihtiyaç +> duyan operatörler `ties_trim_fraction` / `dare_drop_rate` değerlerini +> yükseltebilir (veya mergekit gibi harici bir araçla birleştirebilir). ## `auth` (İsteğe bağlı) @@ -310,7 +343,7 @@ uzatmasını engeller. | Alan | Tip | Varsayılan | Açıklama | |------|-----|-----------|----------| | `output_dir` | string | `"./pipeline_run"` | Zincir seviyesi artefakların kök dizini: `pipeline_state.json`, `compliance/pipeline_manifest.json` ve pipeline-kapsamlı `audit_log.jsonl`. Aşama bazında trainer artefaktları her aşamanın kendi `training.output_dir`'ı altında kalır. | -| `stages` | `List[PipelineStage]` | `[]` (en az 1 zorunlu) | Sıralı aşama listesi. Her aşamanın `model.name_or_path`'ı, aşama explicit `model:` bloğu vermediği sürece, önceki aşamanın `training.output_dir/final_model`'ına otomatik ayarlanır. | +| `stages` | `List[PipelineStage]` | *zorunlu* (en az 1 aşama) | Sıralı aşama listesi. Her aşamanın `model.name_or_path`'ı, aşama explicit `model:` bloğu vermediği sürece, önceki aşamanın `training.output_dir/final_model`'ına otomatik ayarlanır. | ### `pipeline.stages[].*` — PipelineStage alanları diff --git a/docs/reference/configuration.md b/docs/reference/configuration.md index 19a1bb07..23e3b9f5 100644 --- a/docs/reference/configuration.md +++ b/docs/reference/configuration.md @@ -61,7 +61,8 @@ See `config_template.yaml` for a complete annotated example. | `final_model_dir` | string | `"final_model"` | Subdirectory for final artifacts | | `merge_adapters` | bool | `false` | Merge adapters into base model before saving | | `trainer_type` | string | `"sft"` | `"sft"`, `"dpo"`, `"simpo"`, `"kto"`, `"orpo"`, `"grpo"` | -| `num_train_epochs` | int | `3` | Number of training epochs | +| `max_steps` | int | `-1` | Hard step cap. `-1` = use `num_train_epochs`; a positive value overrides epochs. | +| `num_train_epochs` | int | `3` | Number of training epochs (only consulted when `max_steps == -1`). | | `per_device_train_batch_size` | int | `4` | Batch size per GPU | | `gradient_accumulation_steps` | int | `2` | Steps to accumulate before backward pass | | `learning_rate` | float | `2e-5` | Learning rate (lower for alignment: 5e-6) | @@ -70,6 +71,7 @@ See `config_template.yaml` for a complete annotated example. | `eval_steps` | int | `200` | Evaluate every N steps | | `save_steps` | int | `200` | Save checkpoint every N steps | | `save_total_limit` | int | `3` | Max checkpoints to keep | +| `early_stopping_patience` | int | `3` | Stop after N evals without validation-loss improvement (active only when a validation split exists). | | `packing` | bool | `false` | Sequence packing (SFT only) | | `report_to` | string | `"tensorboard"` | `"tensorboard"`, `"wandb"`, `"mlflow"`, `"none"` | | `run_name` | string | `null` | W&B/MLflow run name (auto-generated if null) | @@ -117,7 +119,7 @@ across retries. Each retry attempt is logged to the audit trail. | `rope_scaling` | `Optional[Dict[str, Any]]` | `null` | RoPE scaling method dict (`{"type": "linear", "factor": 2.0}` etc.). Supported types: `"linear"`, `"dynamic"`, `"yarn"`, `"longrope"`. | | `neftune_noise_alpha` | float | `null` | NEFTune noise injection alpha (e.g., `5.0`) | | `sliding_window_attention` | int | `null` | Sliding window attention size in tokens | -| `sample_packing` | bool | `false` | Pack multiple short samples into full-length sequences | +| `sample_packing` | bool | `false` | **Deprecated** alias for `packing` (TRL exposes a single packing knob). Setting `true` forwards to `packing: true` with a `DeprecationWarning`; removed in v0.9.0. Use `packing` instead. | #### GPU Cost Estimation @@ -181,8 +183,11 @@ across retries. Each retry attempt is logged to the audit trail. | `num_fewshot` | int | `null` | Few-shot examples (task default) | | `batch_size` | string | `"auto"` | Evaluation batch size | | `limit` | int | `null` | Samples per task (for quick checks) | +| `output_dir` | string | `null` | Where to write the benchmark results JSON. `null` = the training `output_dir`. | | `min_score` | float | `null` | Minimum average accuracy | +> `enabled: true` requires at least one entry in `tasks` — an enabled benchmark gate with no tasks is rejected at config-load time. + #### `evaluation.safety` (Optional) | Field | Type | Default | Description | @@ -197,6 +202,7 @@ across retries. Each retry attempt is logged to the audit trail. | `track_categories` | bool | `false` | Parse Llama Guard S1-S14 harm categories | | `severity_thresholds` | dict | `null` | Per-severity limits: `{"critical": 0, "high": 0.01, "medium": 0.05}` | | `batch_size` | int | `8` | Batched generation size for safety evaluation. `1` disables batching; raise for throughput on large VRAM, lower to reduce OOM risk on small VRAM. | +| `include_eval_samples` | bool | `false` | Persist raw `prompt` / `response` strings to `safety_results.json`. **Off by default** for GDPR / EU AI Act Art. 10 privacy — adversarial prompts and responses may surface sensitive content. Opt in only for debugging. | #### `evaluation.llm_judge` (Optional) @@ -209,12 +215,21 @@ across retries. Each retry attempt is logged to the audit trail. | `eval_dataset` | string | `"eval_prompts.jsonl"` | Evaluation prompts file | | `min_score` | float | `5.0` | Minimum average score (1-10) | | `batch_size` | int | `8` | Number of (prompt, completion) pairs scored per LLM-judge round. `1` disables batching. | - +| `include_eval_samples` | bool | `false` | Persist raw eval `prompt`, `response`, and judge `reason` strings to `judge_results.json`. **Off by default** for GDPR / EU AI Act Art. 10 privacy — judge reasoning can quote PII from the eval set. Opt in only for debugging. | + +> **Judge input truncation:** when building each scoring prompt the judge +> sees at most the first **500 characters of the eval prompt** and the first +> **1000 characters of the model response**. This keeps the judge prompt +> bounded (and the API path cheap); it is below a typical `max_new_tokens` +> generation budget, so very long answers are judged on a leading fragment. +> ForgeLM logs a one-time `WARNING` when a row is actually trimmed. The limits +> are fixed (not yet config-driven) — keep this in mind when tuning `min_score` +> for long-form fine-tunes. +> > **Deprecated:** `evaluation.staging_ttl_days` is superseded by -> [`retention.staging_ttl_days`](#retention-optional-gdpr-article-17-erasure-horizons). -> The legacy key is alias-forwarded with a `DeprecationWarning` during the -> v0.5.5 → v0.6.x window and removed in v0.7.0. See -> [release.md](../standards/release.md#deprecation-cadence). +> [`retention.staging_ttl_days`](#retention-optional--gdpr-article-17-erasure-horizons). +> The legacy key is alias-forwarded with a `DeprecationWarning` and removed in +> v0.8.0. See [release.md](../standards/release.md#deprecation-cadence). --- @@ -229,14 +244,14 @@ silently extend the retention horizon by re-using a stale workspace. | Field | Type | Default | Description | |-------|------|---------|-------------| | `audit_log_retention_days` | int | `1825` (~5 years) | Days to retain `audit_log.jsonl` before flagging it as overdue under Article 5(1)(e). Set to `0` to retain indefinitely (Article 17(3)(b) defence). | -| `staging_ttl_days` | int | `7` | Days to retain `final_model.staging./` after a `forgelm reject` decision before scheduled cleanup. Set to `0` to retain indefinitely. Replaces the deprecated `evaluation.staging_ttl_days`; both keys accepted with identical values during the v0.5.5 → v0.6.x deprecation window. | +| `staging_ttl_days` | int | `7` | Days to retain `final_model.staging./` after a `forgelm reject` decision before scheduled cleanup. Set to `0` to retain indefinitely. Replaces the deprecated `evaluation.staging_ttl_days`; both keys accepted with identical values during the deprecation window (legacy key removed in v0.8.0). | | `ephemeral_artefact_retention_days` | int | `90` | Days to retain compliance bundles, data audit reports, and other run-scoped derived artefacts. Set to `0` to retain indefinitely. | | `raw_documents_retention_days` | int | `90` | Days to retain ingested raw documents (PDF / DOCX / EPUB / TXT / Markdown) under the operator's ingestion-output directory. Set to `0` to retain indefinitely. | | `enforce` | string | `"log_only"` | Policy enforcement mode: `"log_only"` (audit-log only), `"warn_on_excess"` (structured stderr warning), `"block_on_excess"` (abort trainer pre-flight with `EXIT_EVAL_FAILURE` = 3). | > **Deprecation:** `evaluation.staging_ttl_days` is deprecated as of v0.5.5 in > favour of `retention.staging_ttl_days`. The legacy key is alias-forwarded -> with a `DeprecationWarning` until v0.7.0. See +> with a `DeprecationWarning` until its removal in v0.8.0. See > [release.md](../standards/release.md#deprecation-cadence) for the full > deprecation cadence policy. @@ -253,6 +268,7 @@ silently extend the retention horizon by re-using a stale workspace. | `notify_on_failure` | bool | `true` | Notify on failure | | `timeout` | int | `10` | HTTP request timeout (seconds). Clamped to ≥ 1s by the notifier. Default raised to 10s in v0.5.5 (was 5s) — Slack/Teams gateway latency spikes regularly cross 5s in production, and a webhook timeout silently degrades the audit chain (webhook failure is best-effort). | | `allow_private_destinations` | bool | `false` | Opt in to webhooks pointing at RFC1918 / loopback / link-local hosts (in-cluster Slack proxy, on-prem Teams gateway). Defaults to public-internet only — SSRF guard | +| `require_https` | bool | `false` | TLS-only enforcement. `true` refuses a plaintext `http://` URL (the SSRF chokepoint raises; the POST is skipped) instead of warn-and-send. Default `false` preserves warn-then-send | | `tls_ca_bundle` | string | `null` | Path to a custom CA bundle forwarded to `requests` as `verify=` (e.g. corporate MITM CA). When unset, `certifi`'s bundled store is used | --- @@ -279,6 +295,19 @@ silently extend the retention horizon by re-using a stale workspace. | `method` | string | `"ties"` | `"ties"`, `"dare"`, `"slerp"`, `"linear"` | | `models` | list | `[]` | List of `{path, weight}` dicts | | `output_dir` | string | `"./merged_model"` | Output directory | +| `ties_trim_fraction` | float | `0.2` | TIES: fraction (0.0–1.0) of smallest-magnitude deltas trimmed per task. Only consulted when `method` is `ties`. | +| `dare_drop_rate` | float | `0.3` | DARE: probability (0.0–1.0) each delta is randomly dropped before rescaling. Only consulted when `method` is `dare`. | +| `dare_seed` | int | `42` | DARE: RNG seed for the random drop mask, so a merge is reproducible run-to-run. | + +> **TIES/DARE default hyperparameters are intentionally conservative.** ForgeLM's +> native `ties` merge trims the bottom **20%** of weights by magnitude (keeps +> the top 80%); the `dare` merge uses `drop_rate=0.3` with a fixed seed. These +> defaults are intentionally more conservative than the published TIES (keep +> top ~20%) and DARE (`drop_rate` 0.9+) defaults — they retain more signal so a +> two-adapter merge is less destructive out of the box, but the result will +> differ from a paper-faithful merge. Operators who need the published sparsity +> regimes can raise `ties_trim_fraction` / `dare_drop_rate` (or merge with an +> external tool such as mergekit). --- @@ -339,7 +368,9 @@ silently extend the retention horizon by re-using a stale workspace. | `max_new_tokens` | int | `1024` | Max tokens per teacher response. | | `temperature` | float | `0.7` | Sampling temperature passed to the teacher. | | `output_file` | string | `"synthetic_data.jsonl"` | Output JSONL file path. | -| `output_format` | string | `"messages"` | One of `"messages"` (chat-style array), `"instruction"` (Alpaca-style), `"chatml"`, `"prompt_response"`. | +| `output_format` | string | `"messages"` | One of `"messages"` (chat-style array), `"instruction"` (Alpaca-style), `"chatml"`, `"prompt_response"`. **`chatml` emits ForgeLM's legacy `{User, Assistant}` key layout — NOT OpenAI `<\|im_start\|>` ChatML markup.** Use `messages` for a portable chat format. | +| `min_success_rate` | float | `0.0` | Minimum fraction (0.0–1.0) of seed prompts that must yield a usable example for `forgelm --generate-data` to exit 0. Default `0.0` keeps the legacy "any non-zero yield succeeds" behaviour; raise it so a CI pipeline does not proceed on a near-empty dataset. | +| `sanity_failure_rate` | float | `0.2` | Failure-rate (0.0–1.0) above which `forgelm --generate-data` logs a `WARNING` that the dataset may be small or skewed — independent of `min_success_rate` (which gates the exit code). Default `0.2` warns when more than 20% of prompts fail. | --- @@ -358,7 +389,7 @@ Chains 2+ training stages (typically SFT → DPO → GRPO) into one config-drive | Field | Type | Default | Description | |-------|------|---------|-------------| | `output_dir` | string | `"./pipeline_run"` | Root directory for chain-level artefacts: `pipeline_state.json`, `compliance/pipeline_manifest.json`, and the pipeline-scoped `audit_log.jsonl`. Per-stage trainer artefacts continue to live under each stage's own `training.output_dir`. | -| `stages` | `List[PipelineStage]` | `[]` (required: ≥ 1) | Ordered list of stages. Each stage's `model.name_or_path` is auto-set to the previous stage's `training.output_dir/final_model` unless the stage supplies an explicit `model:` block. | +| `stages` | `List[PipelineStage]` | *required* (≥ 1 stage) | Ordered list of stages. Each stage's `model.name_or_path` is auto-set to the previous stage's `training.output_dir/final_model` unless the stage supplies an explicit `model:` block. | ### `pipeline.stages[].*` — PipelineStage fields diff --git a/docs/reference/doctor_subcommand-tr.md b/docs/reference/doctor_subcommand-tr.md index 67b19f71..eddc9929 100644 --- a/docs/reference/doctor_subcommand-tr.md +++ b/docs/reference/doctor_subcommand-tr.md @@ -99,7 +99,7 @@ Summary: 6 pass, 2 warn, 0 fail. $ HF_HUB_OFFLINE=1 forgelm doctor --offline ``` -`hf_hub.reachable` yerine `hf_hub.offline_cache` görünür. Dolu cache; boyut, dosya sayısı ve `HF_HUB_OFFLINE` değerini raporlar; boş cache, [`cache_subcommands.md`](cache_subcommands.md)'a yönlendiren `warn` üretir. +`hf_hub.reachable` yerine `hf_hub.offline_cache` görünür. Dolu cache; boyut, dosya sayısı ve `HF_HUB_OFFLINE` değerini raporlar; boş cache, [`cache_subcommands.md`](cache_subcommands-tr.md)'a yönlendiren `warn` üretir. ### CI gate (JSON) diff --git a/docs/reference/library_api_reference-tr.md b/docs/reference/library_api_reference-tr.md index 07d1a069..73504721 100644 --- a/docs/reference/library_api_reference-tr.md +++ b/docs/reference/library_api_reference-tr.md @@ -79,7 +79,7 @@ Best-effort. Şekil, major artış olmadan minor sürümde değişebilir. Çağr | Sembol | Katman | İmza | Açıklama | |---|---|---|---| -| `forgelm.AuditLogger` | Stable | `AuditLogger(output_dir: str, run_id: str \| None = None)` | Append-only Article 12 audit logger. POSIX `fcntl.flock` kullanır; Windows `msvcrt.locking` kullanır. Her fork edilmiş alt süreç kendi instance'ını kurmalıdır. | +| `forgelm.AuditLogger` | Stable | `AuditLogger(output_dir: str, run_id: str \| None = None)` | Append-only Article 12 audit logger. POSIX `fcntl.flock` kullanır; Windows'ta süreçler-arası kilit **yoktur** (advisory flock yardımcısı no-op'tur) — Windows'ta eşzamanlı süreçler arasında bir `output_dir`'i paylaşmayın. Her fork edilmiş alt süreç kendi instance'ını kurmalıdır. | | `forgelm.AuditLogger.log_event` | Stable | `log_event(event: str, **fields) -> None` | Yapılandırılmış bir olay ekle. Olay kelime dağarcığı [`audit_event_catalog-tr.md`](audit_event_catalog-tr.md)'da belgelenir. | | `forgelm.verify_audit_log` | Stable | `verify_audit_log(path: str, *, hmac_secret: str \| None = None, require_hmac: bool = False) -> VerifyResult` | SHA-256 hash zincirini yürür. Zincir hatalarında `VerifyResult(valid=False, reason=...)` döndürür (exception değil); yalnızca okunamaz dosyalar için `OSError` fırlatır. | | `forgelm.VerifyResult` | Stable | `dataclass` | Kanonik alanlar (`forgelm/compliance.py:VerifyResult`): `valid: bool`, `entries_count: int`, `first_invalid_index: Optional[int]`, `reason: Optional[str]`. | @@ -92,6 +92,8 @@ Best-effort. Şekil, major artış olmadan minor sürümde değişebilir. Çağr | `forgelm.VerifyAnnexIVResult` | Stable | `dataclass-benzeri` | Kanonik attribute'lar: `valid: bool`, `reason: Optional[str]`, `missing_fields: List[str]`, `manifest_hash_actual: Optional[str]`, `manifest_hash_expected: Optional[str]`. (Not: `verify_annex_iv_artifact(path)` JSON manifest path'i kabul eder, ZIP değil — fonksiyon `path` üzerinde `json.load` çağırır.) | | `forgelm.verify_gguf` | Stable | `verify_gguf(path: str) -> VerifyGgufResult` | Bir GGUF dışa aktarmasını (header + tensor catalogue + tokenizer block) doğrular. | | `forgelm.VerifyGgufResult` | Stable | `dataclass-benzeri` | Kanonik attribute'lar: `valid: bool`, `reason: Optional[str]`, `checks: Dict[str, Any]` (her şey diğer — header / tensor catalogue / tokenizer block — `checks` içinde yaşar). | +| `forgelm.verify_integrity` | Stable | `verify_integrity(model_dir: str) -> VerifyIntegrityResult` | Bir model dizinini `model_integrity.json` SHA-256 manifestine karşı doğrular (Madde 15): değiştirilen / silinen / eklenen yapıtları raporlar. | +| `forgelm.VerifyIntegrityResult` | Stable | `dataclass-benzeri` | Kanonik attribute'lar: `valid: bool`, `reason: Optional[str]`, `changed: List[str]`, `removed: List[str]`, `added: List[str]`, `verified_count: int`. | ### Benchmark + sentetik veri @@ -246,7 +248,7 @@ Bu invariant, hafif CI runner'larının, `forgelm doctor`'un ve `python -m forge |---|---|---| | `ForgeTrainer.train()` | Hayır — TRL GPU state tutar | Hayır | | `audit_dataset()` | Evet — her çağrı self-contained | Evet | -| `AuditLogger.log_event()` | Evet — POSIX'te `flock`, Windows'ta `msvcrt.locking` | Her child için yeni logger kurun; handle'ları fork'lar arasında paylaşmak desteklenmez | +| `AuditLogger.log_event()` | POSIX'te evet — `flock(LOCK_EX)`; Windows'ta süreçler-arası kilit yoktur (no-op), bu yüzden bir `output_dir`'i süreçler arasında paylaşmayın | Her child için yeni logger kurun; handle'ları fork'lar arasında paylaşmak desteklenmez | | `verify_audit_log()` | Evet — read-only | Evet | | `WebhookNotifier.notify_*()` | Evet — her çağrı kendi `requests` session'ını açar | Evet | diff --git a/docs/reference/library_api_reference.md b/docs/reference/library_api_reference.md index 603804dd..a3ee9470 100644 --- a/docs/reference/library_api_reference.md +++ b/docs/reference/library_api_reference.md @@ -79,7 +79,7 @@ Tables grouped by concern. Every cell is a real attribute on the live `forgelm` | Symbol | Tier | Signature | Description | |---|---|---|---| -| `forgelm.AuditLogger` | Stable | `AuditLogger(output_dir: str, run_id: str \| None = None)` | Append-only Article 12 audit logger. POSIX uses `fcntl.flock`; Windows uses `msvcrt.locking`. Each forked child must construct its own instance. | +| `forgelm.AuditLogger` | Stable | `AuditLogger(output_dir: str, run_id: str \| None = None)` | Append-only Article 12 audit logger. POSIX uses `fcntl.flock`; on Windows there is **no** cross-process lock (the advisory flock helper is a no-op) — do not share an `output_dir` across concurrent processes on Windows. Each forked child must construct its own instance. | | `forgelm.AuditLogger.log_event` | Stable | `log_event(event: str, **fields) -> None` | Append a structured event. The event vocabulary is documented in [`audit_event_catalog.md`](audit_event_catalog.md). | | `forgelm.verify_audit_log` | Stable | `verify_audit_log(path: str, *, hmac_secret: str \| None = None, require_hmac: bool = False) -> VerifyResult` | Walk the SHA-256 hash chain. Returns `VerifyResult(valid=False, reason=...)` for chain failures (not an exception); raises `OSError` only for unreadable files. | | `forgelm.VerifyResult` | Stable | `dataclass` | Canonical fields (per `forgelm/compliance.py:VerifyResult`): `valid: bool`, `entries_count: int`, `first_invalid_index: Optional[int]`, `reason: Optional[str]`. | @@ -92,6 +92,8 @@ Tables grouped by concern. Every cell is a real attribute on the live `forgelm` | `forgelm.VerifyAnnexIVResult` | Stable | `dataclass-like` | Canonical attributes: `valid: bool`, `reason: Optional[str]`, `missing_fields: List[str]`, `manifest_hash_actual: Optional[str]`, `manifest_hash_expected: Optional[str]`. (Note: `verify_annex_iv_artifact(path)` accepts a JSON manifest path, not a ZIP — the function calls `json.load` on `path`.) | | `forgelm.verify_gguf` | Stable | `verify_gguf(path: str) -> VerifyGgufResult` | Validate a GGUF export (header + tensor catalogue + tokenizer block). | | `forgelm.VerifyGgufResult` | Stable | `dataclass-like` | Canonical attributes: `valid: bool`, `reason: Optional[str]`, `checks: Dict[str, Any]` (everything else — header / tensor catalogue / tokenizer block — lives inside `checks`). | +| `forgelm.verify_integrity` | Stable | `verify_integrity(model_dir: str) -> VerifyIntegrityResult` | Verify a model directory against its `model_integrity.json` SHA-256 manifest (Article 15): reports changed / removed / added artifacts. | +| `forgelm.VerifyIntegrityResult` | Stable | `dataclass-like` | Canonical attributes: `valid: bool`, `reason: Optional[str]`, `changed: List[str]`, `removed: List[str]`, `added: List[str]`, `verified_count: int`. | ### Benchmark + synthetic data @@ -246,7 +248,7 @@ This invariant exists because lightweight CI runners, `forgelm doctor`, and `pyt |---|---|---| | `ForgeTrainer.train()` | No — TRL holds GPU state | No | | `audit_dataset()` | Yes — each call is self-contained | Yes | -| `AuditLogger.log_event()` | Yes — `flock` on POSIX, `msvcrt.locking` on Windows | Construct a fresh logger per child; sharing handles across forks is unsupported | +| `AuditLogger.log_event()` | Yes on POSIX — `flock(LOCK_EX)`; on Windows there is no cross-process lock (no-op), so do not share an `output_dir` across processes | Construct a fresh logger per child; sharing handles across forks is unsupported | | `verify_audit_log()` | Yes — read-only | Yes | | `WebhookNotifier.notify_*()` | Yes — each call opens its own `requests` session | Yes | diff --git a/docs/reference/purge_subcommand-tr.md b/docs/reference/purge_subcommand-tr.md index 13db9432..de03e4d2 100644 --- a/docs/reference/purge_subcommand-tr.md +++ b/docs/reference/purge_subcommand-tr.md @@ -5,7 +5,7 @@ `forgelm purge`, ForgeLM eğitim corpus'ları ve koşum artefact'ları için **GDPR Madde 17 silinme hakkı**'nın operatör-yüzlü uygulamasıdır (Phase 21). Bir satırı, bir koşumun staging dizinini veya bir koşumun compliance bundle'ını atomik olarak siler ve her adımı tamper-evident `audit_log.jsonl` zincirine kaydeder. -Deployer akışı (DSAR ticket → CLI çağrısı → doğrulama) için bkz. [`../guides/gdpr_erasure.md`](../guides/gdpr_erasure.md). Bu sayfa flag-başına, event-başına referanstır. +Deployer akışı (DSAR ticket → CLI çağrısı → doğrulama) için bkz. [`../guides/gdpr_erasure.md`](../guides/gdpr_erasure-tr.md). Bu sayfa flag-başına, event-başına referanstır. ## Synopsis @@ -73,7 +73,7 @@ Per-output-dir salt `/.forgelm_audit_salt`'ta ilk kullanımda oluşt ## Yayılan audit event'leri -Altı event de [`audit_event_catalog.md`](audit_event_catalog.md)'deki ortak zarfı taşır. Katalog satırları kolaylık için burada da listelenir. +Altı event de [`audit_event_catalog.md`](audit_event_catalog-tr.md)'deki ortak zarfı taşır. Katalog satırları kolaylık için burada da listelenir. | Event | Ne zaman yayılır | Anahtar payload | |---|---|---| @@ -165,7 +165,7 @@ Asimetri notu: row/run başarı zarfları açık `success` field'ı **taşımaz* ## Bkz. -- [`../guides/gdpr_erasure.md`](../guides/gdpr_erasure.md) — deployer akışı (DSAR ticket → CLI → zincir doğrulama). -- [`reverse_pii_subcommand.md`](reverse_pii_subcommand.md) — kardeş Madde 15 erişim hakkı aracı. -- [`audit_event_catalog.md`](audit_event_catalog.md) — zarf spec'iyle birlikte tam event sözlüğü. -- [`../qms/access_control.md`](../qms/access_control.md) §3.4 — operatör kimliği sözleşmesi (her erasure event'ında kayıtlı `FORGELM_OPERATOR`). +- [`../guides/gdpr_erasure.md`](../guides/gdpr_erasure-tr.md) — deployer akışı (DSAR ticket → CLI → zincir doğrulama). +- [`reverse_pii_subcommand.md`](reverse_pii_subcommand-tr.md) — kardeş Madde 15 erişim hakkı aracı. +- [`audit_event_catalog.md`](audit_event_catalog-tr.md) — zarf spec'iyle birlikte tam event sözlüğü. +- [`../qms/access_control.md`](../qms/access_control-tr.md) §3.4 — operatör kimliği sözleşmesi (her erasure event'ında kayıtlı `FORGELM_OPERATOR`). diff --git a/docs/reference/reverse_pii_subcommand-tr.md b/docs/reference/reverse_pii_subcommand-tr.md index da708c5b..55cc2c5e 100644 --- a/docs/reference/reverse_pii_subcommand-tr.md +++ b/docs/reference/reverse_pii_subcommand-tr.md @@ -3,7 +3,7 @@ > **Hedef kitle:** Eğitim corpus'larına karşı GDPR Madde 15 erişim hakkı taleplerini yanıtlayan ForgeLM operatörleri ve ortaya çıkan `data.access_request_query` audit satırını doğrulayan denetçiler. > **Ayna:** [reverse_pii_subcommand.md](reverse_pii_subcommand.md) -`forgelm reverse-pii`, ForgeLM eğitim corpus'ları için **GDPR Madde 15 erişim hakkı**'nın operatör-yüzlü uygulamasıdır (Phase 38). [`forgelm purge`](purge_subcommand.md) "satırımı sil" derken, `reverse-pii` "identifier'ımın geçtiği her satırı bul" der. +`forgelm reverse-pii`, ForgeLM eğitim corpus'ları için **GDPR Madde 15 erişim hakkı**'nın operatör-yüzlü uygulamasıdır (Phase 38). [`forgelm purge`](purge_subcommand-tr.md) "satırımı sil" derken, `reverse-pii` "identifier'ımın geçtiği her satırı bul" der. ## Synopsis @@ -65,7 +65,7 @@ $ forgelm reverse-pii --query "alice@example.com" --type email \ ## Yayılan audit event'leri -Her çağrı tam olarak bir event yayar ([katalog satırı](audit_event_catalog-tr.md#madde-15-gdpr-erişim-hakkı-phase-38-forgelm-reverse-pii)): +Her çağrı tam olarak bir event yayar ([katalog satırı](audit_event_catalog-tr.md#madde-15--gdpr-erişim-hakkı-phase-38--forgelm-reverse-pii)): | Event | Ne zaman yayılır | Anahtar payload | |---|---|---| @@ -123,7 +123,7 @@ Başarısız scan standart hata zarfını yayar (`_output_error_and_exit`): ## Bkz. -- [`../guides/gdpr_erasure.md`](../guides/gdpr_erasure.md) §"Madde 15 erişim hakkı" — deployer akışı. -- [`purge_subcommand.md`](purge_subcommand.md) — kardeş Madde 17 silinme hakkı aracı; cross-tool digest korelasyonu için per-output-dir salt'ı paylaşır. -- [`audit_event_catalog.md`](audit_event_catalog.md) — zarf spec'i ile birlikte tam event sözlüğü. -- [`../qms/access_control.md`](../qms/access_control.md) §3.4 — operatör kimliği sözleşmesi (her erişim talebi event'ında kayıtlı `FORGELM_OPERATOR`). +- [`../guides/gdpr_erasure.md`](../guides/gdpr_erasure-tr.md) §"Madde 15 erişim hakkı" — deployer akışı. +- [`purge_subcommand.md`](purge_subcommand-tr.md) — kardeş Madde 17 silinme hakkı aracı; cross-tool digest korelasyonu için per-output-dir salt'ı paylaşır. +- [`audit_event_catalog.md`](audit_event_catalog-tr.md) — zarf spec'i ile birlikte tam event sözlüğü. +- [`../qms/access_control.md`](../qms/access_control-tr.md) §3.4 — operatör kimliği sözleşmesi (her erişim talebi event'ında kayıtlı `FORGELM_OPERATOR`). diff --git a/docs/reference/reverse_pii_subcommand.md b/docs/reference/reverse_pii_subcommand.md index ca70d7e5..97c9b638 100644 --- a/docs/reference/reverse_pii_subcommand.md +++ b/docs/reference/reverse_pii_subcommand.md @@ -65,7 +65,7 @@ The salt at `/.forgelm_audit_salt` is the **identifier-hash salt onl ## Audit events emitted -Every invocation emits exactly one event ([catalog row](audit_event_catalog.md#article-15-gdpr-right-of-access-phase-38-forgelm-reverse-pii)): +Every invocation emits exactly one event ([catalog row](audit_event_catalog.md#article-15--gdpr-right-of-access-phase-38--forgelm-reverse-pii)): | Event | When emitted | Key payload | |---|---|---| diff --git a/docs/reference/supply_chain_security-tr.md b/docs/reference/supply_chain_security-tr.md index e14c8c45..332b7d20 100644 --- a/docs/reference/supply_chain_security-tr.md +++ b/docs/reference/supply_chain_security-tr.md @@ -92,11 +92,13 @@ Wave 4 / Faz 23 nightly workflow'a `pip-audit` ekler. Davranış: - **MEDIUM / MODERATE** → `::warning::` annotation; nightly yeşil kalır. - **LOW** → sessiz. - - **UNKNOWN** → adet + rapor path'ini listeleyen tek bir özet - `::warning::` annotation'ı; operator-triage SRE'leri workflow - YAML'ını dolaşmadan artefactı grep'leyebilsin diye. Nightly - yeşil kalır (pip-audit'in JSON'u severity taşımadığı için - gerçek raporlarda çoğu bulgu buraya düşer). + - **UNKNOWN** → exit 1 (run'ı fail eder, bulgu başına bir `::error::`). + pip-audit'in JSON'u OSV severity taşımadığı için gerçek bulguların + neredeyse tamamı buraya düşer — fail-closed davranış, eksik bir + severity alanının gate'i sessizce atlamasına izin vermek yerine + operatörü açık triyaja zorlar. Belirli bir CVE'yi kabul etmek için + opt-in ignore dosyasıyla dokümante edin (aşağıdaki Suppression'a + bakın); yeşil kalmak için UNKNOWN kovasına güvenmeyin. - OSV / GHSA veritabanlarını kullanır (pip-audit varsayılanı). Operatörler aynı tooling'i lokalde kurar: diff --git a/docs/reference/supply_chain_security.md b/docs/reference/supply_chain_security.md index 6feccd2e..91057a6a 100644 --- a/docs/reference/supply_chain_security.md +++ b/docs/reference/supply_chain_security.md @@ -92,11 +92,13 @@ Wave 4 / Faz 23 adds `pip-audit` to the nightly workflow. Behaviour: - **MEDIUM / MODERATE** → `::warning::` annotation; nightly stays green. - **LOW** → silent. - - **UNKNOWN** → single summary `::warning::` annotation listing - the count + report path, so operator-triage SREs can grep the - artefact without walking the workflow YAML; nightly stays green - (pip-audit's JSON does not carry severity, so most findings land - here on real reports). + - **UNKNOWN** → exit 1 (fails the run, one `::error::` per finding). + pip-audit's JSON does not carry OSV severity, so almost every real + finding lands here — failing closed forces explicit operator triage + rather than letting a missing-severity field silently skip the gate. + To accept a specific CVE, document it via the opt-in ignore file + (see Suppression below); do not rely on the UNKNOWN bucket to stay + green. - Uses the OSV / GHSA databases (pip-audit's default). Operators install the same tooling locally: diff --git a/docs/reference/usage-tr.md b/docs/reference/usage-tr.md index 885f7b76..e236b3fc 100644 --- a/docs/reference/usage-tr.md +++ b/docs/reference/usage-tr.md @@ -99,7 +99,7 @@ forgelm verify-annex-iv --pipeline ./pipeline_run **Tek-aşama flag reddi:** Config bir `pipeline:` bloğu taşıdığında `--fit-check`, `--merge`, `--generate-data`, `--compliance-export`, `--benchmark-only` desteklenmez — ya bloğu kaldırın ya da flag'i kaldırın. Tersine, `--stage`, `--resume-from`, `--force-resume`, `--input-model` `pipeline:` bloğu gerektirir — tek-aşama config'e karşı çalıştırıldıklarında flag'i sessizce yok saymak yerine `EXIT_CONFIG_ERROR (1)` ile çıkar. -Operatör adım adım: [Çok Aşamalı Pipeline kılavuzu](../guides/pipeline-tr.md). Şema detayları: [`pipeline` config bloğu](configuration-tr.md#pipeline-isteğe-bağlı-çok-aşamalı-eğitim-zincirleri-faz-14). +Operatör adım adım: [Çok Aşamalı Pipeline kılavuzu](../guides/pipeline-tr.md). Şema detayları: [`pipeline` config bloğu](configuration-tr.md#pipeline-isteğe-bağlı--çok-aşamalı-eğitim-zincirleri-faz-14). ### VRAM Fit Check @@ -310,7 +310,7 @@ training: rope_scaling: {type: "linear", factor: 2.0} # dict formu: type ∈ {"linear","dynamic","yarn","longrope"}, factor ≥ 1.0 neftune_noise_alpha: 5.0 # Daha iyi genelleme için NEFTune gürültüsü sliding_window_attention: 4096 # Kayan pencere boyutu (token) - sample_packing: true # Kısa örnekleri tam uzunluklu dizilere paketle + packing: true # Kısa örnekleri tam uzunluklu dizilere paketle ``` ### GPU Maliyet Tahmini diff --git a/docs/reference/usage.md b/docs/reference/usage.md index c4d71388..0a4bbfd3 100644 --- a/docs/reference/usage.md +++ b/docs/reference/usage.md @@ -118,7 +118,7 @@ forgelm verify-annex-iv --pipeline ./pipeline_run **Single-stage flag rejection:** `--fit-check`, `--merge`, `--generate-data`, `--compliance-export`, `--benchmark-only` are not supported when the config carries a `pipeline:` block — drop the block or remove the flag. Conversely, `--stage`, `--resume-from`, `--force-resume`, `--input-model` require a `pipeline:` block — running them against a single-stage config exits with `EXIT_CONFIG_ERROR (1)` rather than silently ignoring the flag. -Full operator walkthrough: [Multi-Stage Pipelines guide](../guides/pipeline.md). Schema details: [`pipeline` config block](configuration.md#pipeline-optional-multi-stage-training-chains-phase-14). +Full operator walkthrough: [Multi-Stage Pipelines guide](../guides/pipeline.md). Schema details: [`pipeline` config block](configuration.md#pipeline-optional--multi-stage-training-chains-phase-14). ### VRAM Fit Check @@ -334,7 +334,7 @@ training: rope_scaling: {type: "linear", factor: 2.0} # dict form: type ∈ {"linear","dynamic","yarn","longrope"}, factor ≥ 1.0 neftune_noise_alpha: 5.0 # NEFTune noise for better generalization sliding_window_attention: 4096 # Sliding window size (tokens) - sample_packing: true # Pack short samples into full-length sequences + packing: true # Pack short samples into full-length sequences ``` ### GPU Cost Estimation diff --git a/docs/reference/verify_annex_iv_subcommand-tr.md b/docs/reference/verify_annex_iv_subcommand-tr.md index 2f998395..f299d2d9 100644 --- a/docs/reference/verify_annex_iv_subcommand-tr.md +++ b/docs/reference/verify_annex_iv_subcommand-tr.md @@ -52,7 +52,7 @@ Doğrulayıcı statik bir katalogu (`_ANNEX_IV_REQUIRED_FIELDS`) yürür; böyle ## Emit edilen audit event'leri -`forgelm verify-annex-iv` **salt-okunur bir doğrulayıcıdır** ve `audit_log.jsonl`'a **hiçbir** kayıt eklemez. Annex IV *üretimini* (doğrulamayı değil) işaretleyen event — `compliance.artifacts_exported` — [audit_event_catalog.md](audit_event_catalog.md)'nin Madde 11 + Annex IV bölümünde kataloglanmıştır. Doğrulama-anı kaydı isteyen operatörler bu alt-komutu CI'dan çağırıp JSON çıktısını artifact paketinin yanında saklayabilir. +`forgelm verify-annex-iv` **salt-okunur bir doğrulayıcıdır** ve `audit_log.jsonl`'a **hiçbir** kayıt eklemez. Annex IV *üretimini* (doğrulamayı değil) işaretleyen event — `compliance.artifacts_exported` — [audit_event_catalog.md](audit_event_catalog-tr.md)'nin Madde 11 + Annex IV bölümünde kataloglanmıştır. Doğrulama-anı kaydı isteyen operatörler bu alt-komutu CI'dan çağırıp JSON çıktısını artifact paketinin yanında saklayabilir. ## Örnekler @@ -113,8 +113,8 @@ $ echo $? ## Bkz. -- [`audit_event_catalog.md`](audit_event_catalog.md) — `compliance.artifacts_exported` (Madde 11 + Annex IV) ve kanonik event sözlüğünün geri kalanı. -- [`verify_audit.md`](verify_audit.md) — `audit_log.jsonl` için kardeş doğrulayıcı. -- [`verify_gguf_subcommand.md`](verify_gguf_subcommand.md) — export edilmiş GGUF artifact'ları için kardeş doğrulayıcı. +- [`audit_event_catalog.md`](audit_event_catalog-tr.md) — `compliance.artifacts_exported` (Madde 11 + Annex IV) ve kanonik event sözlüğünün geri kalanı. +- [`verify_audit.md`](verify_audit-tr.md) — `audit_log.jsonl` için kardeş doğrulayıcı. +- [`verify_gguf_subcommand.md`](verify_gguf_subcommand-tr.md) — export edilmiş GGUF artifact'ları için kardeş doğrulayıcı. - [Annex IV kullanım kılavuzu sayfası](../usermanuals/tr/compliance/annex-iv.md) — tam hızlı başlangıç örneği içeren operatör-odaklı kılavuz. - `forgelm.compliance.build_annex_iv_artifact` ve `forgelm.compliance.compute_annex_iv_manifest_hash` — bu doğrulayıcının yazıcı tarafındaki muadilleri. diff --git a/docs/reference/verify_audit-tr.md b/docs/reference/verify_audit-tr.md index f077702c..09d7c6fc 100644 --- a/docs/reference/verify_audit-tr.md +++ b/docs/reference/verify_audit-tr.md @@ -38,7 +38,7 @@ Kodlar dispatcher tarafından `_run_verify_audit_cmd` (`forgelm/cli/subcommands/ ## Emit edilen audit event'leri -`forgelm verify-audit` **salt-okunur bir doğrulayıcıdır** ve `audit_log.jsonl`'a **hiçbir** kayıt eklemez. Yalnızca zinciri inceler. Doğrulanan log'un *içinde* görünen event'ler [audit_event_catalog.md](audit_event_catalog.md)'de kataloglanmıştır (verify-audit'in yürüdüğü `_hmac`, `prev_hash` ve `run_id` alanları için Ortak zarf satırına bakın). +`forgelm verify-audit` **salt-okunur bir doğrulayıcıdır** ve `audit_log.jsonl`'a **hiçbir** kayıt eklemez. Yalnızca zinciri inceler. Doğrulanan log'un *içinde* görünen event'ler [audit_event_catalog.md](audit_event_catalog-tr.md)'de kataloglanmıştır (verify-audit'in yürüdüğü `_hmac`, `prev_hash` ve `run_id` alanları için Ortak zarf satırına bakın). ## Örnekler @@ -99,8 +99,8 @@ $ echo $? ## Bkz. -- [`audit_event_catalog.md`](audit_event_catalog.md) — bu komutun doğruladığı log'un *içinde* görünen event'ler. -- [`verify_annex_iv_subcommand.md`](verify_annex_iv_subcommand.md) — Annex IV teknik dokümantasyon artifact'ı için kardeş doğrulayıcı. -- [`verify_gguf_subcommand.md`](verify_gguf_subcommand.md) — export edilmiş GGUF model dosyaları için kardeş doğrulayıcı. +- [`audit_event_catalog.md`](audit_event_catalog-tr.md) — bu komutun doğruladığı log'un *içinde* görünen event'ler. +- [`verify_annex_iv_subcommand.md`](verify_annex_iv_subcommand-tr.md) — Annex IV teknik dokümantasyon artifact'ı için kardeş doğrulayıcı. +- [`verify_gguf_subcommand.md`](verify_gguf_subcommand-tr.md) — export edilmiş GGUF model dosyaları için kardeş doğrulayıcı. - [Audit Log kullanım kılavuzu sayfası](../usermanuals/tr/compliance/audit-log.md) — log'un kendisine dair operatör-odaklı kılavuz. - `forgelm.compliance.verify_audit_log` — entegratörlerin CLI'dan geçmeden doğrudan çağırdığı kütüphane giriş noktası. diff --git a/docs/reference/verify_gguf_subcommand-tr.md b/docs/reference/verify_gguf_subcommand-tr.md index e6c371b3..584bfc55 100644 --- a/docs/reference/verify_gguf_subcommand-tr.md +++ b/docs/reference/verify_gguf_subcommand-tr.md @@ -46,7 +46,7 @@ Exporter sidecar'ı varsayılan olarak yazar (bkz. [`docs/usermanuals/tr/deploym ## Emit edilen audit event'leri -`forgelm verify-gguf` **salt-okunur bir doğrulayıcıdır** ve `audit_log.jsonl`'a **hiçbir** kayıt eklemez. GGUF *üretimini* (doğrulamayı değil) işaretleyen event'ler export adımına aittir ve şu anda koşu seviyesindeki `pipeline.completed` zarfı içinde gider; bkz. [audit_event_catalog.md](audit_event_catalog.md). +`forgelm verify-gguf` **salt-okunur bir doğrulayıcıdır** ve `audit_log.jsonl`'a **hiçbir** kayıt eklemez. GGUF *üretimini* (doğrulamayı değil) işaretleyen event'ler export adımına aittir ve şu anda koşu seviyesindeki `pipeline.completed` zarfı içinde gider; bkz. [audit_event_catalog.md](audit_event_catalog-tr.md). ## Örnekler @@ -137,8 +137,8 @@ Meta veri katmanını geri eklemek için isteğe bağlı extra'yı yükleyin: `p ## Bkz. -- [`audit_event_catalog.md`](audit_event_catalog.md) — kanonik event sözlüğü. -- [`verify_audit.md`](verify_audit.md) — `audit_log.jsonl` için kardeş doğrulayıcı. -- [`verify_annex_iv_subcommand.md`](verify_annex_iv_subcommand.md) — Annex IV teknik dokümantasyon artifact'ı için kardeş doğrulayıcı. +- [`audit_event_catalog.md`](audit_event_catalog-tr.md) — kanonik event sözlüğü. +- [`verify_audit.md`](verify_audit-tr.md) — `audit_log.jsonl` için kardeş doğrulayıcı. +- [`verify_annex_iv_subcommand.md`](verify_annex_iv_subcommand-tr.md) — Annex IV teknik dokümantasyon artifact'ı için kardeş doğrulayıcı. - [GGUF Export kullanım kılavuzu sayfası](../usermanuals/tr/deployment/gguf-export.md) — bu doğrulayıcının tükettiği sidecar'ı yazan üretim tarafına dair operatör-odaklı kılavuz. - `forgelm.cli.subcommands._verify_gguf.verify_gguf` — entegratörlerin CLI'dan geçmeden doğrudan çağırdığı kütüphane giriş noktası. diff --git a/docs/reference/verify_integrity_subcommand-tr.md b/docs/reference/verify_integrity_subcommand-tr.md new file mode 100644 index 00000000..b219a928 --- /dev/null +++ b/docs/reference/verify_integrity_subcommand-tr.md @@ -0,0 +1,101 @@ +# `forgelm verify-integrity` — Referans + +> **Hedef kitle:** Eğitilmiş bir model dizininin, eğitim sırasında kaydedilen SHA-256 manifestiyle hâlâ eşleştiğini doğrulayan uyumluluk operatörleri ve CI kapıları (AB YZ Yasası Madde 15). +> **Ayna:** [verify_integrity_subcommand.md](verify_integrity_subcommand.md) + +`verify-integrity` alt komutu, Madde 15 `model_integrity.json` manifestinin tüketici karşılığıdır. Uyumluluk export'u, model dizinindeki her dosyanın SHA-256 özetini yazar; `verify-integrity` bu manifesti geri okur, her dosyanın SHA-256'sını yeniden hesaplar ve manifest üretildikten sonra **değiştirilen**, **silinen** veya **eklenen** her bir yapıtı raporlar. CLI, kütüphane giriş noktası `forgelm.cli.subcommands._verify_integrity.verify_integrity`'ye devreder ve yapılandırılmış bir `VerifyIntegrityResult` döndürür. + +## Söz dizimi + +```text +forgelm verify-integrity [--output-format {text,json}] + [-q] [--log-level {DEBUG,INFO,WARNING,ERROR}] + path +``` + +`path` (konumsal, zorunlu) — `model_integrity.json` içeren model dizininin yolu. + +## Bayraklar + +| Bayrak | Varsayılan | Açıklama | +|---|---|---| +| `--output-format {text,json}` | `text` | `text` (varsayılan) `OK:` / `FAIL:` ile birlikte dosya bazlı dökümü yazar; `json` ise tam `VerifyIntegrityResult` zarfını yazar (`{"success", "valid", "reason", "changed", "removed", "added", "verified_count", "path"}`). | +| `-q`, `--quiet` | _kapalı_ | INFO günlüklerini bastırır. | +| `--log-level {DEBUG,INFO,WARNING,ERROR}` | `INFO` | Günlük ayrıntı düzeyini ayarlar. | +| `-h`, `--help` | — | argparse yardımını gösterip çıkar. | + +## Çıkış kodları + +| Kod | Anlamı | +|---|---| +| `0` | Kaydedilmiş her yapıt mevcut ve SHA-256'sı değişmemiş, dizinde beklenmeyen fazladan dosya yok. | +| `1` | Çağıran / girdi hatası (yol yok, `model_integrity.json` bulunamadı veya normal dosya değil, bozuk JSON) VEYA bir bütünlük uyuşmazlığı: manifest üretildikten sonra en az bir dosya değiştirilmiş, silinmiş veya eklenmiş. Model manifestiyle eşleşmiyor. | +| `2` | Erişilebilir bir yolda gerçek çalışma-zamanı G/Ç hatası — okuma hataları, gezinme sırasında izin reddi vb. Yol erişilebilirdi ancak doğrulama sırasında okunamaz hâle geldi. | + +Kodlar `forgelm/cli/subcommands/_verify_integrity.py::_run_verify_integrity_cmd` tarafından emit edilir. Açık-sözleşme semantiği `docs/standards/error-handling.md` içinde sabitlenmiştir. + +## Neler denetlenir + +| Denetim | Hata durumu | +|---|---| +| **Kaydedilmiş yapıt mevcut** | `model_integrity.json` içinde listelenen ancak diskte artık bulunmayan dosya → `removed`, çıkış `1`. | +| **Kaydedilmiş yapıt değişmemiş** | Yeniden hesaplanan SHA-256'sı manifestten farklı olan dosya → `changed`, çıkış `1`. | +| **Fazladan dosya yok** | Diskte olup manifeste bulunmayan dosya → `added`, çıkış `1`. Manifest dosyasının kendisi (`model_integrity.json`) bu gezinmeden hariç tutulur çünkü model yapıtlarından sonra yazılır. | + +## Emit edilen audit event'leri + +`forgelm verify-integrity` **salt-okunur bir doğrulayıcıdır** ve `audit_log.jsonl`'a **hiçbir** girdi emit etmez. Bütünlük-manifestinin *üretimini* (doğrulamasını değil) işaret eden olaylar çalıştırma düzeyindeki eğitim olaylarına biner; bkz. [audit_event_catalog.md](audit_event_catalog-tr.md). + +## Örnekler + +### Metin çıktısı (varsayılan) + +```shell +$ forgelm verify-integrity checkpoints/run/final_model +OK: checkpoints/run/final_model + All 7 recorded artifact(s) present and unchanged. +``` + +### JSON çıktısı (CI tüketicileri için) + +```shell +$ forgelm verify-integrity --output-format json \ + checkpoints/run/final_model +{ + "success": true, + "valid": true, + "reason": "All 7 recorded artifact(s) present and unchanged.", + "changed": [], + "removed": [], + "added": [], + "verified_count": 7, + "path": "/abs/path/checkpoints/run/final_model" +} +``` + +### Hata: bir ağırlık dosyası eğitimden sonra değiştirildi + +```shell +$ forgelm verify-integrity checkpoints/run/final_model +FAIL: checkpoints/run/final_model + Model artifacts do not match model_integrity.json: 1 changed. + changed: model.safetensors +$ echo $? +1 +``` + +### Hata: eksik manifest + +```shell +$ forgelm verify-integrity checkpoints/run/final_model +Integrity manifest not found: expected 'checkpoints/run/final_model/model_integrity.json' (FileNotFoundError). +$ echo $? +1 +``` + +## Bkz. + +- [`audit_event_catalog.md`](audit_event_catalog-tr.md) — kanonik olay sözcük dağarcığı. +- [`verify_gguf_subcommand.md`](verify_gguf_subcommand-tr.md) — export edilen GGUF dosyaları için eşlik eden doğrulayıcı. +- [`verify_annex_iv_subcommand.md`](verify_annex_iv_subcommand-tr.md) — Annex IV teknik-dokümantasyon yapıtı için eşlik eden doğrulayıcı. +- `forgelm.cli.subcommands._verify_integrity.verify_integrity` — entegratörlerin CLI'den geçmeden doğrudan çağırdığı kütüphane giriş noktası. diff --git a/docs/reference/verify_integrity_subcommand.md b/docs/reference/verify_integrity_subcommand.md new file mode 100644 index 00000000..edf1e5e6 --- /dev/null +++ b/docs/reference/verify_integrity_subcommand.md @@ -0,0 +1,101 @@ +# `forgelm verify-integrity` — Reference + +> **Audience:** Compliance operators and CI gates verifying that a trained model directory still matches the SHA-256 manifest recorded at training time (EU AI Act Article 15). +> **Mirror:** [verify_integrity_subcommand-tr.md](verify_integrity_subcommand-tr.md) + +The `verify-integrity` subcommand is the consuming counterpart to the Article 15 `model_integrity.json` manifest. The compliance export writes a SHA-256 hash of every file in the model directory; `verify-integrity` reads that manifest back, recomputes each file's SHA-256, and reports any artifact that was **changed**, **removed**, or **added** since the manifest was generated. The CLI delegates to the library entry point `forgelm.cli.subcommands._verify_integrity.verify_integrity` and returns a structured `VerifyIntegrityResult`. + +## Synopsis + +```text +forgelm verify-integrity [--output-format {text,json}] + [-q] [--log-level {DEBUG,INFO,WARNING,ERROR}] + path +``` + +`path` (positional, required) — path to the model directory containing `model_integrity.json`. + +## Flags + +| Flag | Default | Description | +|---|---|---| +| `--output-format {text,json}` | `text` | `text` (default) prints `OK:` / `FAIL:` plus the per-file breakdown; `json` prints the full `VerifyIntegrityResult` envelope (`{"success", "valid", "reason", "changed", "removed", "added", "verified_count", "path"}`). | +| `-q`, `--quiet` | _off_ | Suppress INFO logs. | +| `--log-level {DEBUG,INFO,WARNING,ERROR}` | `INFO` | Set logging verbosity. | +| `-h`, `--help` | — | Show argparse help and exit. | + +## Exit codes + +| Code | Meaning | +|---|---| +| `0` | Every recorded artifact is present and its SHA-256 is unchanged, and no unexpected extra files exist in the directory. | +| `1` | Caller / input error (path missing, `model_integrity.json` not found or not a regular file, malformed JSON) OR an integrity mismatch: at least one file was changed, removed, or added since the manifest was generated. The model does not match its manifest. | +| `2` | Genuine runtime I/O failure on a reachable path — read errors, permission denied mid-walk, etc. The path was accessible but became unreadable during verification. | + +The codes are emitted by `forgelm/cli/subcommands/_verify_integrity.py::_run_verify_integrity_cmd`. Public-contract semantics are pinned in `docs/standards/error-handling.md`. + +## What is checked + +| Check | Failure mode | +|---|---| +| **Recorded artifact present** | A file listed in `model_integrity.json` that no longer exists on disk → `removed`, exit `1`. | +| **Recorded artifact unchanged** | A file whose recomputed SHA-256 differs from the manifest → `changed`, exit `1`. | +| **No extra files** | A file on disk that is not in the manifest → `added`, exit `1`. The manifest itself (`model_integrity.json`) is excluded from this walk because it is written after the model artifacts. | + +## Audit events emitted + +`forgelm verify-integrity` is a **read-only verifier** and emits **no** entries to `audit_log.jsonl`. The events that signal integrity-manifest *production* (not verification) ride the run-level training events; see [audit_event_catalog.md](audit_event_catalog.md). + +## Examples + +### Text output (default) + +```shell +$ forgelm verify-integrity checkpoints/run/final_model +OK: checkpoints/run/final_model + All 7 recorded artifact(s) present and unchanged. +``` + +### JSON output (CI consumers) + +```shell +$ forgelm verify-integrity --output-format json \ + checkpoints/run/final_model +{ + "success": true, + "valid": true, + "reason": "All 7 recorded artifact(s) present and unchanged.", + "changed": [], + "removed": [], + "added": [], + "verified_count": 7, + "path": "/abs/path/checkpoints/run/final_model" +} +``` + +### Failure: a weights file was modified after training + +```shell +$ forgelm verify-integrity checkpoints/run/final_model +FAIL: checkpoints/run/final_model + Model artifacts do not match model_integrity.json: 1 changed. + changed: model.safetensors +$ echo $? +1 +``` + +### Failure: missing manifest + +```shell +$ forgelm verify-integrity checkpoints/run/final_model +Integrity manifest not found: expected 'checkpoints/run/final_model/model_integrity.json' (FileNotFoundError). +$ echo $? +1 +``` + +## See also + +- [`audit_event_catalog.md`](audit_event_catalog.md) — canonical event vocabulary. +- [`verify_gguf_subcommand.md`](verify_gguf_subcommand.md) — companion verifier for exported GGUF files. +- [`verify_annex_iv_subcommand.md`](verify_annex_iv_subcommand.md) — companion verifier for the Annex IV technical-documentation artifact. +- `forgelm.cli.subcommands._verify_integrity.verify_integrity` — the library entry point integrators call directly without going through the CLI. diff --git a/docs/roadmap-tr.md b/docs/roadmap-tr.md index e85fa61a..bb10d67d 100644 --- a/docs/roadmap-tr.md +++ b/docs/roadmap-tr.md @@ -9,13 +9,13 @@ | ✅ Tamam | [Faz 1-9](roadmap/completed-phases.md) | SOTA iyileştirmeleri, değerlendirme, güvenilirlik, kurumsal entegrasyon, ekosistem, hizalama stack'i, güvenlik, EU AI Act uyumluluğu (Madde 9-17 + Ek IV), gelişmiş güvenlik zekası | | ✅ Tamam | [Faz 10 — Post-Training Tamamlama](roadmap/completed-phases.md) | `inference.py`, `chat`, `export` (GGUF), `--fit-check`, `deploy` — `v0.4.0` | | ✅ Tamam | [Faz 10.5 — Quickstart Katmanı ve Onboarding](roadmap/completed-phases.md) | `forgelm quickstart