Skip to content

Qwen: add gated 0.2.11 acceptance experiments - #31

Draft
atimics wants to merge 16 commits into
AnOversizedMooseWithSocks:0.2.11from
atimics:codex/qwen-ilxyr-experiments
Draft

Qwen: add gated 0.2.11 acceptance experiments#31
atimics wants to merge 16 commits into
AnOversizedMooseWithSocks:0.2.11from
atimics:codex/qwen-ilxyr-experiments

Conversation

@atimics

@atimics atimics commented Aug 12, 2026

Copy link
Copy Markdown

Summary

This narrows the Qwen work for 0.2.11 to the loader, tokenizer, configuration, text-runtime, and diagnostic surface, while keeping checkpoint-changing work explicitly experimental.

The committee-facing proposal and terminal v1 execution record are in stacked draft PR atimics/holostuff#3. Review implementation here and experiment policy/evidence there.

  • disables spectral filtering by default and labels it research-only
  • requires --experimental for the layer-prepending installer
  • fixes paired statistical inference and the preregistered non-inferiority gate
  • removes avoidable float64 amplification and uses file-backed lazy loading
  • fixes the caller-relative shell installer path
  • separates installation and held-out evaluation corpora and rejects identical content
  • requires clean source, tokenizer parity, and reference-logit parity before installation
  • records a complete top-level model-file manifest
  • preserves the requested virtual-environment interpreter instead of resolving its symlink to the system Python
  • replaces the machine-specific fixture with a portable, offline miniature Qwen fixture and CI contract

Acceptance contract

The real-model contract requires:

  • exact tokenizer IDs and reference-logit parity before installation
  • at least 4,096 paired evaluation positions
  • a 95% paired moving-block bootstrap and preregistered +1% perplexity-regression ceiling
  • peak process and accelerator memory reporting
  • reload and finite-logit execution of the emitted checkpoint
  • text generation through official Transformers
  • an official Transformers image-input generation smoke test
  • spectral filtering disabled throughout

First real execution

The owner-authorized v1 AWS attempt is complete with ilxyr outcome execution_failure. It passed all frozen-input checks and all twelve ilxyr admission gates, then exited after 51 ms before model evaluation because the generated command invoked /usr/bin/python3.12 rather than the prepared virtual environment. No scientific acceptance/rejection claim can be made from v1.

That concrete runtime bug is fixed by 0bda4f1, with a regression test. The full result, verified .ilxyr ledger, native/RO-Crate/in-toto exports, raw logs, and receipt are committed in results PR #3 and permanently indexed at Arweave root jguW…mHBk.

This PR does not claim that Qwen3.5-0.8B has completed the acceptance contract. A corrected v2 run needs a new experiment identity and explicit authorization. The installer remains experimental.

Validation

  • 10 passed in tests/test_qwen_acceptance.py
  • portable Qwen fixture loads, tokenizes, and runs from an arbitrary working directory
  • experimental miniature-checkpoint installation emits and reloads its output
  • generated project exercises ilxyr contribution, compile, forecasts, funding, admission, execution record, and workspace verification
  • v1 ilxyr ledger verifies: 15 objects, 16 events, valid: true
  • v1 permanent publication: 63 confirmed Arweave transactions, zero winc charged, public re-download hashes matched
  • focused Qwen and wiring regression suite before launch: 22 passed
  • UNICRON safetensors/GGUF round-trip self-test: passed
  • generated-doc and build-ergonomics regression suite: 12 passed
  • hosted Qwen contract and aggregate gates previously passed in run 31627998803

Release gate

For 0.2.11, retain the compatibility/runtime work and diagnostics. Keep spectral filtering research-only and the layer-prepending installer behind its experimental acknowledgement. Do not promote the installer until a new complete real-model acceptance run reaches and passes every gate.

PR #30 remains the separate long-term liblecore architecture track.

@atimics atimics changed the title Qwen: add gated 0.2.11 acceptance experiments RFC: Qwen 0.2.11 acceptance experiment and safety gates Aug 12, 2026
@atimics
atimics force-pushed the codex/qwen-ilxyr-experiments branch from 8994959 to e4be27b Compare August 12, 2026 17:47
@atimics atimics changed the title RFC: Qwen 0.2.11 acceptance experiment and safety gates Qwen: add gated 0.2.11 acceptance experiments Aug 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants