Skip to content

chore: sync local phase work to dev - #3187

Merged
yastman merged 82 commits into
devfrom
chore/sync-dev-20260722
Jul 23, 2026
Merged

chore: sync local phase work to dev#3187
yastman merged 82 commits into
devfrom
chore/sync-dev-20260722

Conversation

@yastman

@yastman yastman commented Jul 22, 2026

Copy link
Copy Markdown
Owner

Publishes the local P23-P29 phase commits and reconciles the two origin/dev commits through merge commit c04d7db.

Verification:

  • uv lock --check
  • core gate: 258 passed
  • graph paths: 5 passed
  • portable no-service lane: 26 passed, 4 skipped, 1 xfailed
  • full no-service lane: 2 Windows-only orchestrator/tmux failures; 26 passed

Direct push to protected dev was rejected because 5 required checks must pass.

yastman and others added 30 commits July 6, 2026 08:27
card_4972d1f2f598 — Dockerfile.ingestion offline-mode bug

HF_HUB_OFFLINE=1 and TRANSFORMERS_OFFLINE=1 were hardcoded in ENV, but
model pre-warm is best-effort. If pre-warm silently fails during docker
build, the image has no models and the offline flags block runtime download.

Fix: Move offline flag enforcement to entrypoint.sh with a conditional
check — only enforce offline mode if the HF cache directory is non-empty.
Otherwise unset the vars to allow runtime download.

Adds contract test test_dockerfile_offline_mode_contract.py verifying
the policy (no hardcoded offline ENV, entrypoint has conditional logic).

Verified: hf_offline grep >0, pytest tests/contract/ 2 passed, make test-core 152 passed.
… gone

card_ef037d6f10de — README doc-lie fix

README:151 falsely claimed '@observe decorators are gone'. Reality:
- Langfuse SDK is gone (no from langfuse imports) ✓
- @observe decorators REMAIN as no-op shims (src.observability /
  telegram_bot.observability) — consistent with AGENTS.md

Rewrites the Langfuse blurb to match AGENTS.md accurately.

Verified: grep 'decorators are gone' → 0 hits, grep 'no-op' → hit found,
make test-core → 152 passed.
card_20ee99b1cb15

U14: delete services/bge-m3-api/requirements.txt
U12: hide internal errors from HTTPException 500 body
U9:  BGEM3SyncClient context manager support
The BGE-M3 sidecar is a single-worker CPU-ONNX process that returns
429/503 under load. Two asymmetries left the client failing without retry:

- bge_retry had retry_on_http_status=False -> only transport errors/timeouts
  were retried, never 5xx/429.
- BGEM3SyncClient methods had no @bge_retry at all (async path had it).

Fix:
- Flip bge_retry to retry_on_http_status=True (429/503/5xx already in the
  shared RETRYABLE_HTTP_STATUS_CODES set).
- Decorate the 4 BGEM3SyncClient encode_* methods with @bge_retry.
- Add retry-policy tests (sync 503/429 retry+succeed, exhaust+reraise,
  no-retry on 400, async 503); mark the module no_services and zero
  tenacity back-off in an autouse fixture so the gate runs them fast.
… access

- Add test_poison_index_skipped_and_warning_logged in TestUpsertChunksSyncBehavior:
  feeds a 2-chunk batch where encode_hybrid returns partial_failures=[{index:1}],
  asserts exactly 1 PointStruct is built (index 0), verifies vectors come from
  index 0, and asserts a warning was logged.
- Change poison_indices set-build to guard against malformed server responses:
  {pf['index'] for pf in ... if 'index' in pf}

Resolves review blocker B1 for card_06c91625a24c.
- bge-m3-api: drop FastAPIInstrumentor / LoggingInstrumentor activation +
  propagate/attach/detach trace-context middleware; remove opentelemetry-api
  and opentelemetry-instrumentation-* deps + constraint pins; relock (0 otel).
- src/observability: delete dead bootstrap.py (disable_otel_exporter,
  is_endpoint_reachable — 0 production callers); drop the OTEL shim imports +
  __all__ entries from __init__.py.
- telegram_bot: delete observability_bootstrap.py (pure OTEL re-export shim).
- compose.yml: drop OTEL_SERVICE_NAME / OTEL_PROPAGATORS from bot service.
- logging_config.py: kept trace-context fields (observability via structured logs).
- tests: delete obsolete OTEL contracts/tests (propagators contract,
  TestOtelServiceNameDefaults, googleapis-otel-transitive, bge-m3 langfuse/otel
  env, observability OTEL-helper tests); drop vestigial _is_endpoint_reachable
  patches in test_main / test_bot_log_triage. Guardians kept
  (test_no_monolith_otel_auto_instrumentation, product_events no-otel import).
- src/runtime/services/rag_core.py: remove stub def + 15 call-sites
- src/runtime/services/query_preprocessor.py: remove stub def + 4 call-sites
- telegram_bot/services/rag/rag_core.py: remove stub def + call-sites,
  drop now-unused contextlib import and get_client import
- telegram_bot/services/rag/query_preprocessor.py: remove stub def
- src/observability/__init__.py: update stale docstring reference

Closes card_1a615feb22f1 (P25 Remove Langfuse residue)
Add EXPECTED_OUTPUTS class constant and assertion to guard against
silent dense↔colbert transposition if model is re-exported with
different output order.

Add three output-contract tests (correct names OK, wrong order raises,
wrong names raises) and one get_model FileNotFoundError test.

card_40c2e1606e62
…d_17e07f2a213e)

- Add bge-extras optional dep to pyproject.toml (fastapi, httpx)
- Add test-bge-extras Makefile target (uv sync --extra bge-extras, pytest both files)
- Replace pytest.importorskip with explicit pytest.skip + actionable message
- Fix _import_lexical_weights_fn: save/restore sys.modules['app'] to avoid
  corrupting module-scoped bge_app fixture (was causing TestRunEncode failures)
- Remove stale test_traceparent_header_is_extracted (OTel removed, card_81add5ba4a66)

make test-bge-extras: 32 passed, exit 0
make test-core: 152 passed, exit 0
…equest

- Add ENCODE_MAX_ITEMS = 64 constant to app.py
- Add Field(max_length=ENCODE_MAX_ITEMS) validator to EncodeRequest.texts
  (mirrors RerankRequest.documents pattern with le/max_length validator)
- Remove dead batch_size parameter from EncodeRequest (ignored by single
  session.run; now uses settings.BATCH_SIZE directly in _run_encode)
- Add TestEncodeMaxItems: over-limit → 422, at-limit → 200

Closes card_7ac0f91b8132
* fix(docker): conditional HF_HUB_OFFLINE in entrypoint, not hardcoded ENV

card_4972d1f2f598 — Dockerfile.ingestion offline-mode bug

HF_HUB_OFFLINE=1 and TRANSFORMERS_OFFLINE=1 were hardcoded in ENV, but
model pre-warm is best-effort. If pre-warm silently fails during docker
build, the image has no models and the offline flags block runtime download.

Fix: Move offline flag enforcement to entrypoint.sh with a conditional
check — only enforce offline mode if the HF cache directory is non-empty.
Otherwise unset the vars to allow runtime download.

Adds contract test test_dockerfile_offline_mode_contract.py verifying
the policy (no hardcoded offline ENV, entrypoint has conditional logic).

Verified: hf_offline grep >0, pytest tests/contract/ 2 passed, make test-core 152 passed.

* docs: fix README Honest Current State — Langfuse shims are no-op, not gone

card_ef037d6f10de — README doc-lie fix

README:151 falsely claimed '@observe decorators are gone'. Reality:
- Langfuse SDK is gone (no from langfuse imports) ✓
- @observe decorators REMAIN as no-op shims (src.observability /
  telegram_bot.observability) — consistent with AGENTS.md

Rewrites the Langfuse blurb to match AGENTS.md accurately.

Verified: grep 'decorators are gone' → 0 hits, grep 'no-op' → hit found,
make test-core → 152 passed.

* test(bge-m3): add unit tests for _lexical_weights_to_qdrant_sparse non-passthrough branch

card_a8456a6e7190

* fix(bge-m3): tech debt cleanup U9/U12/U14

card_20ee99b1cb15

U14: delete services/bge-m3-api/requirements.txt
U12: hide internal errors from HTTPException 500 body
U9:  BGEM3SyncClient context manager support

* fix(bge-m3): retry sync client on 429/503 (card_7cc460feaec0)

The BGE-M3 sidecar is a single-worker CPU-ONNX process that returns
429/503 under load. Two asymmetries left the client failing without retry:

- bge_retry had retry_on_http_status=False -> only transport errors/timeouts
  were retried, never 5xx/429.
- BGEM3SyncClient methods had no @bge_retry at all (async path had it).

Fix:
- Flip bge_retry to retry_on_http_status=True (429/503/5xx already in the
  shared RETRYABLE_HTTP_STATUS_CODES set).
- Decorate the 4 BGEM3SyncClient encode_* methods with @bge_retry.
- Add retry-policy tests (sync 503/429 retry+succeed, exhaust+reraise,
  no-retry on 400, async 503); mark the module no_services and zero
  tenacity back-off in an autouse fixture so the gate runs them fast.

* fix(bge-m3): carry partial_failures through client, skip poison vectors on ingest (card_06c91625a24c)

* fix(ingestion): add poison-skip test + defensive partial_failures key access

- Add test_poison_index_skipped_and_warning_logged in TestUpsertChunksSyncBehavior:
  feeds a 2-chunk batch where encode_hybrid returns partial_failures=[{index:1}],
  asserts exactly 1 PointStruct is built (index 0), verifies vectors come from
  index 0, and asserts a warning was logged.
- Change poison_indices set-build to guard against malformed server responses:
  {pf['index'] for pf in ... if 'index' in pf}

Resolves review blocker B1 for card_06c91625a24c.

* Remove OpenTelemetry fully (card_81add5ba4a66)

- bge-m3-api: drop FastAPIInstrumentor / LoggingInstrumentor activation +
  propagate/attach/detach trace-context middleware; remove opentelemetry-api
  and opentelemetry-instrumentation-* deps + constraint pins; relock (0 otel).
- src/observability: delete dead bootstrap.py (disable_otel_exporter,
  is_endpoint_reachable — 0 production callers); drop the OTEL shim imports +
  __all__ entries from __init__.py.
- telegram_bot: delete observability_bootstrap.py (pure OTEL re-export shim).
- compose.yml: drop OTEL_SERVICE_NAME / OTEL_PROPAGATORS from bot service.
- logging_config.py: kept trace-context fields (observability via structured logs).
- tests: delete obsolete OTEL contracts/tests (propagators contract,
  TestOtelServiceNameDefaults, googleapis-otel-transitive, bge-m3 langfuse/otel
  env, observability OTEL-helper tests); drop vestigial _is_endpoint_reachable
  patches in test_main / test_bot_log_triage. Guardians kept
  (test_no_monolith_otel_auto_instrumentation, product_events no-otel import).

* Remove _update_current_span no-op stubs and call-sites (#2844)

- src/runtime/services/rag_core.py: remove stub def + 15 call-sites
- src/runtime/services/query_preprocessor.py: remove stub def + 4 call-sites
- telegram_bot/services/rag/rag_core.py: remove stub def + call-sites,
  drop now-unused contextlib import and get_client import
- telegram_bot/services/rag/query_preprocessor.py: remove stub def
- src/observability/__init__.py: update stale docstring reference

Closes card_1a615feb22f1 (P25 Remove Langfuse residue)

* test: delete stale TestRagCoreObserveContract class (#2162 contract, langfuse removed #2844)

* feat: assert ONNX output name contract in ONNXEmbeddingModel.__init__

Add EXPECTED_OUTPUTS class constant and assertion to guard against
silent dense↔colbert transposition if model is re-exported with
different output order.

Add three output-contract tests (correct names OK, wrong order raises,
wrong names raises) and one get_model FileNotFoundError test.

card_40c2e1606e62

* chore(observability): remove @observe/traced_pipeline/propagate_attributes no-op call-sites (card_40feeaff139c)

* feat: add test-bge-extras lane for BGE-M3 FastAPI endpoint tests (card_17e07f2a213e)

- Add bge-extras optional dep to pyproject.toml (fastapi, httpx)
- Add test-bge-extras Makefile target (uv sync --extra bge-extras, pytest both files)
- Replace pytest.importorskip with explicit pytest.skip + actionable message
- Fix _import_lexical_weights_fn: save/restore sys.modules['app'] to avoid
  corrupting module-scoped bge_app fixture (was causing TestRunEncode failures)
- Remove stale test_traceparent_header_is_extracted (OTel removed, card_81add5ba4a66)

make test-bge-extras: 32 passed, exit 0
make test-core: 152 passed, exit 0

* test: fix TestRunEncode isolation — patch get_model via autouse fixture

* fix: add ENCODE_MAX_ITEMS limit + remove dead batch_size from EncodeRequest

- Add ENCODE_MAX_ITEMS = 64 constant to app.py
- Add Field(max_length=ENCODE_MAX_ITEMS) validator to EncodeRequest.texts
  (mirrors RerankRequest.documents pattern with le/max_length validator)
- Remove dead batch_size parameter from EncodeRequest (ignored by single
  session.run; now uses settings.BATCH_SIZE directly in _run_encode)
- Add TestEncodeMaxItems: over-limit → 422, at-limit → 200

Closes card_7ac0f91b8132

* remove propagate_attributes and get_client call-sites from 9 remaining files (card_76e6a84b83f5)

* docs: reconcile module READMEs with code + add 8 missing ones

Doc-gardening sweep (34 docs):
- fix drifted claims/paths/examples: src/runtime scaffold->built engine,
  src/retrieval nonexistent classes, docker/ phantom subdirs, tests/ dead
  langfuse/traces gate, api delegation target, telegram_bot dir/service paths
- repair broken CONTRIBUTING -> docs/ONBOARDING link; honest 'planned docs' note
- add code-derived READMEs: src/{services,observability,adapters},
  telegram_bot/{pipeline,lifecycle,graph(legacy facade),observability,preflight}
- link new docs into docs/README hub

Verified: markdown-links contract green, 0 broken links.
yastman and others added 29 commits July 6, 2026 15:07
* fix(docker): conditional HF_HUB_OFFLINE in entrypoint, not hardcoded ENV

card_4972d1f2f598 — Dockerfile.ingestion offline-mode bug

HF_HUB_OFFLINE=1 and TRANSFORMERS_OFFLINE=1 were hardcoded in ENV, but
model pre-warm is best-effort. If pre-warm silently fails during docker
build, the image has no models and the offline flags block runtime download.

Fix: Move offline flag enforcement to entrypoint.sh with a conditional
check — only enforce offline mode if the HF cache directory is non-empty.
Otherwise unset the vars to allow runtime download.

Adds contract test test_dockerfile_offline_mode_contract.py verifying
the policy (no hardcoded offline ENV, entrypoint has conditional logic).

Verified: hf_offline grep >0, pytest tests/contract/ 2 passed, make test-core 152 passed.

* docs: fix README Honest Current State — Langfuse shims are no-op, not gone

card_ef037d6f10de — README doc-lie fix

README:151 falsely claimed '@observe decorators are gone'. Reality:
- Langfuse SDK is gone (no from langfuse imports) ✓
- @observe decorators REMAIN as no-op shims (src.observability /
  telegram_bot.observability) — consistent with AGENTS.md

Rewrites the Langfuse blurb to match AGENTS.md accurately.

Verified: grep 'decorators are gone' → 0 hits, grep 'no-op' → hit found,
make test-core → 152 passed.

* test(bge-m3): add unit tests for _lexical_weights_to_qdrant_sparse non-passthrough branch

card_a8456a6e7190

* fix(bge-m3): tech debt cleanup U9/U12/U14

card_20ee99b1cb15

U14: delete services/bge-m3-api/requirements.txt
U12: hide internal errors from HTTPException 500 body
U9:  BGEM3SyncClient context manager support

* fix(bge-m3): retry sync client on 429/503 (card_7cc460feaec0)

The BGE-M3 sidecar is a single-worker CPU-ONNX process that returns
429/503 under load. Two asymmetries left the client failing without retry:

- bge_retry had retry_on_http_status=False -> only transport errors/timeouts
  were retried, never 5xx/429.
- BGEM3SyncClient methods had no @bge_retry at all (async path had it).

Fix:
- Flip bge_retry to retry_on_http_status=True (429/503/5xx already in the
  shared RETRYABLE_HTTP_STATUS_CODES set).
- Decorate the 4 BGEM3SyncClient encode_* methods with @bge_retry.
- Add retry-policy tests (sync 503/429 retry+succeed, exhaust+reraise,
  no-retry on 400, async 503); mark the module no_services and zero
  tenacity back-off in an autouse fixture so the gate runs them fast.

* fix(bge-m3): carry partial_failures through client, skip poison vectors on ingest (card_06c91625a24c)

* fix(ingestion): add poison-skip test + defensive partial_failures key access

- Add test_poison_index_skipped_and_warning_logged in TestUpsertChunksSyncBehavior:
  feeds a 2-chunk batch where encode_hybrid returns partial_failures=[{index:1}],
  asserts exactly 1 PointStruct is built (index 0), verifies vectors come from
  index 0, and asserts a warning was logged.
- Change poison_indices set-build to guard against malformed server responses:
  {pf['index'] for pf in ... if 'index' in pf}

Resolves review blocker B1 for card_06c91625a24c.

* Remove OpenTelemetry fully (card_81add5ba4a66)

- bge-m3-api: drop FastAPIInstrumentor / LoggingInstrumentor activation +
  propagate/attach/detach trace-context middleware; remove opentelemetry-api
  and opentelemetry-instrumentation-* deps + constraint pins; relock (0 otel).
- src/observability: delete dead bootstrap.py (disable_otel_exporter,
  is_endpoint_reachable — 0 production callers); drop the OTEL shim imports +
  __all__ entries from __init__.py.
- telegram_bot: delete observability_bootstrap.py (pure OTEL re-export shim).
- compose.yml: drop OTEL_SERVICE_NAME / OTEL_PROPAGATORS from bot service.
- logging_config.py: kept trace-context fields (observability via structured logs).
- tests: delete obsolete OTEL contracts/tests (propagators contract,
  TestOtelServiceNameDefaults, googleapis-otel-transitive, bge-m3 langfuse/otel
  env, observability OTEL-helper tests); drop vestigial _is_endpoint_reachable
  patches in test_main / test_bot_log_triage. Guardians kept
  (test_no_monolith_otel_auto_instrumentation, product_events no-otel import).

* Remove _update_current_span no-op stubs and call-sites (#2844)

- src/runtime/services/rag_core.py: remove stub def + 15 call-sites
- src/runtime/services/query_preprocessor.py: remove stub def + 4 call-sites
- telegram_bot/services/rag/rag_core.py: remove stub def + call-sites,
  drop now-unused contextlib import and get_client import
- telegram_bot/services/rag/query_preprocessor.py: remove stub def
- src/observability/__init__.py: update stale docstring reference

Closes card_1a615feb22f1 (P25 Remove Langfuse residue)

* test: delete stale TestRagCoreObserveContract class (#2162 contract, langfuse removed #2844)

* feat: assert ONNX output name contract in ONNXEmbeddingModel.__init__

Add EXPECTED_OUTPUTS class constant and assertion to guard against
silent dense↔colbert transposition if model is re-exported with
different output order.

Add three output-contract tests (correct names OK, wrong order raises,
wrong names raises) and one get_model FileNotFoundError test.

card_40c2e1606e62

* chore(observability): remove @observe/traced_pipeline/propagate_attributes no-op call-sites (card_40feeaff139c)

* feat: add test-bge-extras lane for BGE-M3 FastAPI endpoint tests (card_17e07f2a213e)

- Add bge-extras optional dep to pyproject.toml (fastapi, httpx)
- Add test-bge-extras Makefile target (uv sync --extra bge-extras, pytest both files)
- Replace pytest.importorskip with explicit pytest.skip + actionable message
- Fix _import_lexical_weights_fn: save/restore sys.modules['app'] to avoid
  corrupting module-scoped bge_app fixture (was causing TestRunEncode failures)
- Remove stale test_traceparent_header_is_extracted (OTel removed, card_81add5ba4a66)

make test-bge-extras: 32 passed, exit 0
make test-core: 152 passed, exit 0

* test: fix TestRunEncode isolation — patch get_model via autouse fixture

* fix: add ENCODE_MAX_ITEMS limit + remove dead batch_size from EncodeRequest

- Add ENCODE_MAX_ITEMS = 64 constant to app.py
- Add Field(max_length=ENCODE_MAX_ITEMS) validator to EncodeRequest.texts
  (mirrors RerankRequest.documents pattern with le/max_length validator)
- Remove dead batch_size parameter from EncodeRequest (ignored by single
  session.run; now uses settings.BATCH_SIZE directly in _run_encode)
- Add TestEncodeMaxItems: over-limit → 422, at-limit → 200

Closes card_7ac0f91b8132

* remove propagate_attributes and get_client call-sites from 9 remaining files (card_76e6a84b83f5)

* docs: reconcile module READMEs with code + add 8 missing ones

Doc-gardening sweep (34 docs):
- fix drifted claims/paths/examples: src/runtime scaffold->built engine,
  src/retrieval nonexistent classes, docker/ phantom subdirs, tests/ dead
  langfuse/traces gate, api delegation target, telegram_bot dir/service paths
- repair broken CONTRIBUTING -> docs/ONBOARDING link; honest 'planned docs' note
- add code-derived READMEs: src/{services,observability,adapters},
  telegram_bot/{pipeline,lifecycle,graph(legacy facade),observability,preflight}
- link new docs into docs/README hub

Verified: markdown-links contract green, 0 broken links.
- Delete two forward-trace-id tests from tests/unit/agents/test_rag_tool.py
- Replace langfuse_trace_id with trace_id in tests/unit/api/test_schemas.py
- Replace langfuse_trace_id with trace_id in tests/unit/api/test_rag_api.py
- Delete test_query_propagates_explicit_langfuse_trace_id from test_rag_api_runtime.py
- Delete test_generic_error_handler_uses_langfuse_trace_id_when_available from test_rag_api_runtime.py
- Delete dead skip-marked test_cmd_call_dispatch_includes_langfuse_trace_id from test_bot_handlers.py
- Also resolve rebase conflict on pyproject.toml (duplicate bge-extras key)
* fix(docker): conditional HF_HUB_OFFLINE in entrypoint, not hardcoded ENV

card_4972d1f2f598 — Dockerfile.ingestion offline-mode bug

HF_HUB_OFFLINE=1 and TRANSFORMERS_OFFLINE=1 were hardcoded in ENV, but
model pre-warm is best-effort. If pre-warm silently fails during docker
build, the image has no models and the offline flags block runtime download.

Fix: Move offline flag enforcement to entrypoint.sh with a conditional
check — only enforce offline mode if the HF cache directory is non-empty.
Otherwise unset the vars to allow runtime download.

Adds contract test test_dockerfile_offline_mode_contract.py verifying
the policy (no hardcoded offline ENV, entrypoint has conditional logic).

Verified: hf_offline grep >0, pytest tests/contract/ 2 passed, make test-core 152 passed.

* docs: fix README Honest Current State — Langfuse shims are no-op, not gone

card_ef037d6f10de — README doc-lie fix

README:151 falsely claimed '@observe decorators are gone'. Reality:
- Langfuse SDK is gone (no from langfuse imports) ✓
- @observe decorators REMAIN as no-op shims (src.observability /
  telegram_bot.observability) — consistent with AGENTS.md

Rewrites the Langfuse blurb to match AGENTS.md accurately.

Verified: grep 'decorators are gone' → 0 hits, grep 'no-op' → hit found,
make test-core → 152 passed.

* test(bge-m3): add unit tests for _lexical_weights_to_qdrant_sparse non-passthrough branch

card_a8456a6e7190

* fix(bge-m3): tech debt cleanup U9/U12/U14

card_20ee99b1cb15

U14: delete services/bge-m3-api/requirements.txt
U12: hide internal errors from HTTPException 500 body
U9:  BGEM3SyncClient context manager support

* fix(bge-m3): retry sync client on 429/503 (card_7cc460feaec0)

The BGE-M3 sidecar is a single-worker CPU-ONNX process that returns
429/503 under load. Two asymmetries left the client failing without retry:

- bge_retry had retry_on_http_status=False -> only transport errors/timeouts
  were retried, never 5xx/429.
- BGEM3SyncClient methods had no @bge_retry at all (async path had it).

Fix:
- Flip bge_retry to retry_on_http_status=True (429/503/5xx already in the
  shared RETRYABLE_HTTP_STATUS_CODES set).
- Decorate the 4 BGEM3SyncClient encode_* methods with @bge_retry.
- Add retry-policy tests (sync 503/429 retry+succeed, exhaust+reraise,
  no-retry on 400, async 503); mark the module no_services and zero
  tenacity back-off in an autouse fixture so the gate runs them fast.

* fix(bge-m3): carry partial_failures through client, skip poison vectors on ingest (card_06c91625a24c)

* fix(ingestion): add poison-skip test + defensive partial_failures key access

- Add test_poison_index_skipped_and_warning_logged in TestUpsertChunksSyncBehavior:
  feeds a 2-chunk batch where encode_hybrid returns partial_failures=[{index:1}],
  asserts exactly 1 PointStruct is built (index 0), verifies vectors come from
  index 0, and asserts a warning was logged.
- Change poison_indices set-build to guard against malformed server responses:
  {pf['index'] for pf in ... if 'index' in pf}

Resolves review blocker B1 for card_06c91625a24c.

* Remove OpenTelemetry fully (card_81add5ba4a66)

- bge-m3-api: drop FastAPIInstrumentor / LoggingInstrumentor activation +
  propagate/attach/detach trace-context middleware; remove opentelemetry-api
  and opentelemetry-instrumentation-* deps + constraint pins; relock (0 otel).
- src/observability: delete dead bootstrap.py (disable_otel_exporter,
  is_endpoint_reachable — 0 production callers); drop the OTEL shim imports +
  __all__ entries from __init__.py.
- telegram_bot: delete observability_bootstrap.py (pure OTEL re-export shim).
- compose.yml: drop OTEL_SERVICE_NAME / OTEL_PROPAGATORS from bot service.
- logging_config.py: kept trace-context fields (observability via structured logs).
- tests: delete obsolete OTEL contracts/tests (propagators contract,
  TestOtelServiceNameDefaults, googleapis-otel-transitive, bge-m3 langfuse/otel
  env, observability OTEL-helper tests); drop vestigial _is_endpoint_reachable
  patches in test_main / test_bot_log_triage. Guardians kept
  (test_no_monolith_otel_auto_instrumentation, product_events no-otel import).

* Remove _update_current_span no-op stubs and call-sites (#2844)

- src/runtime/services/rag_core.py: remove stub def + 15 call-sites
- src/runtime/services/query_preprocessor.py: remove stub def + 4 call-sites
- telegram_bot/services/rag/rag_core.py: remove stub def + call-sites,
  drop now-unused contextlib import and get_client import
- telegram_bot/services/rag/query_preprocessor.py: remove stub def
- src/observability/__init__.py: update stale docstring reference

Closes card_1a615feb22f1 (P25 Remove Langfuse residue)

* test: delete stale TestRagCoreObserveContract class (#2162 contract, langfuse removed #2844)

* feat: assert ONNX output name contract in ONNXEmbeddingModel.__init__

Add EXPECTED_OUTPUTS class constant and assertion to guard against
silent dense↔colbert transposition if model is re-exported with
different output order.

Add three output-contract tests (correct names OK, wrong order raises,
wrong names raises) and one get_model FileNotFoundError test.

card_40c2e1606e62

* chore(observability): remove @observe/traced_pipeline/propagate_attributes no-op call-sites (card_40feeaff139c)

* feat: add test-bge-extras lane for BGE-M3 FastAPI endpoint tests (card_17e07f2a213e)

- Add bge-extras optional dep to pyproject.toml (fastapi, httpx)
- Add test-bge-extras Makefile target (uv sync --extra bge-extras, pytest both files)
- Replace pytest.importorskip with explicit pytest.skip + actionable message
- Fix _import_lexical_weights_fn: save/restore sys.modules['app'] to avoid
  corrupting module-scoped bge_app fixture (was causing TestRunEncode failures)
- Remove stale test_traceparent_header_is_extracted (OTel removed, card_81add5ba4a66)

make test-bge-extras: 32 passed, exit 0
make test-core: 152 passed, exit 0

* test: fix TestRunEncode isolation — patch get_model via autouse fixture

* fix: add ENCODE_MAX_ITEMS limit + remove dead batch_size from EncodeRequest

- Add ENCODE_MAX_ITEMS = 64 constant to app.py
- Add Field(max_length=ENCODE_MAX_ITEMS) validator to EncodeRequest.texts
  (mirrors RerankRequest.documents pattern with le/max_length validator)
- Remove dead batch_size parameter from EncodeRequest (ignored by single
  session.run; now uses settings.BATCH_SIZE directly in _run_encode)
- Add TestEncodeMaxItems: over-limit → 422, at-limit → 200

Closes card_7ac0f91b8132

* remove propagate_attributes and get_client call-sites from 9 remaining files (card_76e6a84b83f5)

* docs: reconcile module READMEs with code + add 8 missing ones

Doc-gardening sweep (34 docs):
- fix drifted claims/paths/examples: src/runtime scaffold->built engine,
  src/retrieval nonexistent classes, docker/ phantom subdirs, tests/ dead
  langfuse/traces gate, api delegation target, telegram_bot dir/service paths
- repair broken CONTRIBUTING -> docs/ONBOARDING link; honest 'planned docs' note
- add code-derived READMEs: src/{services,observability,adapters},
  telegram_bot/{pipeline,lifecycle,graph(legacy facade),observability,preflight}
- link new docs into docs/README hub

Verified: markdown-links contract green, 0 broken links.
…s to sync

The async upsert_chunks and delete_file methods diverged from the sync path,
lacking the text-size guard and poison/empty-vector skip. Their only caller
was the E2E test harness, making deletion the cleaner fix.

Changes:
- Delete async upsert_chunks, delete_file and their embed helpers
  (_embed_documents_local, _embed_sparse, _embed_colbert) from qdrant_writer.py
- Remove unused threading import, _dense_semaphore, and try_update_ingestion_trace
- Repoint live_harness.py index_fixture_documents to upsert_chunks_sync
  via asyncio.to_thread (harness is async)
- Update contract tests: remove upsert_chunks from parametrize (deleted)
- Remove test_embed_colbert tests (tested deleted async-only helper)
- Clean up test helper (remove _dense_semaphore injection)

Fixes card_9aef142cdfb8
…ument /rerank no live client (P24 card_a6f95c6865a8)
…(nurturing_scheduler + session_summary_worker + tests)
@yastman
yastman merged commit c04d7db into dev Jul 23, 2026
10 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant