Deepen eval2otel contract and adapter coverage

docs/contract/eval2otel-v1.md

@@ -31,6 +31,24 @@ Every span must include:
 Provider-aware conversions should include `gen_ai.provider.name` using the
 normalized provider names exported by `normalizeProviderName`.
 
+## Attribute Registry
+
+Eval2Otel exports `ATTRIBUTE_REGISTRY`, `isRegisteredAttribute`,
+`collectUnknownAttributes`, and `assertRegisteredAttributes` from the root
+entrypoint. New adapters should use these helpers in tests to keep emitted
+attributes intentionally aligned with:
+
+- OpenTelemetry GenAI semantic convention attributes;
+- OpenLLMetry-compatible RAG quality names where no stable OTel equivalent
+  exists yet;
+- Eval2Otel-owned `evalops.*` contract, privacy, provenance, and evidence
+  attributes;
+- provider-prefixed diagnostic attributes such as `openai.*`,
+  `anthropic.*`, and `google.vertex.*`.
+
+New unregistered attributes must either be added to the registry with source and
+stability metadata or moved into an existing provider/custom namespace.
+
 ## Provenance And Evidence
 
 The optional `EvalResult.provenance` and `EvalResult.evidence` fields are for
@@ -65,6 +83,57 @@ enabled:
 These counters are part of the contract so dashboards and CI can detect privacy
 or cardinality regressions.
 
+The adversarial fixtures include redaction-to-fingerprint and oversized payload
+cases. They are intended to catch prompt injection strings, secret-like values,
+tool argument leakage, and content cap regressions before adapter changes merge.
+
+## RAG Contract Additions
+
+RAG telemetry may include retrieval inputs and derived ranking metrics:
+
+- `gen_ai.data_source.id`
+- `gen_ai.rag.query_sha256`
+- `gen_ai.rag.context_window_tokens`
+- `gen_ai.rag.context_tokens_used`
+- `gen_ai.rag.context_truncated`
+- `gen_ai.rag.chunk_size`
+- `gen_ai.rag.overlap_size`
+- `gen_ai.rag.mean_reciprocal_rank`
+- `gen_ai.rag.ndcg`
+- `gen_ai.rag.citation_coverage`
+- `gen_ai.rag.retrieval_used_ratio`
+- `gen_ai.rag.top_k_relevance_mean`
+- `gen_ai.rag.top_k_relevance_min`
+
+The raw retrieval query is not emitted. Adapters provide it as `rag.query` and
+Eval2Otel emits only the SHA-256 fingerprint.
+
+RAG chunk events may include:
+
+- `gen_ai.rag.chunk.used`
+- `gen_ai.rag.chunk.citation_id`
+- `gen_ai.rag.chunk.evidence_sha256`
+
+If explicit RAG metric values are present in `EvalResult.rag.metrics`, they win.
+Otherwise Eval2Otel derives values from the chunk list when enough information is
+available.
+
+## Framework And Provider Adapter Contract
+
+Framework adapters should populate `provenance.sourceFramework`,
+`provenance.adapter`, `provenance.adapterVersion`, and evidence hashes.
+
+Provider-native adapters should return a `ProviderConversionResult` with:
+
+- `mode`;
+- `confidence`;
+- `evalResult`, or `null` on failure;
+- structured warnings;
+- evidence containing at least `rawPayloadSha256`.
+
+Promptfoo conversion additionally emits `eval.promptfoo.*` attributes for pass
+state, score, assertion counts, failed assertion counts, and metric names.
+
 ## Operational Telemetry
 
 `Eval2Otel.processEvaluation` records eval2otel self-telemetry:
@@ -88,6 +157,7 @@ contract. Each fixture asserts:
 - contract, semantic convention, provider, and provenance attributes;
 - emitted event order and event attributes;
 - redaction, truncation, warning, and dropped-event counters.
+- semconv registry coverage for every expected fixture attribute.
 
 Any adapter or converter change that alters these outputs must update this
 document and fixture expectations in the same pull request.

docs/security/adversarial-fixtures.md

@@ -0,0 +1,44 @@
+# Adversarial Fixture Guardrails
+
+Eval2Otel conformance fixtures include adversarial privacy and cardinality cases
+so adapter changes cannot silently leak content or overwhelm spans.
+
+## Current Fixtures
+
+- `chat-redaction.json`: verifies string redaction, provenance attributes, and
+  redaction counters.
+- `tool-provenance.json`: verifies tool argument truncation and adapter evidence
+  attributes.
+- `event-cap-rag.json`: verifies event caps and dropped-event counters for RAG
+  metadata.
+- `adversarial-redaction-null.json`: verifies prompt injection strings and
+  secret-like tool arguments are removed, replaced with fingerprints, and kept
+  out of event attributes.
+- `adversarial-payload-cap.json`: verifies oversized content is capped,
+  truncation is marked, and uncapped tails are not emitted.
+
+## What These Prevent
+
+- Secret-like values appearing in `gen_ai.message.content` or
+  `gen_ai.tool.arguments`.
+- Redaction hooks returning `null` without leaving a stable content fingerprint.
+- Tool argument leakage through nested JSON serialization.
+- Unbounded message or RAG chunk events on a single span.
+- New expected fixture attributes bypassing the semantic convention registry.
+
+## Adding A Fixture
+
+1. Add a JSON fixture under `test/fixtures/conformance`.
+2. Include `config`, `evalResult`, and `expected` sections.
+3. Use `redactPattern` plus `redactMode: "null"` when the fixture should assert
+   fingerprint-only output.
+4. Add `forbiddenContent` on events for any string that must never appear.
+5. Add `absentAttributes` for content attributes that must be omitted.
+6. Run:
+
+```bash
+npm test -- --runTestsByPath test/conformance-contract.test.ts test/semconv.test.ts
+```
+
+The fixture should prove both the positive output shape and the negative leak
+condition.

docs/semconv-mapping.md

@@ -0,0 +1,80 @@
+# Semantic Convention Mapping
+
+Eval2Otel follows OpenTelemetry GenAI semantic conventions first and keeps a
+small explicit registry for attributes that are not yet stable upstream or are
+Eval2Otel-specific.
+
+## Sources
+
+Each registry entry has a source:
+
+- `otel-genai`: an OpenTelemetry GenAI or closely related OpenTelemetry
+  attribute/metric name.
+- `openllmetry-compatible`: a RAG quality name that follows common OpenLLMetry
+  style and should stay easy to query across LLM observability tools.
+- `eval2otel-extension`: an Eval2Otel-owned field for contract, provenance,
+  evidence, privacy, framework, or provider-adapter behavior.
+
+## Policy
+
+- Prefer stable OpenTelemetry GenAI names when they exist.
+- Use `evalops.*` for Eval2Otel contract and evidence fields.
+- Use `eval.*` for framework-level evaluation attributes.
+- Use provider prefixes such as `openai.*`, `anthropic.*`, `cohere.*`,
+  `aws.bedrock.*`, and `google.vertex.*` only for provider-native diagnostics.
+- Do not emit raw RAG queries or prompt content as attributes. Hash them or put
+  them behind opt-in content events.
+- Add tests with `assertRegisteredAttributes` whenever a converter emits a new
+  attribute.
+
+## RAG Names
+
+Eval2Otel currently emits these RAG attributes:
+
+- `gen_ai.data_source.id`
+- `gen_ai.rag.retrieval_method`
+- `gen_ai.rag.documents_retrieved`
+- `gen_ai.rag.documents_used`
+- `gen_ai.rag.context_precision`
+- `gen_ai.rag.context_recall`
+- `gen_ai.rag.answer_relevance`
+- `gen_ai.rag.faithfulness`
+- `gen_ai.rag.query_sha256`
+- `gen_ai.rag.context_window_tokens`
+- `gen_ai.rag.context_tokens_used`
+- `gen_ai.rag.context_truncated`
+- `gen_ai.rag.chunk_size`
+- `gen_ai.rag.overlap_size`
+- `gen_ai.rag.mean_reciprocal_rank`
+- `gen_ai.rag.ndcg`
+- `gen_ai.rag.citation_coverage`
+- `gen_ai.rag.retrieval_used_ratio`
+- `gen_ai.rag.top_k_relevance_mean`
+- `gen_ai.rag.top_k_relevance_min`
+
+Chunk events may add:
+
+- `gen_ai.rag.chunk.index`
+- `gen_ai.rag.chunk.id`
+- `gen_ai.rag.chunk.source`
+- `gen_ai.rag.chunk.relevance_score`
+- `gen_ai.rag.chunk.position`
+- `gen_ai.rag.chunk.tokens`
+- `gen_ai.rag.chunk.used`
+- `gen_ai.rag.chunk.citation_id`
+- `gen_ai.rag.chunk.evidence_sha256`
+
+## Adapter Test Pattern
+
+```ts
+import { assertRegisteredAttributes } from 'eval2otel';
+
+assertRegisteredAttributes({
+  'gen_ai.provider.name': 'openai',
+  'evalops.contract.version': 'eval2otel.v1',
+  'eval.promptfoo.score': 0.9,
+});
+```
+
+If this throws, either move the attribute into an allowed namespace or add it to
+`src/semconv.ts` with a source, signal, stability, and description.