Skip to content

#POC-4 — Reliability scoring #36

Description

@nsunku

Summary

A deterministic step that blends the three signals into the 1–100 number, assigns a confidence band, and applies hard overrides for decisive red flags. No model call — pure and fully testable.

What to build:

  • score(document_signal, authenticity_signal, consistency_signal) -> ReliabilityResult
  • Weighted blend, starting weights (tunable, recorded into the result): document plausibility 0.45, image authenticity 0.30, document–narrative consistency 0.25
  • Map blended 0–1 → 1–100
  • Override: high-confidence AI-generated OR a hard document-vs-narrative contradiction → cap at the escalate band (≤30) regardless of the blend
  • Band assignment: high 80–100, medium 55–79, low 31–54, escalate 1–30
  • Build a plain-language justification string from the driving signals — no thresholds or prompts leaked, no “approved”/“rejected” language

Acceptance criteria

  • Each band boundary covered by a unit test
  • A decisive fraud signal forces the escalate band regardless of the blend
  • Weights echoed into the result
  • Identical input always yields an identical score (determinism test)
  • Result includes a human-readable justification; no approve/reject language anywhere

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions