Skip to content

Add artifact validator layer - #48

Open
Permea-lab-admin wants to merge 1 commit into
mainfrom
p-core-038-artifact-validator-layer
Open

Add artifact validator layer#48
Permea-lab-admin wants to merge 1 commit into
mainfrom
p-core-038-artifact-validator-layer

Conversation

@Permea-lab-admin

Copy link
Copy Markdown
Collaborator

Summary

This PR adds the first public artifact validator layer for Permea Core.

It adds:

  • scripts/permea_check.py for checking public example artifacts
  • a lightweight permea_core.validator module
  • validation documentation at docs/specs/VALIDATION.md
  • a public technical report for P-CORE-038
  • tests for validator CLI behavior, family coverage, report structure, boundary checks, and documentation links
  • public documentation links from README, Quickstart, Evaluation, Reproducibility, specs, and the generated evidence surface

Scope

This PR lets researchers and developers check current public example artifact families against public Permea artifact standards.

Supported families:

  • dataset cards
  • benchmark cards
  • evidence cards
  • run manifests
  • output packages

The validator is intentionally lightweight and dependency-minimal. It checks file presence, known families, required schema concepts, claim-boundary language, public/private boundary hygiene, credential-like signatures, and deterministic report status.

Claim boundaries

This PR keeps claims bounded to local public artifact checking and metadata/spec conformance checks.

It does not claim:

  • data has been downloaded
  • acquisition has been executed
  • redistribution rights have been confirmed
  • wet-lab validation by Permea
  • clinical efficacy
  • universal prediction
  • solved delivery
  • SOTA performance
  • model performance has been measured

Public/private boundary

The added files were reviewed to avoid private program, resource, workflow, prompt, credential, local path, or infrastructure context.

Validation

  • git diff --check
  • python3 scripts/permea_check.py --all
  • python3 scripts/permea_check.py --family dataset_card
  • python3 scripts/permea_check.py --family benchmark_card
  • python3 scripts/permea_check.py --family evidence_card
  • python3 scripts/permea_check.py --family run_manifest
  • python3 scripts/permea_check.py --family output_package
  • python3 scripts/permea_specs.py
  • python3 scripts/permea_evaluate.py
  • python3 scripts/permea_reproduce.py
  • python3 scripts/permea_validate.py
  • python3 scripts/generate_evidence_surface.py
  • python3 scripts/generate_demo_packet.py
  • python3 scripts/generate_artifact_index.py
  • python3 scripts/generate_evidence_matrix.py
  • python3 scripts/run_permea_dry_run.py
  • python3 scripts/generate_permea_artifacts.py
  • python3 scripts/validate_permea_artifacts.py
  • python3 -m pytest tests/test_artifact_validator.py tests/test_artifact_specs.py tests/test_evaluation_bundle.py tests/test_reproducibility_bundle.py tests/test_evidence_surface.py tests/test_demo_packet.py tests/test_artifact_index.py tests/test_evidence_matrix.py tests/test_permea_dry_run.py tests/test_generate_permea_artifacts.py tests/test_validate_permea_artifacts.py
  • public/private scan passed
  • claim-boundary scan passed with explicit non-claim matches reviewed in context
  • secret scan passed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant