Skip to content

feat: add physical action validator with domain-specific rules (GSSoC)#274

Open
purvask2006-collab wants to merge 1 commit into
sahoo-tech:mainfrom
purvask2006-collab:feat/physical-action-validator
Open

feat: add physical action validator with domain-specific rules (GSSoC)#274
purvask2006-collab wants to merge 1 commit into
sahoo-tech:mainfrom
purvask2006-collab:feat/physical-action-validator

Conversation

@purvask2006-collab

Copy link
Copy Markdown

🔗 Related Issue

Closes #148


📝 Summary of Changes

Implements the physical action validation system for the Execra physical domain.

Before Execra marks a step complete, ActionValidator.validate() inspects the
current CV observations (object detections, hand pose results, optional depth map)
and returns a ValidationResult describing whether the user performed the action
correctly and safely. If is_valid=False, the caller should dispatch a corrective
GuidanceInstruction before advancing the step.

Three domain validators are implemented, each with spatial safety rules:

  • Cooking — knife must be near the cutting board and away from the user's body
  • Hardware — screwdriver must be oriented vertically (not horizontally); optional depth-map alignment check against the workpiece
  • Form filling — pen must be in the dominant hand, near paper, and not near a phone/tablet screen

Domain classification is automatic from the step's natural-language description using keyword matching.


🔍 Type of Change

  • 🐛 Bug fix
  • ✨ New feature (non-breaking change that adds functionality)
  • 💥 Breaking change
  • 📖 Documentation update
  • 🧪 Test addition or improvement
  • 🧹 Refactor / Code cleanup
  • ⚙️ Build / CI / Chore

🧪 How Was This Tested?

55 unit tests written in tests/test_action_validator.py using fully synthetic
detection data — no camera, CV runtime, or external services required.

Tests cover:

  • Happy path (valid action) for all three domains
  • Every failure mode: missing tool, wrong orientation, proximity violations, wrong surface
  • Label variants: cleaver, chopping board, pencil, marker, drill
  • Depth-map alignment (correct and misaligned)
  • ValidationResult.merge() combining two results
  • ActionValidator._classify_step() with 15+ step descriptions
python -m pytest tests/test_action_validator.py -v --noconftest

Test environment:

  • OS: Windows 11
  • Python version: 3.14.5

📸 Screenshots / Recordings

Not applicable — no UI changes. Coverage report in test output above.


✅ Pre-Submission Checklist

  • My branch is up to date with upstream/main
  • My code follows the project's coding standards (type hints and docstrings on all public functions)
  • I have written/updated tests for my changes
  • All new tests pass (55 passed, 98% coverage)
  • I have added docstrings and type hints to all new public functions
  • I have updated relevant documentation (README) — not required for this module; no user-facing config added
  • My PR title follows the Conventional Commits format
  • I have not committed any .env files, secrets, or model weights
  • This PR addresses only the scope of the linked issue

💬 Additional Notes for Reviewer

  • All spatial thresholds (proximity distances, aspect ratios, depth deltas) are defined as class-level constants so they can be tuned without touching logic.
  • The depth-map check in the hardware validator is opt-in — passing depth_map=None skips it gracefully, so the validator works even without a depth sensor.
  • ValidationResult.merge() is provided for callers that need to combine results from multiple validators (AND semantics: both must be valid).
  • The four uncovered lines (98% → lines 63, 145, 154, 158) are defensive None-guard branches that require a malformed depth map to trigger; not worth a test for near-zero real-world probability.

- ActionValidator.validate(step, detections, hand_results, depth_map)
- ValidationResult dataclass (is_valid, confidence, issues, corrections)
- Cooking: knife/board proximity + knife-body safety check
- Hardware: screwdriver vertical orientation + depth alignment
- Form: pen in dominant hand, near paper, not phone
- 55 unit tests, all passing (98% coverage on action_validator.py)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Build an Action Validator for Physical Task Safety

1 participant