feat: add physical action validator with domain-specific rules (GSSoC)#274
Open
purvask2006-collab wants to merge 1 commit into
Open
feat: add physical action validator with domain-specific rules (GSSoC)#274purvask2006-collab wants to merge 1 commit into
purvask2006-collab wants to merge 1 commit into
Conversation
- ActionValidator.validate(step, detections, hand_results, depth_map) - ValidationResult dataclass (is_valid, confidence, issues, corrections) - Cooking: knife/board proximity + knife-body safety check - Hardware: screwdriver vertical orientation + depth alignment - Form: pen in dominant hand, near paper, not phone - 55 unit tests, all passing (98% coverage on action_validator.py)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🔗 Related Issue
Closes #148
📝 Summary of Changes
Implements the physical action validation system for the Execra physical domain.
Before Execra marks a step complete,
ActionValidator.validate()inspects thecurrent CV observations (object detections, hand pose results, optional depth map)
and returns a
ValidationResultdescribing whether the user performed the actioncorrectly and safely. If
is_valid=False, the caller should dispatch a correctiveGuidanceInstructionbefore advancing the step.Three domain validators are implemented, each with spatial safety rules:
Domain classification is automatic from the step's natural-language description using keyword matching.
🔍 Type of Change
🧪 How Was This Tested?
55 unit tests written in
tests/test_action_validator.pyusing fully syntheticdetection data — no camera, CV runtime, or external services required.
Tests cover:
cleaver,chopping board,pencil,marker,drillValidationResult.merge()combining two resultsActionValidator._classify_step()with 15+ step descriptionsTest environment:
📸 Screenshots / Recordings
Not applicable — no UI changes. Coverage report in test output above.
✅ Pre-Submission Checklist
upstream/main55 passed, 98% coverage).envfiles, secrets, or model weights💬 Additional Notes for Reviewer
depth_map=Noneskips it gracefully, so the validator works even without a depth sensor.ValidationResult.merge()is provided for callers that need to combine results from multiple validators (AND semantics: both must be valid).None-guard branches that require a malformed depth map to trigger; not worth a test for near-zero real-world probability.