Skip to content

Add Dexterous Triage v17 real-world transfer evidence#463

Open
Brian5216 wants to merge 13 commits into
Faraday-Future-AI:mainfrom
Brian5216:codex/dexterous-triage-v17-realworld-evidence
Open

Add Dexterous Triage v17 real-world transfer evidence#463
Brian5216 wants to merge 13 commits into
Faraday-Future-AI:mainfrom
Brian5216:codex/dexterous-triage-v17-realworld-evidence

Conversation

@Brian5216

Copy link
Copy Markdown

Registration UUID: 9e089c2c-c100-4dad-8652-052d998ce23e

Project Summary

  • Project name: Dexterous Triage Lab
  • Robot platform: Procedural MuJoCo five-finger dexterous hand with 22 controllable channels, thumb opposition, wrist, gantry, sensors, actuators, and task objects.
  • Task goal: Complete a disaster-response medication triage sequence: scan, approach, grasp, uncap, place into sterile pod, confirm with button press, recover from slip, and export labels.
  • Technical approach: Deterministic imitation-style stage prior plus closed-loop residual policy using visual-servo/contact/slip feedback, with MJCF joints, actuators, free bodies, collision geometry, frame sensors, touch sensors, generated narrative video overlays, SRT subtitles, trajectory JSON, five-finger contact timeline, policy card, fixed-seed stress evaluation, real-world transfer evidence, and self-audit report.
  • Core features: Five-finger grasp, in-hand cap rotation, fragile-object handling, audit button confirmation, slip recovery, raw-vs-corrected visual-servo metrics, per-sample residual actions, controller confidence, machine-readable contact timeline, 32-rollout baseline-vs-residual evaluation, physical bench-test pass/fail thresholds, and data-collection export.
  • Highlights: Designed to cover all eight official scoring dimensions, especially MuJoCo depth, task design, control, dexterous manipulation, presentation, and engineering quality.
  • v17 highlights: post-residual median visual-servo error is 0.00672 m, down 68.61% from raw error; demo video uses concise subtitles plus dynamic route trails, residual arrows, scan beams, cap-rotation arcs, slip-recovery ripples, and explicit key-moment labels; the contact timeline exposes five-finger active contact, stable hold, and recovery-window evidence; the real-world transfer package maps the sim evidence to physical vial, slip impulse, cap-torque, and audit-button bench tests.
  • Scope notes: High-level plan is deterministic for reproducible judging; residual control is closed-loop, logged, and stress-tested against a no-residual baseline.
  • Future improvements: Learned visuomotor policy weights, randomized task generation, additional failure modes, and policy training from exported trajectories.

How to Run

python3 -m pip install -r requirements.txt
python3 submissions/dexterous_triage/run_dexterous_triage.py

Quick smoke test:

python3 submissions/dexterous_triage/run_dexterous_triage.py --quick

Demo Video

  • Demo video is included in the submission folder: submissions/dexterous_triage/artifacts/dexterous_triage_demo.mp4
  • Demo video was generated by the submitted code.
  • Closed-loop policy card is included: submissions/dexterous_triage/artifacts/dexterous_triage_policy_card.json
  • Stress evaluation is included: submissions/dexterous_triage/artifacts/dexterous_triage_eval.json
  • Narration subtitles are included: submissions/dexterous_triage/artifacts/dexterous_triage_narration.srt
  • Contact timeline is included: submissions/dexterous_triage/artifacts/dexterous_triage_contact_timeline.json
  • Real-world transfer evidence is included: submissions/dexterous_triage/artifacts/dexterous_triage_real_world_transfer.json
  • Real-world physical test plan is included: submissions/dexterous_triage/REAL_WORLD_TEST_PLAN.md
  • Judge brief is included: submissions/dexterous_triage/JUDGE_BRIEF.md
  • Rubric scorecard is included: submissions/dexterous_triage/rubric_scorecard.json
  • Machine-readable manifest is included: submissions/dexterous_triage/submission_manifest.json

Checklist

  • submissions/dexterous_triage/registration.json contains my UUID
  • PR description contains the same UUID
  • Code runs from documented instructions
  • Demo video was generated by the submitted code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants