ngary/keyboard teleop

[hello-neranti]

Description

This pull request adds keyboard control functionality to the web interface. There is a new customizable component called “Keyboard Control” that, when added to the layout, enables keyboard inputs for controlling the stretch robot.

Keyboard Controls

Base Mode
W - Base Forward
A - Base Rotate Left
S - Base Backward
D - Base Rotate Right

Arm Mode
W - Arm Lift
A - Arm Retract
S - Arm Lower
D - Arm Extend

Wrist Mode
Q - Wrist Roll Left
E - Wrist Roll Right
W - Wrist Pitch Up
A - Wrist Rotate In
S - Wrist Pitch Down
D - Wrist Rotate Out

Gripper
J - Close Gripper
K - Open Gripper

Camera
Up Arrow - Camera Tilt Up
Left Arrow - Camera Pan Left
Down Arrow - Camera Tilt Down
Right Arrow - Camera Pan Right

Testing procedure

• Pull on ngary/keyboard_teleop and launch the web interface
• Click on “Keyboard Control” in the customization panel and add it to the layout
• FOR ALL ACTION MODES - Step-action, Press-and-hold, click-click:
  • Test the camera controls (arrow keys) making sure all directions work
  • Test the gripper controls (J - close gripper, K - open gripper)
  • Test the button modes (1,2,3) making sure they switch modes, the text below should update to show the current mode
    • 1 = base, 2 = wrist, 3 = arm
  • Test the controls (WASD) for each mode, making sure all inputs work
    • The keyboard controls will be QEWASD for wrist mode
    • Test that keyboard buttons work regardless of letter case
• All keyboard buttons on the component should be highlighted when active, in collision, or at their joint limit
• Hold a key input the press the toggle button labeled “KEYS ON” to OFF, the input should stop
  • Then check that the keyboard input works again when the button is pressed back to ON
  • If you hold a key then toggle keys ON, the input will start
• Click into customization mode
  • Test that the toggle button is no longer clickable
  • Test that the keyboard component can be selected, moved to any of the drop zones, and deleted from the layout
• While holding down a key, disconnect the keyboard. The input should stop running
• Add the keyboard component and save the layout. When you load the layout, the toggle but should be ON regardless of its state when it was saved
• Try holding a key input and pressing another button on the button pad
  • The button pad button should override the key press
  • The button for the clicked key should still be highlighted, but not inputting an action
  • If you press more than one key, the last one will get inputted
• Try holding a key input and pressing another speed
  • Current input should keep the original speed of when the key was pressed

🪲 Bugs - Test Cases (Checked means bugs are fixed)

• The keyboard input toggle button doesn't stop any running input commands when clicked off

• Switch to Press-and-Hold or Click-Click mode

  • Hold/press any key input (I use a base input) and while holding down the key, press the toggle key input button to "OFF"
    • The input will continue to run even as you release the key
  • If you hold any key then switch to customizing mode, the same will happen
  • To stop this, you must toggle key controls back on then press the key again

• The toggle button is still clickable while in customizing state, but not while component is selected.

• When in customizing mode:

  • Press any key. Stretch will take the first key input and run that indefinitely
    • No other key inputs after that will be processed

• When leaving the customizing mode, no key inputs are processed regardless of what state the toggle is in

  • Toggling the keys off then back on will fix this issue

Before opening a pull request

From the top-level of this repository, run:

• pre-commit run --all-files

To merge

• Squash & Merge