per-key led feature

[Retic1337]

creating this to start work on the per key led lighting.

[joelspadin]

I think the first thing to do is to get a list of use cases for per key lighting. Here are a few I can think of:

1. Set different solid colors per key (e.g. mods one color and everything else a second color).
2. Dim/disable lighting for keys that have no function in the current layer.
3. Status LEDs (caps lock, num lock)
4. Static animations (e.g. color wave across keyboard)
5. Dynamic animations (e.g. ripples from pressed keys)

Other considerations:

It would be nice to have one system that manages all LEDs and then have mappings on top of that for per key LEDs, underglow LEDs, and standalone status LEDs. Having separate systems for each of those would likely lead to inconsistencies and make it difficult to do things like creating an animation that syncs between key and underglow LEDs.

Just like with the existing kscan and composite kscan drivers, we should support multiple LED drivers (e.g. direct GPIO, daisy chained ws2818, Lumissil matrix driver) and combinations of them controlling different subsets of LEDs.

Storing colors as red, green, and blue bytes is probably the format that works best across all LED drivers. We can still convert to/from HSV as needed for things like adjusting the underglow color. To support single color LEDs with the same system, the driver can simply read the red channel and adjust a PWM or use an on/off threshold.

We need to somehow support wireless splits. Central could send per LED color changes to the peripheral, but that probably won't perform well with fast animations. The peripheral likely needs to run the same LED control code as the central and just get high level commands like hue/brightness changes from central.

[Retic1337]

for #1 and #2 we will need to import the keymap into the driver to determine colors.
#3 will require reading status of caps, num, etc..
#4 i think would have to be a map of the leds position x and y relative to their placement on the pcb. so esc on a tkl would be 0,255 and right arrow would be 255,0.
#5 would be easy once #4 is done.
Currently have a background color working with per key color changing based upon keypress. Where are we going to want to be creating led maps and defining colors etc, board.keymap? or in the device-tree?

[joelspadin]

for #1 and #2 we will need to import the keymap into the driver to determine colors.
#3 will require reading status of caps, num, etc..

I think we should structure things so that the low-level LED drivers don't have any knowledge of the keymaps. Instead, something built on top of that (not sure if it would also be considered a driver or just application code) should be deciding what color each LED should be and sending commands to the LED driver(s). It is this higher level code that would need to know about the keymap and lock statuses.

One idea I had for this was to have a base LED map/animation that defines the color of each LED and then stack "transforms" on top of that which can optionally modify the colors before they are sent to the LED driver(s). For example, a status indicator could be a transform that overrides the color of one specific key based on the lock status. Dimming keys based on layer could be another transform that reads the keymap and overrides LEDs accordingly.

Pseudocode for how this might work:

# User-configurable part
def base_animation(colors):
    for n in range(len(colors)):
        colors[n] = LED_MAP[n]

def caps_transform(colors):
    if is_caps_on:
        colors[CAPS_INDEX] = (0, 255, 0)

def layer_highlight_transform(colors):
    for n in range(len(colors))
        if not current_layer_has_function(n):
            colors[n] = (0, 0, 0)

TRANSFORMS = [layer_highlight_transform, caps_transform]

# This part happens the same way no matter how things are configured
colors = [(0, 0, 0)] * LED_COUNT

def update_colors():
    base_animation(colors)
 
    for transform in TRANSFORMS:
        transform(colors)

    send_to_driver(colors)

Both the base animation and transforms would also need a way to trigger a color update (e.g. periodic for an animation, on layer change for layer-based dimming, etc.).

Where are we going to want to be creating led maps and defining colors etc, board.keymap? or in the device-tree?

The organization that makes the most sense to me is to define all the driver and hardware configuration in the board DTS/shield overlay and then put the LED color maps and any other user configuration (e.g. enable/disable dimming keys based on layer, etc.) in the keymap.

[KingCoinless]

No notes on how to write this yet, but I do have a recommendation for a specific kind of reactive per-key effect. On my Kemove 64/66 keyboard, there is a lighting mode which, when pressing any key, reacts on that key in a random color and disappears. Fn + Up and Down Arrow cause the brightness levels to raise or lower. Fn + Left and Right Arrow cause the effect's dissipation rate to speed up or slow down. It might be interesting to offer something like this in ZMK, or perhaps even a lighting mode which reacts around a specific color (for example, blue, or tints of blue). The color (or tints of) could be adjusted through the 'RGB_HUE' and 'RGB_HUD' key commands like normal. We could give the light effects a dissipation rate range of maybe 200ms - 2s, with nine steps of 200ms in between (200 -> 400 - > 600, etc). It's a very pretty effect, and one that I think would be great to include here.

[ice9js]

I've been thinking about the kind of data structures that make sense to support this.
Specifically how @joelspadin already pointed out, it'd be nice to have the animation 'engine' support different LED drivers and combinations of them. I'd even expand on those guidelines:

• One IC should be able to support multiple logic groups of the LEDs as well. Like, per-key-rgb AND underglow to give an extreme example. With the new ISSI chips, one or at most two would certainly be enough to support all of your lighting needs and so it'd be cool to take advantage instead of needing to use and support additional hardware.
• Make it possible to have different LEDs follow different animations even if the behavior is the same. One might have a different effect on the bottom and on the top, or define multiple light strips each responsible for its own thing. Effectively, fine grained control.
• Eventually all of this should be reconfigurable on the fly without having to flash the board.
• _It might be interesting to expose an API for other parts of the firmware to control the animation/parameters for a given area. Like, using your underglow LEDs as a charging indicator or supporting power on/off animations.

Let's assume we'll be using Zephyr's led_strip API as a common starting point.
A single GPIO-driven LED can be represented as a strip with length of 1, a driver would be simple enough. Similarly, LED matrices can be transposed into a 1d array and vice versa.
Thus, we will need a pixels[] array:

This would be a 1D array containing the values for all LEDs on the board.
In the example above, LEDs 0...m-1 are driven by one IC while m...n are driven by another. Organizing it this way allows to pass the appropriate pointers straight to led_strip.update_rgb() saving memory and processing overhead.

That was the easy part. Now, let's add a second pixel_meta[0;n] array in parallel. This array would contain all the necessary settings for each pixel. For the purpose of illustration, I'm going to portray it as an array of structs. However it could just be multiple arrays containing individual props if getting the right device tree values into the right places proves tricky.

The meta would look something like this: (pseudocode)

meta[0...n] {
    animation:
        Pointer to the animation function

    key:
        the key identifier or NULL if it's not a key LED

    position:
        x: [0;255]
        y: [0;255]

    flags: ?
}

Let's take it step by step:

• Animations would be functions. Each could potentially live in its own file and maintain its own state necessary to keep it going depending on what kind of animation it might be - not interfering with any other ones that might be going on. It would be called with the pixel meta and expected to return the pixel values for the new frame.
• Key marks what key the LED belongs to. Which can be used for overriding the default animation. In which case we might need another alt_animation property or something?
• Let's assume your keyboard PCB is the reference plane, in which case these would be the relative x and y coordinates. Keyboards aren't squares though so potentially, somewhere, an x/y ratio needs to exist as well to be references by the animations if necessary.
• Any additional flags we might need I guess? Could be useful for dedicated indicator LEDs or to account for any edge cases.

So this is more or less my angle on it, I'd be curious to hear what others think before I start making inroads towards an actual implementation.

One catch I can already think of would be that in this state, it'd not allow for a dynamic animation from one key to override the animation on another key unless it was already assigned to the same animation.
Or we could force all LEDs through all active _key pixels' animations and merge the result. Flags would probably be a good candidate to define the merge behavior on a per-pixel basis. In case, say, you have some indicator LEDs in the middle and don't want them to be affected by that.

[joelspadin]

I was thinking that rather than having each key hold a pointer to an animation function, you could implement something like an animation driver API and then have each animation be an instance of a driver that implements it. Then you could set up the animations like layers and enable/disable each one individually. Every time you want to update the LEDs, you'd loop through the enabled animations and just call one update function on each, giving it a pixel buffer (and maybe also a list of which pixels it's assigned to update?).

Then you could easily implement things like an "animation" that just takes the output from the previous animation and zeroes out every key that doesn't have a function on the current layer, or one that overrides just the caps key based on the indicator state. As a bonus, using the driver API means each instance of the driver maintains its own state, so you could make much more complex animations like one that listens to key events and lights up the keys you press. I think that would eliminate any need for flags or an alt_animation property. Instead of flagging a key as being an indicator, you'd just put a devicetree property on the indicator animation to tell it that key 20 is caps lock.

Animations should also be able to tell the animation system when they need to update. Say you have a solid color animation as the base, an animation that changes based on the layer, and one for indicator LEDs, then you only need to refresh when the layer or indicator state changes. The animation drivers could listen for ZMK events and then call "request animation update" function which queues up a work item to do an animation update.

For animations that smoothly animate instead of just updating on specific events, you wouldn't want to have multiple animations asking the animation system to update on their own timers. If you had two animations requesting updates at different frequencies/phases, you'd update far more often than you need to. To handle this, you could have a global periodic timer, and have the animations' update functions return a Boolean. If any animation returns true, then the global timer will schedule another update. If not, the timer will stop and wait for an animation to manually trigger an update. (Would also let us do other optimizations like throttling the update rate when on battery if that ends up being a big power user.)