Should fire events instead of using passive model

[AshleyScirra]

(This is based on an email sent to public-webapps)

The Gamepad API is the only web platform input API that relies on passively polling the input state. Gamepads should fire events like "gamepadbuttondown", "gamepadbuttonup" and "gamepadaxismove". This would have the following benefits:

• consistent with other input APIs (e.g. "mousedown", "mouseup", "mousemove" / "touchstart", "touchmove", "touchend" / "pointerdown", "pointermove", "pointerup")
• easier to program for: if you want a "gamepadbuttondown" event, you have to implement it yourself by polling the state, keeping the previous polled state, and comparing the differences looking for a previously up but currently down state and then run your handler
• easily integrates with browser's existing "user gesture" model for sensitive features like opening popup windows, copying to the clipboard, and playing audio or video on mobile. Since gamepads do not fire events, currently it is not possible to use any "user gesture" features with gamepad input alone in any existing browsers.

[pyalot]

Note that using events exlusively without control over the event loop has severe data handling drawbacks.

If events are introduced, they should be issued at a controlled point in time (when polling) which clears the entire event queue up to that point, and they should not substitute "passive" (it's not really passive, it's active polling) querying of values, particularly not axis values.

[luser]

/cc @sgraham @toji

This has been discussed on public-webapps in the past:
https://lists.w3.org/Archives/Public/public-webapps/2012AprJun/0469.html

and it's on my list of features for a next version of the spec:
https://www.w3.org/wiki/Webapps/GamepadFeatures

Firefox includes non-standard events that you can enable by flipping a pref (dom.gamepad.non_standard_events.enabled), you'll get GamepadButton{Down,Up} and GamepadAxisMove events.

I think spec'ing the button events would be fairly simple and uncontroversial and solves the most important use case--missing button presses that happen between polls.

Events for axis values (or analog button values) are harder to get right. They're likely to be spammy and not incredibly useful--most consumers just want to know the current value of an axis. It would be nice to have some sort of event for this so that consumers aren't forced to constantly poll. Maybe we could write some spec text such that we'd fire a "datachanged" event whenever anything changes, but the browser can roll up a series of them into a single event, like if you had:

<button 1 pressed>
<axis 1 moved>
<axis 2 moved>
<axis 1 moved>
<axis 2 moved>
<button 1 released>
<axis 1 moved>

the browser could compress this to:

ButtonPress {button: 1}
DataChanged {axes: [1,2]}
ButtonRelease {button: 1}
DataChanged {axes: [1]}

[tkodw]

When thinking about what sort of event api would be useful, I think it is helpful to divide the use cases into categories based on how they expect input.

Reactive: input is a cue for something to start happening. Examples: video player, slideshow viewer

In this category we mostly get web apps that want to be able to interact or navigate occasionally with a gamepad. These apps want events because the current api forces them to poll continuously even though they rarely process input.

For the majority of cases, reactive apps only want to know about:

1. Button fully pressed
2. Button no longer fully pressed
3. Axis reaches +1.0 or -1.0
4. Axis no longer at +1.0 or -1.0

Tracking slight changes in joystick inputs does not make sense in events, because it produces many more events than a reactive app can reasonably respond to. An additional catch-all event GamepadChanged could be used if necessary but it might need both rate limiting and noise tolerance to be useful.

Animated: input is processed continuously and at regular intervals. Examples: games, simulations

An event driven api could help these types of applications catch missed button presses and improve timing, but the reality is that events are a poor fit for these types of apps. @pyalot pointed this problem out earlier.

Note that using events exclusively without control over the event loop has severe data handling drawbacks.

These classes of applications aren't idle, and therefore don't need an event to fire to wake them up. The need to listen for and store events to be used for the next game/simulation loop is a nuisance.

What these apps ideally want is a way to get all available information about the gamepad between the current game/simulation loop and the previous one. Specifically that means providing a function that can retrieve for snapshots of gamepad state or gamepad state changes over a specified time interval.

An example api change that can accomplish this is to add an optional parameter of type DOMHighResTimeStamp on getGamepads, which will cause each gamepad to instead be returned as a sequence of all gamepad states after the provided time instead of a single gamepad.

Providing all gamepad state updates has important benefits:

1. Timing accuracy no longer limited by polling rate, and by extension, frame rate.
2. Inputs will not be missed or have inaccurate timing if the browser freezes briefly.
3. Can consistently determine the order and combinations buttons are pressed in.

[pyalot]

A footnote on events for axes. A common failure mode of this kind of thing is when you have say an X and Y axis and you're plotting say a position on screen from them. So the X axis event arrives first, you draw a line there, then the Y axis event arrives, and you draw a line there. Now you've got a staircase instead of a continous line.

Although getting the whole state (as is presently the case) avoids that kind of issue (and it is among the reasons that shouldn't be dismantled), there is also sometimes a case to be made for having the ability to collect the events for axes and then aggregate them yourself (for instance instead of just using the last known, you can do a weighted average). For very twitchy games, this can provide players with a more satisfactory response.

[tkodw]

A common failure mode of this kind of thing is when you have say an X and Y axis and you're plotting say a position on screen from them. So the X axis event arrives first, you draw a line there, then the Y axis event arrives, and you draw a line there. Now you've got a staircase instead of a continous line.

This problem is easily avoidable if you can get multiple events at once. In your example you get an X axis but have no way of checking if the Y axis event has already fired and is waiting to be processed.

• If you implement an event that provides an array of changes, you always see the events together.
• If you implement an event that provides only a timestamp of when changes occured and allow getGamepads to request states over a time range, you will see both axis changes within the states.
• As long as the api provides a mechanism to deal with this problem, it may be okay to leave in some events that suffer from this issue if it makes the implementation of gamepad support easier for simple use cases.

Although getting the whole state (as is presently the case) avoids that kind of issue

Take the simple case that a controller has a directional pad, and you want to perform different actions for up, diagonal (up + left) , and right. When polling the complete state, just left or just right can (and frequently will) show up in a poll before both show up together unless the are pressed together at exactly the same time. You can't avoid doing the left or right action before diagonal unless you buffer inputs for some arbitrary interval before performing the left or right actions. There are a lot of similar issues like this that need to be tackled by the application and are unlikely to be solved directly by the api.

there is also sometimes a case to be made for having the ability to collect the events for axes and then aggregate them yourself (for instance instead of just using the last known, you can do a weighted average). For very twitchy games, this can provide players with a more satisfactory response.

I am very much in favor of games aggregating events or state provided from the API, and believe this is the ideal way to handle input. The question is how to collect the aggregated events.

1. Poll Spamming (the current way)
Call getGamepads on a timer with a short interval and collect the gamepad states.

• It already works and is a simple and understandable solution.
• There is no way to know what polling rate the hardware is using or sync to it, so we are frequently undersampling (increased latency, missed inputs) or oversampling (wasted timer calls).
• If the main javascript context stalls for any reason inputs are missed and timing loses accuracy.
• The polling loop is constantly active even if the controller is idle.

note: Timing is much worse than accessing hardware directly in Chrome due to a counter being provided instead of timestamps and internally buffering with a long polling interval. (In my test case 125hz hardware is polled and buffered at 60hz)

2. Event Listeners (same as mouse and keyboard)
Api would provide events to subscribe to that fire for every possible state change on connected gamepads. The application subscribes to all event types with functions that collect the events.

• Fits in better with existing input handling of mouse and keyboard events.
• When the controller is idle, no events need to be handled.
• High update rate hardware can result in quite a lot of event handling spam.
• No missed inputs or inaccurate timing.

3. Query State/Events (my suggested method)
Have the browser internally poll or accept input from all active controllers with as much accuracy as possible and buffer this information over a short period. Expose an API that takes a time range and returns an collection of states or events from gamepads over that time range.

• No input events or timers in javascript. (better efficiency?)
• No missed inputs or inaccurate timing.

A minimal API to satisfy all possible use cases:

• Add one event for when any gamepads have change state after a specified time.
• Add an optional start time range to getGamepads so it can return multiple states.

[jakearchibald]

Could a model like https://developer.mozilla.org/en-US/docs/Web/API/PointerEvent/getCoalescedEvents help here?

[AshleyScirra]

I think there still need to be events to be able to work with user gesture requirements. For example, you still cannot start any media playback with a gamepad on Android, because you never get an input event (crbug.com/454849). Something like getCoalescedEvents doesn't seem to change that.

[jakearchibald]

There'd still be a root event to get the coalesced events from, and that would be a gesture event.

[AshleyScirra]

Oh, right. Yeah, it makes sense to use a similar mechanism to pointer events for better-than-rAF resolution.