ideas for accelerometer and other scenarios.

[venkateshpotluri]

This is a collection on a set of ideas and scenario examples related to accelerometers and other types of data. I would like to have a discussion and get some feedback to develop useful scenarios to build for.

• Magic Wand example from this sparkfun tutorial could be tweeked to build detectable gestures using a cane to control play/pause functionality. Can we use the Micro Speech example to perform sound recognition?
• this accelerometer tutorial to build a pedometer.
• an adapted version of this embedded teachable machine tutorial
• the plots in this article to look at large scale activity data.

Please comment if you have questions or other suggested ideas.

[venkateshpotluri]

Here is an update on this set. The below scenarios are more refined and I would like some feedback on the applicability of these to our design space. Note that the potential study section for each of these scenarios is still incomplete; I am working on it. Just wanted to get some scenarios for discussion in the meantime.

scenarios based on the design space.

source use cases from: the first part of this tutorial to build a pedometer

Scenario 1. user tries to understand G in their accelerometer.

1. user connects an accelerometer (Micro:Bit) to our data display.
2. User selects the option to sonify the data, and display it in a table on the display webpage.
3. user places the accelerometer facing up on a table and presses the play button.
4. The sensor data display starts playing audio corresponding to the data from the sensors, and also populates a data table on the web interface.
5. On the web interface i.e. our interactive sensor data display, user selects the 3 columns corresponding to the x,y,z values to hear if there is a non-0 data.
6. user compares the data they hear with the values in the table.

Why is this generally of value?

Sensors provide large volumes of data and are prone to noise and errors. Additionally, there is little available in terms of tools, documentation and resources that is nonvisually accessible that informs how these sensors work. nonvisually understanding this first step in the tutorial is key to understand how an accelerometer works, even outside of the context of building a pedometer.

What part of the design space does this illustrate?

reading live sensor data: screen reader, reading live data: audio.

What is new about it?

This scenario demonstrates the new capability our display introduces, of listening to and interacting with real-time, live sensor data accessibly.

what would be a user study to evaluate this?

TBD.

Scenario 2. user tries to understand what accelerometer data looks like.

1. user connects accelerometer wirelesly to the web display.
2. user selects the options to sonify one of the axis of the accelerometer.
3. user also selects the option to record the data trace with a 3-second delayed start (like a selfy timer).
4. user puts the accelerometer on their head or holds it in an up-right position in the 3 seconds.
5. user starts to walk and hears the data being sonified.
6. user walks for a few seconds and returns to the computer.
7. user stops streaming and recording the data.
8. user returns to the computer and sees a table of several hundred rows.
9. user listens to data corresponding to each of the axes.
10. user resets the display to sonify from the beginning.
11. user switches NVDA to focus mode,disables the vurtual cursor in JAWS, or disables quick nav in VoiceOver. This is necessary to pass through keyboard shortcuts to the website.
12. user presses p to start playing the audio corresponding to the data.
13. user presses and holds the m key when they hear the pulses as mentioned by the article author and releases them once the pulse is done. user repeats this for a few pulses.
14. user uses the web interface to show only the data in the pulses in the table.
15. user reads the data and understands it.

Why is this generally of value? What is new about it?

to develop applications that sense the world around us, it is important for us to understand how sensors convey information about the activities we perform with them. This is well-described, and visualized in the article. The state-of-the art way to make this accessible right now is to record the data and read later using a screen reader, or generate a tactile representation of the data after it is recorded. Both these methods give users a delayed access to data, and the state of the art tactile representation specifically may not be suitable for live data. The set of interaction techniques related to hearing the data sonification and reading the data mentioned in this scenario provide access to live, updating sensor data thereby making the understanding of how sensors generate data nonvisually accessible.

At the same time, sensors generate a lot of data making screen reading of the data time consuming and impracticle. Consumers may have to listen to, and sift through a huge volume of data even to recognise patterns or thresholds to filter data with. The set of selection techniques simplify this process by offering capabilities to set filters and other selection criteria by listening to approximate values of data and recognising patterns to filter for.

What part of the design space does this illustrate?

reading live data: audio, reading pre-recorded data: audio, selecting data using live sonification of data

what would be a user study to evaluate this scenario?

TBD.

scenario 3. user visualizes steps.

1. user captures data in a more realistic scenario as mentioned in scenario 2.
2. user exports data to a CSV file using the display.
3. user applies the filters and computes the time series data from the CSV data in a programming language of their choyce or Microsoft Excel.
4. user imports the time-series data in the form of a CSV file back into the display.
5. user sets to sonify the data and listens to the sonification to understand the steps. (note: I need to understand what the author meant by visualize the steps).

Why is this generally of value? What is new about it?

In many domains such as IOT and machine learning, drag-and-drop tools and notebook interfaces allow developers to explor data and model it for data intensive applications. These rapid prototyping tools are not accessible to BVI developers. BVI developers often are required to build the necessary infrastructure such as code to read data from a sensor, and helper functions to present data in nonvisually explorable formats increasing the iteration time and effort required to model data. The scenario above uses our display's capabilities of collecting sensor data, exporting it in ways that are already accessible to screen readers in other tools to perform, and import back the processed data for further experimentation and modeling.

What part of the design space does this illustrate?

Reading pre-recorded data: screen reading

what would be a user study to evaluate this scenario?

TBD.