Webtics background

Game metrics fall into two types

• internal to the game, or
• recorded by external sources.

Internal logging comes from monitoring the actions in the game, either created by the player or from the game itself. There are many types of external sources of telemetry and metrics information. Standard user testing with a trained observer provides large amounts of external information.

The focus of WebTics is to support internal logging, but it is designed to be able to integrate with external systems. The main telemetry included in the systems comes from calls embedded in the game. These internal logging calls are added by the developer during the implementation of the game, usually with a specific logging objective in mind. This type of logging will only be able to tell us about the player actions or the operations on the game client. It does not, for example, give us additional information about the player, such as the context of play, gender, distractions, accessibility issues or how the player experiences the game.

External sources of information include biometrics, production metrics and performance metrics. Even within the scope of behavioural telemetry, it is difficult to incorporate different types of sources and types of game metrics data into a single unified resource.

When collecting different streams of data synchronization becomes an issue. There are various ways to engineer the environment of a game telemetry logging system to assist synchronizing different sources borrowing from synchronization methods used by other media. For example, in traditional film production a “clapper” is used to produce both a visual event and an audio event. These events are used to align the audio recording with the video recording. Another example would be using a psycho-physiological device such as electroencephalograph (EEG) which records brain activity, and linking this with a video recording of a player; a simple tap of a finger on a specific sensor will give a detectable event in both data streams. Thus, for synchronization of game telemetry, we can use similar approaches. To assist in the synchronization of various data sources, WebTics provides microsecond timestamps for all logged events. This can be used post-session to synchronize the various sources, including situations where external data are used, e.g. from psycho-physiological sensors.

Once we have created an environment that supports the collection of various metrics sources we can improve the usability of the data by standardizing the way in which we store the logged events. There are a set of features that are common to all events that can be logged. For events to be measurable they must be:

• Present: you cannot measure when something does not happen. You cannot measure the fact that there are no bugs in the system. However, you measure when errors occur, and the length of time between errors. It is important to remember to adage “absence of evidence is not evidence of absence”.
• Observable: there is no point trying to measure something that cannot be directly observed either as events in the game or by using some external sensors, such as heart rate monitors.
• Specific: the event must have a clear definition of when it occurs. If the intention is to measure Actions Per Minute (APM) you must define what constitutes an action.