living_document.tex

\documentclass{tufte-handout}

%\geometry{showframe}% for debugging purposes -- displays the margins

\usepackage{amsmath}

% Set up the images/graphics package
\usepackage{graphicx}
\setkeys{Gin}{width=\linewidth,totalheight=\textheight,keepaspectratio}
\graphicspath{{graphics/}}

\title{Analysis of RPGLite data}
\author[]{Tom Wallis \& William Kavanagh}
\date{April 2020}  % if the \date{} command is left out, the current date will be used

\usepackage{hyperref}
\hypersetup{
    colorlinks=true,
    linkcolor=blue,
    filecolor=magenta,      
    urlcolor=cyan,
}

% The following package makes prettier tables.  We're all about the bling!
\usepackage{booktabs}

% The units package provides nice, non-stacked fractions and better spacing
% for units.
\usepackage{units}

% The fancyvrb package lets us customize the formatting of verbatim
% environments.  We use a slightly smaller font.
\usepackage{fancyvrb}
\fvset{fontsize=\normalsize}

% Small sections of multiple columns
\usepackage{multicol}

% Provides paragraphs of dummy text
\usepackage{lipsum}

\usepackage{tikz}
\usetikzlibrary{arrows.meta}


\usepackage[obeyFinal,textsize=footnotesize]{todonotes}

% These commands are used to pretty-print LaTeX commands
\newcommand{\doccmd}[1]{\texttt{\textbackslash#1}}% command name -- adds backslash automatically
\newcommand{\docopt}[1]{\ensuremath{\langle}\textrm{\textit{#1}}\ensuremath{\rangle}}% optional command argument
\newcommand{\docarg}[1]{\textrm{\textit{#1}}}% (required) command argument
\newenvironment{docspec}{\begin{quote}\noindent}{\end{quote}}% command specification environment
\newcommand{\docenv}[1]{\textsf{#1}}% environment name
\newcommand{\docpkg}[1]{\texttt{#1}}% package name
\newcommand{\doccls}[1]{\texttt{#1}}% document class name
\newcommand{\docclsopt}[1]{\texttt{#1}}% document class option name

\newcommand{\WilliamToDo}[1]{\todo[color=cyan]{\textbf{W:~{#1}}}}
\newcommand{\TomToDo}[1]{\todo[color=orange]{\emph{T:~{#1}}}}

\begin{document}

\maketitle% this prints the handout title, author, and date

\begin{abstract}
\noindent This document describes the data generated by the RPGLite experiment, the analysis we have performed on it thus far and areas where we should like to perform further analysis
\end{abstract}

%\printclassoptions

\section{research questions}



\section{The Data}


Data\footnote{this may not be a necessary section to include, though it may be useful if this document is to be used as a reference for \textit{outsiders}.} is collected in the form of a Mongo database and organised into six collections of similar documents detailed here with their notable fields:
\begin{enumerate}
    \item \textbf{games}: in progress games
    \begin{itemize}
        \item usernames -- an array of the usernames for players 1 and 2 respectively,
        \item start\_time -- python datetime object given moment of game creation to millisecond precision,
        \item Moves -- list of moves made in notation of the form $pxC^1pyC^2\_nn$ which would denote player $x \in \{1,2\}$ using character $C^1 \in \{K,A,R,H,W,B,M,G\}$  to attack player $y$'s character $C^2$ and rolling $nn \in \{0..99\}$ 
    \end{itemize}
    \item \textbf{completed\_games}: games now completed
        \begin{itemize}
        \item as above.
        \item winner -- {1,2} denoting winner if one was found (can be non present if game was abandon)
    \end{itemize}
    \item \textbf{page\_hits}:
    \item \textbf{player\_backup}
    \item \textbf{players}:
    \item \textbf{special\_data}:
\end{enumerate}

\section{Initial Analysis}

\subsection{win-delta / pick-rate}\footnote{The three figures described here are all generated by running \textit{rolling\_analysis.py}}

A common form for game analytics, also known as pick-rate/win-rate, win-delta / pick-rate (herein WD/PR) plots game material in a way that allows for easy comparison of how popular and powerful they are. WD/PR is commonly used as the basis for balance changes, however it is flawed. Whilst it does give a good account of material which should be tweaked to bring it more in line with the rest, it does not account for personal preference or represent mismatches between specific material. 

\begin{figure}
    \centering
    \includegraphics{pick-win.png}
    \caption{WD/PR generated 14-4, 10 days after initial release. Guidelines are drawn at x=0.125 or 1/8 as that is what would be the value if each were chosen equally, and at y = 0\% as that would be the value of material that wins as often as it loses.}
    \label{fig:WD/PR}
\end{figure}

% Lad, I can't get this warning to buzz off.

Fig. ~\ref{fig:WD/PR}~shows that the Healer is both the most successful material and the least popular. \href{'https://staticctf.akamaized.net/J3yJr34U2pZ2Ieem48Dwy9uqj5PNUQTn/2MXEODt9QteOU3WGh4vmNE/51261bebfc558521b7411ccb78ece4fb/Y5S1_Matrix_Attackers.png'}{There are examples of this peculiar occurrence in other games.} Perhaps the low popularity makes the win-delta value more volatile, or the players are choosing based on factors other than their effectiveness. However popularity should be balanced for just as effectiveness is, as game designer Jeff Kaplan states, \textit{"The perception of balance is more powerful than balance itself."}

As our previous work pertains to metagame development, we wanted to investigate if we could use WD/PR variants to measure this development over time. To do this we have plotted WD/PR for every day since release. 

\begin{figure}
    \centering
    \includegraphics{pick-win-per-day-snipped.PNG}
    \caption{Snippet of WD/PR over days}
    \label{fig:my_label}
\end{figure}

As a measure of how \textit{settled} the metagame is, we plotted each character on their own axes with their WD/PR values over days. Between each point there is a vector giving the disparity between points, the average of these vectors between each day should give a good indication of how \textit{settled} the metagame is.

\begin{figure}
    \centering
    \includegraphics{pick-win-per-char.png}
    \caption{WD/PR, per character, over days}
    \label{fig:my_label}
\end{figure}

\subsection{Elo and non-skill factors on placement}

RPGLite has two systems for player ranking, \textit{skill}, a measure of our own creation which is public to all players, and \textit{Elo}, which we also measure, but that we do not make public. Elo is a standard mechanism for measuring player skill, devised for Chess in the 1950s. It forms the basis for a lot of competitive matchmaking ranking systems used today, including \textit{TrueSkill} developed by Microsoft \footnote{For more on Elo, TrueSkill and competitive matchmaking, \href{https://www.microsoft.com/en-us/research/wp-content/uploads/2006/10/Game-Developer-Feature-Article-Graepel-Herbrich.pdf}{this Microsoft white paper} is a good source.}. Elo is a far better measure of player skill than the "friendly" measure of \textit{skill} which rewards games played over success. 

\begin{figure}
    \centering
    \includegraphics{luck_over_Elo.png}
    \caption{Elo of the playerbase plotted against luck}
    \label{fig:luck-over-Elo}
\end{figure}

We can show that there is a relationship between Elo and the average roll in the player-base, this stands to reason.

It should also be mentioned that RPGLite does have an issue with first-move bias, due to its simplistic nature. A testament to this is that from the first 1653 games completed, 982 were won by the player going first, or 59.4\%.

\subsection{Skill}

This work is devised specifically so that we can say of any given move made in the game whether or not it was \textit{correct}, and if it was not, then what the \textit{cost} of playing the move was. The values referred to here are the optimal values (or \textit{minimax} values), the probability of winning given your opponent tries to minimise the probability of you winning and you try to maximise it. Through exhaustive state exploration we can attribute to every possible action, in every reachable state, for every configuration of the game, a single value of their optimal probability of winning. When faced with a decision to make, the players optimal probability of winning is equivalent to that of when they have chosen to take the optimal action from that state, before the outcome is determined (i.e. before the action has succeeded or failed). Should they take a different action where their optimal probability of winning is less having chosen an action than it was before, then they have chosen a \textit{sub-optimal} action and the difference between their current and past optimal values can be considered the \textit{cost} of the move. 

The cost function is more easily understood with the aid of diagrams where the game is represented as a tree. Other than at the root, every node on our tree can be attributed to one of the two players, we will colour them red and blue. Red nodes represent turns of the red player where they must decide on a single action to take. These actions then have a probabilistic outcome, for RPGLite this is often \textit{hit} or \textit{miss}. Using a simplified example where it is the blue player's turn in state \emph{s} and they have only two actions available \emph{Knight attack} and \emph{Rogue attack}, here is what part of the tree would look like.


\definecolor{lightgrey}{rgb}{0.7, 0.7, .7}
\definecolor{lightblue}{rgb}{0.5, 0.5, 1}
\definecolor{lightred}{rgb}{1,.5,.5}
\usetikzlibrary{shapes}
\begin{center}\begin{tikzpicture}[]%{{{
    
    \node[draw, circle, fill=lightblue] (n1) at (3,7) {$S$};
    \node[align=center,anchor=south] (lab) at (n1.north) {p(\textcolor{lightblue}{win}) = 0.7};
    \node[draw, diamond, fill=lightgrey] (n3) at (1.5,4) {};
    \node[align=center,anchor=east] (lab) at (n3.west) {Knight\\attack};
    \node[draw, diamond, fill=lightgrey] (n4) at (4.5,4) {};
    \node[align=center,anchor=west] (lab) at (n4.east) {Rogue\\attack};
    \node[draw, circle, fill=lightred] (n5) at (0,1) {$O_1$};
    \node[align=center,anchor=north] (lab) at (n5.south) {p(\textcolor{lightblue}{win}) = 0.9};
    \node[draw, circle, fill=lightred] (n6) at (3,1) {$O_2$};
    \node[align=center,anchor=north] (lab) at (n6.south) {p(\textcolor{lightblue}{win}) = 0.6};
    \node[draw, circle, fill=lightred] (n7) at (6,1) {$O_3$};
    \node[align=center,anchor=north] (lab) at (n7.south) {p(\textcolor{lightblue}{win}) = 0.8};

    \draw [-{Latex[lightblue, width=6mm, length=6mm]}] (n1) -> (n3);
    \draw [-{Latex[lightblue, width=6mm, length=6mm]}] (n1) -> (n4);
    \draw [-{Latex[lightgrey, width=6mm, length=6mm]}] (n3) -> (n5) 
        node[midway,sloped,above] {hit}
        node[midway,sloped,below] {$0.6$};
    \draw [-{Latex[lightgrey, width=6mm, length=6mm]}] (n3) -> (n6) 
        node[midway,sloped,above] {miss}
        node[midway,sloped,below] {$0.4$};
    \draw [-{Latex[lightgrey, width=6mm, length=6mm]}] (n4) -> (n6) 
        node[midway,sloped,above] {miss}
        node[midway,sloped,below] {$0.3$};
    \draw [-{Latex[lightgrey, width=6mm, length=6mm]}] (n4) -> (n7) 
        node[midway,sloped,above] {hit}
        node[midway,sloped,below] {$0.7$};

\end{tikzpicture}\end{center}

There are 3 states that could result from the blue players decision, \emph{O$_1$, O$_2$ and O$_3$} and all states are labelled with the optimal probability that the blue player wins (as p(\textcolor{lightblue}{win})). In \emph{S} blue has a 0.7 minimax probability of winning and in state \emph{O$_1$}, where they have successfully performed \emph{Knight attack}, that value has increased up to 0.9. To calculate which action is optimal we must sum the minimax value of each state by the probability of reaching that state. Therefore the optimal probability of the blue player winning having chosen \emph{Knight attack} in state \emph{S} is $0.9\times 0.6 + 0.6\times 0.4 = 0.78$. Similarly, if they chose \emph{Rogue attack} it would be 0.74. From state \emph{S} the optimal action for blue is \emph{Knight attack}, and we calculate the cost of \emph{Rogue attack} as being the difference between 0.78 and 0.74, 0.04. 


A rudimentary measure of how skilful a player is at RPGLite is to average the cost of every move made in every game they have played. In Fig. \ref{fig:cost-over-Elo} we show that there is a relationship between this rudimentary measure of skill and a player's Elo value, which unsurprisingly implies that skill is a factor in success.

\begin{figure}
    \centering
    \includegraphics{cost_over_Elo.png}
    \caption{Elo of the playerbase plotted against skill}
    \label{fig:cost-over-Elo}
\end{figure}

\subsection{Player types}

By considering the experience of the player when looking at character selection we can investigate what characters players \textit{learn} to play. Fig. \ref{fig:char-select-by-exp}~can be recreated by user$\_$picks$\_$over$\_$time.py

\begin{figure}
    \centering
    \includegraphics{character_selection_by_games_played.png}
    \caption{Proportion of characters chosen by players of various experience, in 5 game intervals. The number of games played in each interval is also shown to show values that may be less accurate.}
    \label{fig:char-select-by-exp}
\end{figure}

\subsection{Predictive value of chained strategy generation}

Using chained strategy generation (CSG), we claim that the configuration of RPGLite used 

\end{document}