Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions benchData/time.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
Section,Start Time,End Time,Time Elapsed
Create Tasks,0.01173190000008617,0.011608300001171301,-0.0001235999989148695
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is section? @kylevitale2

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Section is just a label for us to determine what section of code we were timing

Create Tasks,0.011499300000650692,0.012321500000325614,0.0008221999996749219
Create Tasks,0.012110600000596605,0.011474599999928614,-0.0006360000006679911
Create Tasks,0.0071184999997058185,0.011835499999506283,0.004716999999800464
Create Tasks,0.011849700000311714,0.01152170000023034,-0.0003280000000813743
Create Tasks,0.011755899999116082,0.016899099999136524,0.0051432000000204425
Create Tasks,0.0073439999996480765,0.007071200001519173,-0.0002727999981289031
Create Tasks,0.006634500001382548,0.00645040000017616,-0.0001841000012063887
Create Tasks,0.006856500000139931,0.006442500000048312,-0.00041400000009161886
Create Tasks,0.00744479999957548,0.00603179999961867,-0.00141299999995681
Create Tasks,0.006359799999700044,0.006122100001448416,-0.00023769999825162813
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we have negative time? Is this a bug or is there something else wrong? What tasks did we get these data points from?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm unsure on the negative times -- it could be something with the API that was wrong, or it could be from creating the start time at one point and then repeatedly trying to clock out. I'll do more testing, and on sections that don't get repeated. I'll also look into some different APIs to track time.

Create Tasks,0.006877599998915684,0.006041900000127498,-0.0008356999987881863
Create Tasks,0.009612300000299001,0.006650999999692431,-0.0029613000006065704
Create Tasks,0.0063289999998232815,0.0062319000007846626,-9.709999903861899e-05
Create Tasks,0.006966599999941536,0.006507399999463814,-0.0004592000004777219
Create Tasks,0.006575199999133474,0.006429600000046776,-0.00014559999908669852
Create Tasks,0.008982899998954963,0.00653020000027027,-0.002452699998684693
Create Tasks,0.0068334000006871065,0.0062576999989687465,-0.00057570000171836
Create Tasks,0.006512800000564312,0.006429000000935048,-8.379999962926377e-05
Create Tasks,0.006611800001337542,0.007598100000905106,0.0009862999995675636
56 changes: 56 additions & 0 deletions benchmarking.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
import matplotlib.pyplot as plt
import timeit
import csv

class Graph:
#static 2d array to store data
taskData = [[],[],[],[],[]]

#initialize function to start our graph
def __init__(self):
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is this code actually being ran? In server.py? Do we need a constructor there and need to be calling things like addPoint in certain places?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

During testing I was running this in server.py, but I was unsure where we wanted to pull in these values. We would need to create the constructor there and call these functions -- I took it out after doing time testing but we would just place it in where appropriate.

self.title = 'Tasks per Request vs Average Reward'

#add data point for the graph. Use x to specify the tasks per request
def addPoint(self, x, y):
taskData[x].append(y)

#plot graph when finished, export to benchData
def plot(self, numTasks):
xvals = []
yvals = []
for x in range(0, numTasks):
xvals.append(x)
yvals.append(sum(taskData(x))/len(taskData(x)))
plt.plot(xvals,yvals)
plt.xlabel('Tasks per Request')
plt.ylabel('Average Reward')
plt.title(self.title)
plt.savefig('benchData/taskVreward.png')
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have any idea what this png looks like currently? Can you add that to the PR? @kylevitale2

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't tested this yet, I was unsure of where to pull the rewards from, and during the times I ran server.py and client.py it seemed I only got one run, which can be graphed but defeats the average aspect. I'll do tests on this soon.


class Time:
#initialize function to give label
def __init__(self, label):
self.label = label

#start timer
def startClock(self):
self.startTime = timeit.timeit()

#stop timer and add data to csv
def endClock(self):
self.endTime = timeit.timeit()
timeElapsed = self.endTime - self.startTime
runData = [self.label, self.startTime, self.endTime, timeElapsed]
with open('benchData/time.csv', 'a') as f:
writer = csv.writer(f)
writer.writerow(runData)

#reset CSV file
def resetCSV(self):
header = ['Section','Start Time','End Time','Time Elapsed']
with open('benchData/time.csv' , 'w') as f:
writer = csv.writer(f)
writer.writerow(header)



2 changes: 2 additions & 0 deletions server.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@
from stage import Stage
from flask import Flask
from runner import Runner
from benchmarking import Graph
from benchmarking import Time

app = Flask(__name__)

Expand Down