-
Notifications
You must be signed in to change notification settings - Fork 2
Benchmarking #23
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Benchmarking #23
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| Section,Start Time,End Time,Time Elapsed | ||
| Create Tasks,0.01173190000008617,0.011608300001171301,-0.0001235999989148695 | ||
| Create Tasks,0.011499300000650692,0.012321500000325614,0.0008221999996749219 | ||
| Create Tasks,0.012110600000596605,0.011474599999928614,-0.0006360000006679911 | ||
| Create Tasks,0.0071184999997058185,0.011835499999506283,0.004716999999800464 | ||
| Create Tasks,0.011849700000311714,0.01152170000023034,-0.0003280000000813743 | ||
| Create Tasks,0.011755899999116082,0.016899099999136524,0.0051432000000204425 | ||
| Create Tasks,0.0073439999996480765,0.007071200001519173,-0.0002727999981289031 | ||
| Create Tasks,0.006634500001382548,0.00645040000017616,-0.0001841000012063887 | ||
| Create Tasks,0.006856500000139931,0.006442500000048312,-0.00041400000009161886 | ||
| Create Tasks,0.00744479999957548,0.00603179999961867,-0.00141299999995681 | ||
| Create Tasks,0.006359799999700044,0.006122100001448416,-0.00023769999825162813 | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Why do we have negative time? Is this a bug or is there something else wrong? What tasks did we get these data points from?
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm unsure on the negative times -- it could be something with the API that was wrong, or it could be from creating the start time at one point and then repeatedly trying to clock out. I'll do more testing, and on sections that don't get repeated. I'll also look into some different APIs to track time. |
||
| Create Tasks,0.006877599998915684,0.006041900000127498,-0.0008356999987881863 | ||
| Create Tasks,0.009612300000299001,0.006650999999692431,-0.0029613000006065704 | ||
| Create Tasks,0.0063289999998232815,0.0062319000007846626,-9.709999903861899e-05 | ||
| Create Tasks,0.006966599999941536,0.006507399999463814,-0.0004592000004777219 | ||
| Create Tasks,0.006575199999133474,0.006429600000046776,-0.00014559999908669852 | ||
| Create Tasks,0.008982899998954963,0.00653020000027027,-0.002452699998684693 | ||
| Create Tasks,0.0068334000006871065,0.0062576999989687465,-0.00057570000171836 | ||
| Create Tasks,0.006512800000564312,0.006429000000935048,-8.379999962926377e-05 | ||
| Create Tasks,0.006611800001337542,0.007598100000905106,0.0009862999995675636 | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,56 @@ | ||
| import matplotlib.pyplot as plt | ||
| import timeit | ||
| import csv | ||
|
|
||
| class Graph: | ||
| #static 2d array to store data | ||
| taskData = [[],[],[],[],[]] | ||
|
|
||
| #initialize function to start our graph | ||
| def __init__(self): | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Where is this code actually being ran? In server.py? Do we need a constructor there and need to be calling things like addPoint in certain places?
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. During testing I was running this in server.py, but I was unsure where we wanted to pull in these values. We would need to create the constructor there and call these functions -- I took it out after doing time testing but we would just place it in where appropriate. |
||
| self.title = 'Tasks per Request vs Average Reward' | ||
|
|
||
| #add data point for the graph. Use x to specify the tasks per request | ||
| def addPoint(self, x, y): | ||
| taskData[x].append(y) | ||
|
|
||
| #plot graph when finished, export to benchData | ||
| def plot(self, numTasks): | ||
| xvals = [] | ||
| yvals = [] | ||
| for x in range(0, numTasks): | ||
| xvals.append(x) | ||
| yvals.append(sum(taskData(x))/len(taskData(x))) | ||
| plt.plot(xvals,yvals) | ||
| plt.xlabel('Tasks per Request') | ||
| plt.ylabel('Average Reward') | ||
| plt.title(self.title) | ||
| plt.savefig('benchData/taskVreward.png') | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do we have any idea what this png looks like currently? Can you add that to the PR? @kylevitale2
Collaborator
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I haven't tested this yet, I was unsure of where to pull the rewards from, and during the times I ran server.py and client.py it seemed I only got one run, which can be graphed but defeats the average aspect. I'll do tests on this soon. |
||
|
|
||
| class Time: | ||
| #initialize function to give label | ||
| def __init__(self, label): | ||
| self.label = label | ||
|
|
||
| #start timer | ||
| def startClock(self): | ||
| self.startTime = timeit.timeit() | ||
|
|
||
| #stop timer and add data to csv | ||
| def endClock(self): | ||
| self.endTime = timeit.timeit() | ||
| timeElapsed = self.endTime - self.startTime | ||
| runData = [self.label, self.startTime, self.endTime, timeElapsed] | ||
| with open('benchData/time.csv', 'a') as f: | ||
| writer = csv.writer(f) | ||
| writer.writerow(runData) | ||
|
|
||
| #reset CSV file | ||
| def resetCSV(self): | ||
| header = ['Section','Start Time','End Time','Time Elapsed'] | ||
| with open('benchData/time.csv' , 'w') as f: | ||
| writer = csv.writer(f) | ||
| writer.writerow(header) | ||
|
|
||
|
|
||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What is section? @kylevitale2
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Section is just a label for us to determine what section of code we were timing