Optimization of training model by val_loss/epoch

[andruuhurst]

Implemented the retraining of the model with the best epoch by finding the lowest validation loss value from model history.

In my experience at work and in the CS499 deep learning class, this is the best practice for optimizing the training model.

I also doubled the number of epochs trained for the initial model. Realistically, this will not affect the final trained model. I did this to prevent the potential possibility of GD having the case of underfitting with only 20 epochs since they could possibly get different results with their data.

Let me know what you guys think.

[David-Rod]

@andruuhurst I have a question regarding this PR to improve my understanding of what is happening.

It seems like you are finding the minimum loss value and saving the epoch number where the val_loss is lowest. In order to do this, you need a history object to be generated, so you initialized it at 40 epochs and then found the "best" epoch value. Your history_optimized_alldata variable uses best_epoch as the epoch value, but the 'x' and 'y' inputs are set to x_data and y_data.

I believe they should be set to x_train and y_train instead. Does this seem correct regarding what your code is trying to accomplish? Otherwise, you are finding the best epoch for all the data, and not just the training data that has been determined by the train_test_split() function call at the beginning of the function.

[andruuhurst]

@David-Rod That is a good question.

We have found our ideal amount of epochs from setting aside the validation set initially. When retraining the model it is no longer necessary to set aside a validation set since we were only using that metric to gage underfitting/overfitting of the model. From my understanding, this allows us to instead use the entirety of the training data on the model and in a way, it can theoretically a see more full range of data to be trained on

[David-Rod]

The master branch will need to be merged with this branch to avoid merge conflicts. Several changes have been merged since this PR was requested.

[David-Rod]

Branch not up to date with current changes. Need to merge with master and remove commented print statements.