DevOps Implementation and Requirements Update for Streamlined Project Development

.github/workflows/main.yml

@@ -0,0 +1,27 @@
+name: Merge to main and Deploy
+
+on:
+  pull_request:
+    branches:
+      - main
+  push:
+    branches-ignore:
+      - main
+
+jobs:
+  merge-to-main-and-deploy:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v2
+        with:
+          ref: ${{ github.ref }}
+          fetch-depth: 0
+      - name: Merge pull request
+        uses: peter-evans/create-pull-request@v3.10.0
+        with:
+          commit-message: "Merge pull request"
+          title: "Merge pull request"
+          branch: main
+          delete-branch: true
+          token: ${{ secrets.GITHUB_TOKEN }}

Dockerfile

@@ -0,0 +1,13 @@
+# syntax=docker/dockerfile:1
+
+FROM python:3.8-slim-buster
+
+WORKDIR /app
+
+COPY requirements.txt requirements.txt
+
+RUN pip3 install -r requirements.txt
+
+COPY . .
+
+CMD ["python3", "/usr/src/app/main.py"]

JenkinsPipeline.txt

@@ -0,0 +1,34 @@
+pipeline {
+    agent any 
+    environment {
+        registry = "ikramkhan1/mlops_a2"
+        registryCredential = 'dockerhub_id'
+        dockerImage = ''
+    }
+
+    stages {
+        stage('Cloning Git') {
+            steps {
+                checkout scmGit(branches: [[name: '*/main']], extensions: [], userRemoteConfigs: [[url: 'https://github.com/ikram554/spf']])
+            }
+        }
+
+        stage('Building image') {
+          steps{
+            script {
+              dockerImage = docker.build registry
+            }
+          }
+        }
+
+        stage('Upload Image') {
+         steps{    
+             script {
+                docker.withRegistry( '', registryCredential ) {
+                dockerImage.push()
+                }
+            }
+          }
+        }
+    }
+}

README.md

@@ -1,53 +1,75 @@
 # Stock Price Forecasting Flask Web App
 
-### Predicting Appple's Adj. Close Price For The Next 7 Days 
-### WEB APP - https://aritheanalyst.com/ir
-
- # Table Of Contents
-* [Purpose](#purpose)
-* [Data Description](#data-description)
-* [Model](#model)
-* [Covid-19 Models Results](#results)
-* [Recommendation](#recommendation)
-* [Web App Features](#web-app-features)
-* [Web App Structure](#web-app-structure)
-* [Setup](#setup)
-* [Configuration](#configuration)
-* [Future Work](#future-work)
-
+### Dockerhub Repository: 
+https://hub.docker.com/repository/docker/ikramkhan1/mlops_a2/general
+# NOTE:
+#### (Image upload speed to Dockerhub Repository is considerably slower than expected. It has only uploaded 40 MB in 1 hour, and the total size of image is 3.53 GB. This may cause some delays in the upload process or in worse case I might not be able to upload the docker image to dockerhub.)
+
+
+
+## JENKINS PIPELINE
+
+```
+pipeline {
+    agent any 
+    environment {
+        registry = "ikramkhan1/mlops_a2"
+        registryCredential = 'dockerhub_id'
+        dockerImage = ''
+    }
+
+    stages {
+        stage('Cloning Git') {
+            steps {
+                checkout scmGit(branches: [[name: '*/main']], extensions: [], userRemoteConfigs: [[url: 'https://github.com/ikram554/spf']])
+            }
+        }
+
+        stage('Building image') {
+          steps{
+            script {
+              dockerImage = docker.build registry
+            }
+          }
+        }
+
+        stage('Upload Image') {
+         steps{    
+             script {
+                docker.withRegistry( '', registryCredential ) {
+                dockerImage.push()
+                }
+            }
+          }
+        }
+    }
+}
+
+```
+### Note:
+If you're considering using the Jenkins pipeline, please keep in mind the following important instructions for customization:
+- To add your DockerHub credentials, navigate to Manage Jenkins → Manage Credentials in Jenkins, and update the appropriate field.
+- To replace the GitHub URL in the checkout stage, simply update it to reflect your own repository's URL.
+- Finally, ensure that you update the "registry" field to match the URL for your own DockerHub repository.
+By following these steps, you can easily customize the Jenkins pipeline for your own purposes.
+
+
  ## Preview
   <img src='screenshots/home.PNG' width='50%'/>
   <img src='screenshots/results.PNG' width='50%'/>
   <img src='screenshots/results2.PNG' width='50%'/>
   <img src='screenshots/results3.PNG' width='50%'/>
-
-## Purpose
-The purpose of this research is to build a model that can efficiently predict a company's Adj. Close price for the next 7 days.
-
-## Data Description
-For this research, I used the Apple's historical stock data extracted from Yahoo Finance: 
-* https://finance.yahoo.com/q/hp?s=YHOO
 
-
-## Model
-Firstly, I loaded the specific Apple (AAPL) dataset from yahoo finance. This dataset is composed of 12 different features but I just used the Adj. Close price column since that's what I'm going to be forecasting using the ARMA model. Next step is to test for stationarity but given that this is a stock data, its highly likely that it's not going to be stationary. This is because stock prices usually increase over time. So given its not stationary, the mean of the data increases over time which might ultimately lead to a degradation of the model. 
-To test if the data is stationary, I used the Augmented Dickey-Fuller Test. As we can see, the p-value is larger than the significance level of 0.05 so I will accept the null hypothesis which is that the data is non-stationary. 
-
   <img src='screenshots/trends.PNG' width='50%'/>
   <img src='screenshots/corr.PNG' width='50%'/>
 
-## [Results](./arima_and_nlp.ipynb)
-Next, I created ACF and PACF plots to determine the p and q input parameters for our ARIMA model. From the plots, 1 and 2 seems to be a good values for p and q respectively but I'll still run an autoarima model to be sure I utilize the most optimal parameters. I then went on into fitting the model in train half of the data using 0 as my p and q parameters gotten from running the autoarima model with 1 as my order of differencing. After validating the model with test results, my root mean square error is 2.5 rmse which is not too bad. 
 
   <img src='screenshots/autoarima.PNG' width='50%'/>
     <img src='screenshots/error.PNG' width='50%'/>
   <img src='screenshots/twitter.PNG' width='50%'/>
 
 
-### Recommendation
-* 0 seems to be the best parameters for p and q with 1 as the order of differencing to use when forecasting AAPL stock dataset but I recommend using an autoarima model to be sure the best parameters are picked before fitting in the training data.
-
-## Setup
+## For Manual Setup
 - Install the requirements and setup the development environment.
 
 	`pip3 install -r requirements.txt`
@@ -58,6 +80,3 @@ Next, I created ACF and PACF plots to determine the p and q input parameters for
 		`python3 main.py`
 
 - Navigate to `localhost:5000`.
-
-## Future Work
-   * Use a simple LSTM model to forecast 7 days out then do the same with a Multivariate LSTM model. 

main.py

@@ -739,6 +739,7 @@ def recommending(df, global_polarity,today_stock,mean):
     # plt.legend()
     # plt.savefig('static/QL.png')
     # plt.show()
+    # extra comment
 
 
 

requirements.txt

@@ -1,5 +1,5 @@
 nltk
-keras==2.3.1
+keras
 numpy
 streamlit==0.52.1
 seaborn
@@ -9,8 +9,10 @@ flask
 pandas
 matplotlib
 scikit_learn
-statsmodels==0.10.1
+statsmodels
 yfinance==0.1.54
 alpha_vantage==2.1.3
-tensorflow==1.14.0
+tensorflow
+xgboost
+simplejson
 tweet-preprocessor