Projects 1-3 were submitted for the Methods of Data Science Course at Imperial College London, they are in Python Notebook form (to view in HTML form use https://htmlpreview.github.io/).

Project 1: Exporing regression and classification methods including ridge and logitstic regression and Naïve Bayes classifiers in Python

Overview

Goal: Predicting climbing success rate from weather predictors

• Data cleaning using Pandas
• Exploratory data analysis
• Compared regression methods including linear and ridge
• Compared classifiers including Logistic Regression and Naïve Bayes using different metrics

Project 2: Exploring supervised Random Forests, Support Vector Machines and Neural Network in-depth for classification on noisy data in Python

Overview

• Data Balancing and Standardisation Methodologies for predicting car ratings
• Explaining K-Fold Stratified Cross-Validation
• Random Forest optimasation for number of trees, depth and number of split predictors
• Support Vector Machines: optimisation of kernels (linear, polynomial and RBF)
• Neural Network optimsation for batch size, learning rate and dropout
• Number of Trees for Random Forest Hyperparameters of RBF kernel SVM

Project 3: Exploring unsupervised graph-based learning methods, clustering methods and neural networks for clothing image classification and academic paper clustering in Python

Overview

• Clustering of academic papers using K-Means, comparing different scoring metrics and analyzing optimal number of clusters and randomness in K-Means
• Graph-based learning methods for clustering including community detection and centrality measures
• Image classification of clothing items using unsupervised methods
• Used PCA to visualise clustering of clothing items and centroids
• Comparison of kNN, hierachical clustering and neural networks for supervised image classification
• Comparison of MLP neural networks and CNNs
• Alteration of CNNs using dropout, other layers and alterations to kernel sizes to increase accuracy
• K-Fold Stratified Cross-Validation used throughout
• Community Detection using CNM Clustering visualised with PCA

Python Projects:

Insurance project

• Basic Data cleaning exercise of some Head on Collision Claims Data.

Stocks project

• Explores the possibility of predicting daily returns (or the sign) on some random stocks given 15 random features.
• Cleaned and combined 3 data frames, splitting time series data into training and test sets.
• Used various statistics like Pearson Correlation and methods such as PCA to analyse features.
• Fitted a basic Ridge Regression to data, optimising hyper parameters.
• Fitted and analysed an ARIMA model.

R Projects

Diet vs Country R Project

• Sourced and cleaned dietary data for 53 random countries.
• Performed eigenvalue analysis to see if there was any dimensionality to the data.
• Completed Classical and Ordinal Scaling with a variety of metrics seeing if there were any trends.
• Perfomed K-means on Scalings to further compare scalings or explainable trends.
• Compared Scalings using Procrustes.