Lab | Natural Language Processing

Introduction

In Challenge 1 of this lab we will walk you through how to prepare HTML data for NLP analysis. In Challenge 2 we will walk you through how to perform sentiment analysis on Tweets using NLTK.

Getting Started

Complete challenge-1.ipynb and challenge-2.ipynb in the your-code directory. Follow the instructions and add your code and explanations as necessary. There are two YouTube videos in Challenge 2 that you can watch to better understand how to use NLTK to perform sentiment analysis.

Deliverables

• challenge-1.ipynb and challenge-2.ipynb containing your responses.

Submission

Upon completion, add your deliverables to git. Then commit git and push your branch to the remote.

Resources

• Matrix of document-term frequency

• NLTK stemming libraries

  • Porter
  • Snowball
  • Lancaster

• Word Net lemmatizer

• Sentiment Analysis

• NLTK sentiment analysis package

• Sentiment140 dataset with 1.6 million tweets on Kaggle

• NLTK FreqDist

• NLTK building features (video)

• Converting words to Features with NLTK

• nltk.NaiveBayesClassifier.train

• NLTK training Naive Bayes (video)

• Naive Bayes Classifier with NLTK