diff --git a/lab-functional-programming/your-code/.ipynb_checkpoints/Q1-checkpoint.ipynb b/lab-functional-programming/your-code/.ipynb_checkpoints/Q1-checkpoint.ipynb new file mode 100644 index 0000000..9c5da04 --- /dev/null +++ b/lab-functional-programming/your-code/.ipynb_checkpoints/Q1-checkpoint.ipynb @@ -0,0 +1,178 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In the cell below, create a Python function that wraps your previous solution for the Bag of Words lab.\n", + "\n", + "Requirements:\n", + "\n", + "1. Your function should accept the following parameters:\n", + " * `docs` [REQUIRED] - array of document paths.\n", + " * `stop_words` [OPTIONAL] - array of stop words. The default value is an empty array.\n", + "\n", + "1. Your function should return a Python object that contains the following:\n", + " * `bag_of_words` - array of strings of normalized unique words in the corpus.\n", + " * `term_freq` - array of the term-frequency vectors." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Import required libraries\n", + "\n", + "# Define function\n", + "def get_bow_from_docs(docs, stop_words=[]):\n", + " \n", + " # In the function, first define the variables you will use such as `corpus`, `bag_of_words`, and `term_freq`.\n", + " \n", + " \n", + " \n", + " \"\"\"\n", + " Loop `docs` and read the content of each doc into a string in `corpus`.\n", + " Remember to convert the doc content to lowercases and remove punctuation.\n", + " \"\"\"\n", + "\n", + " \n", + " \n", + " \"\"\"\n", + " Loop `corpus`. Append the terms in each doc into the `bag_of_words` array. The terms in `bag_of_words` \n", + " should be unique which means before adding each term you need to check if it's already added to the array.\n", + " In addition, check if each term is in the `stop_words` array. Only append the term to `bag_of_words`\n", + " if it is not a stop word.\n", + " \"\"\"\n", + "\n", + " \n", + " \n", + " \n", + " \"\"\"\n", + " Loop `corpus` again. For each doc string, count the number of occurrences of each term in `bag_of_words`. \n", + " Create an array for each doc's term frequency and append it to `term_freq`.\n", + " \"\"\"\n", + "\n", + " \n", + " \n", + " # Now return your output as an object\n", + " return {\n", + " \"bag_of_words\": bag_of_words,\n", + " \"term_freq\": term_freq\n", + " }\n", + " " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Test your function without stop words. You should see the output like below:\n", + "\n", + "```{'bag_of_words': ['ironhack', 'is', 'cool', 'i', 'love', 'am', 'a', 'student', 'at'], 'term_freq': [[1, 1, 1, 0, 0, 0, 0, 0, 0], [1, 0, 0, 1, 1, 0, 0, 0, 0], [1, 0, 0, 1, 0, 1, 1, 1, 1]]}```" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Define doc paths array\n", + "docs = []\n", + "\n", + "# Obtain BoW from your function\n", + "bow = get_bow_from_docs(docs)\n", + "\n", + "# Print BoW\n", + "print(bow)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "If your attempt above is successful, nice work done!\n", + "\n", + "Now test your function again with the stop words. In the previous lab we defined the stop words in a large array. In this lab, we'll import the stop words from Scikit-Learn." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from sklearn.feature_extraction import stop_words\n", + "print(stop_words.ENGLISH_STOP_WORDS)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "You should have seen a large list of words that looks like:\n", + "\n", + "```frozenset({'across', 'mine', 'cannot', ...})```\n", + "\n", + "`frozenset` is a type of Python object that is immutable. In this lab you can use it just like an array without conversion." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Next, test your function with supplying `stop_words.ENGLISH_STOP_WORDS` as the second parameter." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "bow = get_bow_from_docs(bow, stop_words.ENGLISH_STOP_WORDS)\n", + "\n", + "print(bow)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "You should have seen:\n", + "\n", + "```{'bag_of_words': ['ironhack', 'cool', 'love', 'student'], 'term_freq': [[1, 1, 0, 0], [1, 0, 1, 0], [1, 0, 0, 1]]}```" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.12" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/lab-functional-programming/your-code/.ipynb_checkpoints/Q2-checkpoint.ipynb b/lab-functional-programming/your-code/.ipynb_checkpoints/Q2-checkpoint.ipynb new file mode 100644 index 0000000..eaedba9 --- /dev/null +++ b/lab-functional-programming/your-code/.ipynb_checkpoints/Q2-checkpoint.ipynb @@ -0,0 +1,131 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now we want to enhance the `get_bow_from_docs` function so that it will work with HTML webpages. In HTML, there are a lot of messy codes such as HTML tags, Javascripts, [unicodes](https://www.w3schools.com/charsets/ref_utf_misc_symbols.asp) that will mess up your bag of words. We need to clean up those junk before generating BoW.\n", + "\n", + "Next, what you will do is to define several new functions each of which is specialized to clean up the HTML codes in one aspect. For instance, you can have a `strip_html_tags` function to remove all HTML tags, a `remove_punctuation` function to remove all punctuation, a `to_lower_case` function to convert string to lowercase, and a `remove_unicode` function to remove all unicodes.\n", + "\n", + "Then in your `get_bow_from_doc` function, you will call each of those functions you created to clean up the HTML before you generate the corpus.\n", + "\n", + "Note: Please use Python string operations and regular expression only in this lab. Do not use extra libraries such as `beautifulsoup` because otherwise you loose the purpose of practicing." + ] + }, + { + "cell_type": "code", + "execution_count": 60, + "metadata": {}, + "outputs": [], + "source": [ + "# Define your string handling functions below\n", + "# Minimal 3 functions\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Next, paste your previously written `get_bow_from_docs` function below. Call your functions above at the appropriate place." + ] + }, + { + "cell_type": "code", + "execution_count": 61, + "metadata": {}, + "outputs": [], + "source": [ + "def get_bow_from_docs(docs, stop_words=[]):\n", + " # In the function, first define the variables you will use such as `corpus`, `bag_of_words`, and `term_freq`.\n", + " corpus = []\n", + " bag_of_words = []\n", + " term_freq = []\n", + " \n", + " # write your codes here\n", + " \n", + " return {\n", + " \"bag_of_words\": bag_of_words,\n", + " \"term_freq\": term_freq\n", + " }\n", + " " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Next, read the content from the three HTML webpages in the `your-codes` directory to test your function." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [ + { + "ename": "ImportError", + "evalue": "cannot import name 'stop_words' from 'sklearn.feature_extraction' (C:\\Users\\angel\\anaconda3\\lib\\site-packages\\sklearn\\feature_extraction\\__init__.py)", + "output_type": "error", + "traceback": [ + "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[1;31mImportError\u001b[0m Traceback (most recent call last)", + "Input \u001b[1;32mIn [1]\u001b[0m, in \u001b[0;36m\u001b[1;34m()\u001b[0m\n\u001b[1;32m----> 1\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01msklearn\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mfeature_extraction\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m stop_words\n\u001b[0;32m 2\u001b[0m bow \u001b[38;5;241m=\u001b[39m get_bow_from_docs([\n\u001b[0;32m 3\u001b[0m \u001b[38;5;124m'\u001b[39m\u001b[38;5;124mwww.coursereport.com_ironhack.html\u001b[39m\u001b[38;5;124m'\u001b[39m,\n\u001b[0;32m 4\u001b[0m \u001b[38;5;124m'\u001b[39m\u001b[38;5;124men.wikipedia.org_Data_analysis.html\u001b[39m\u001b[38;5;124m'\u001b[39m,\n\u001b[1;32m (...)\u001b[0m\n\u001b[0;32m 7\u001b[0m stop_words\u001b[38;5;241m.\u001b[39mENGLISH_STOP_WORDS\n\u001b[0;32m 8\u001b[0m )\n\u001b[0;32m 10\u001b[0m \u001b[38;5;28mprint\u001b[39m(bow)\n", + "\u001b[1;31mImportError\u001b[0m: cannot import name 'stop_words' from 'sklearn.feature_extraction' (C:\\Users\\angel\\anaconda3\\lib\\site-packages\\sklearn\\feature_extraction\\__init__.py)" + ] + } + ], + "source": [ + "from sklearn.feature_extraction import stop_words\n", + "bow = get_bow_from_docs([\n", + " 'www.coursereport.com_ironhack.html',\n", + " 'en.wikipedia.org_Data_analysis.html',\n", + " 'www.lipsum.com.html'\n", + " ],\n", + " stop_words.ENGLISH_STOP_WORDS\n", + ")\n", + "\n", + "print(bow)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Do you see any problem in the output? How do you improve the output?\n", + "\n", + "A good way to improve your codes is to look into the HTML data sources and try to understand where the messy output came from. A good data analyst always learns about the data in depth in order to perform the job well.\n", + "\n", + "Spend 20-30 minutes to improve your functions or until you feel you are good at string operations. This lab is just a practice so you don't need to stress yourself out. If you feel you've practiced enough you can stop and move on the next challenge question." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.12" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/lab-functional-programming/your-code/.ipynb_checkpoints/Q3-checkpoint.ipynb b/lab-functional-programming/your-code/.ipynb_checkpoints/Q3-checkpoint.ipynb new file mode 100644 index 0000000..7c8db00 --- /dev/null +++ b/lab-functional-programming/your-code/.ipynb_checkpoints/Q3-checkpoint.ipynb @@ -0,0 +1,207 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "**Lambda** is a special Python function type that is **anonymous**. By *anonymous* it means a lambda function does not have name. Lambda functions are embedded inside codes so that you don't call them like calling regular Python functions.\n", + "\n", + "**`Map`** applies a function to all the items in an input list. The function that is applied can be a standard or a lambda function.\n", + "\n", + "For instance, below is an example of multiplying number tuples in a list:" + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[2, 12, 30]" + ] + }, + "execution_count": 11, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "items = [(1,2), (3,4), (5,6)]\n", + "\n", + "def multiply(num_tuple):\n", + " return num_tuple[0]*num_tuple[1]\n", + "list(map(multiply, items))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "...is the same as:" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [ + { + "data": { + "text/plain": [ + "[2, 12, 30]" + ] + }, + "execution_count": 10, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "items = [(1,2), (3,4), (5,6)]\n", + "list(map(lambda item: item[0]*item[1], items))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Why do we sometimes use `lambda` and `map`? Because, as you see in the example above, they make your code really concise by combining 3 lines of code to 1 line.\n", + "\n", + "Besides `map`, there is also **`filter`** that selectively returns elements in an array according to whether you return `True`. There is also **`reduce`** that performs computation on a list of items then returns result.\n", + "\n", + "Here is a [good tutorial](http://book.pythontips.com/en/latest/map_filter.html) about `map`, `filter`, and `reduce`. Read it if you are not familiar with how they are used. Then proceed to the next cell." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In the next cell, use `filter` and `lambda` to return a list that contains positive numbers only. The output should be:\n", + "\n", + "```[1, 4, 5]```" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "numbers = [1, 4, -1, -100, 0, 5, -99]\n", + "\n", + "# Enter your code below" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Next, use `reduce` and `lambda` to return a string that only contains English terms. The English terms are separated with a whitespace ` `.\n", + "\n", + "Hints: \n", + "\n", + "* If your Jupyter Notebook cannot import `langdetect`, you need to install it with `pip install langdetect`. If Jupyter Notebook still cannot find the library, try install with `python3 -m pip install langdetect`. This is because you need to install `langdetect` in the same Python run environment where Jupyter Notebook is running.\n", + "\n", + "* If a word is English, `langdetect.detect(word)=='en'` will return `True`.\n", + "\n", + "Your output should read:\n", + "\n", + "```'good morning everyone'```" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import langdetect\n", + "from functools import reduce\n", + "words = ['good morning', '早上好', 'доброго', 'おはようございます', 'everyone', '大家', 'каждый', 'みんな']\n", + "\n", + "# Enter your code below" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Bonus Question" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Finally, if you still have time, convert your response in Q2 by using `lambda`, `map`, `filter`, or `reduce` where applicable. Enter your function in the cell below.\n", + "\n", + "As you write Python functions, generally you don't want to make your function too long. Long functions are difficult to read and difficult to debug. If a function is getting too long, consider breaking it into sever shorter functions where the main function calls other functions. If anything goes wrong, you can output debug messages in each of the functions to check where exactly the error is." + ] + }, + { + "cell_type": "code", + "execution_count": 80, + "metadata": {}, + "outputs": [], + "source": [ + "# Enter your code below" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Test your function below with the Bag of Words lab docs (it's easier for you to debug your code). Your output should be:\n", + "\n", + "```{'bag_of_words': ['ironhack', 'cool', 'love', 'student'], 'term_freq': [[1, 1, 0, 0], [1, 0, 1, 0], [1, 0, 0, 1]]}```" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from sklearn.feature_extraction import stop_words\n", + "bow = get_bow_from_docs([\n", + " '../../lab-bag-of-words/your-code/doc1.txt', \n", + " '../../lab-bag-of-words/your-code/doc2.txt', \n", + " '../../lab-bag-of-words/your-code/doc3.txt'],\n", + " stop_words.ENGLISH_STOP_WORDS\n", + ")\n", + "\n", + "print(bow)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.12" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/lab-functional-programming/your-code/.ipynb_checkpoints/main-checkpoint.ipynb b/lab-functional-programming/your-code/.ipynb_checkpoints/main-checkpoint.ipynb new file mode 100644 index 0000000..d87b149 --- /dev/null +++ b/lab-functional-programming/your-code/.ipynb_checkpoints/main-checkpoint.ipynb @@ -0,0 +1,464 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Before your start:\n", + "- Read the README.md file\n", + "- Comment as much as you can and use the resources in the README.md file\n", + "- Happy learning!" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": {}, + "outputs": [], + "source": [ + "import numpy as np\n", + "import pandas as pd" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Challenge 1 - Iterators, Generators and `yield`. \n", + "\n", + "In iterator in Python is an object that represents a stream of data. However, iterators contain a countable number of values. We traverse through the iterator and return one value at a time. All iterators support a `next` function that allows us to traverse through the iterator. We can create an iterator using the `iter` function that comes with the base package of Python. Below is an example of an iterator." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "1\n" + ] + } + ], + "source": [ + "# We first define our iterator:\n", + "\n", + "iterator = iter([1,2,3])\n", + "\n", + "# We can now iterate through the object using the next function\n", + "\n", + "print(next(iterator))" + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "2\n" + ] + } + ], + "source": [ + "# We continue to iterate through the iterator.\n", + "\n", + "print(next(iterator))" + ] + }, + { + "cell_type": "code", + "execution_count": 4, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "3\n" + ] + } + ], + "source": [ + "print(next(iterator))" + ] + }, + { + "cell_type": "code", + "execution_count": 5, + "metadata": {}, + "outputs": [ + { + "ename": "StopIteration", + "evalue": "", + "output_type": "error", + "traceback": [ + "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[1;31mStopIteration\u001b[0m Traceback (most recent call last)", + "\u001b[1;32m\u001b[0m in \u001b[0;36m\u001b[1;34m()\u001b[0m\n\u001b[0;32m 1\u001b[0m \u001b[1;31m# After we have iterated through all elements, we will get a StopIteration Error\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m 2\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 3\u001b[1;33m \u001b[0mprint\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mnext\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0miterator\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m", + "\u001b[1;31mStopIteration\u001b[0m: " + ] + } + ], + "source": [ + "# After we have iterated through all elements, we will get a StopIteration Error\n", + "\n", + "print(next(iterator))" + ] + }, + { + "cell_type": "code", + "execution_count": 6, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "1\n", + "2\n", + "3\n" + ] + } + ], + "source": [ + "# We can also iterate through an iterator using a for loop like this:\n", + "# Note: we cannot go back directly in an iterator once we have traversed through the elements. \n", + "# This is why we are redefining the iterator below\n", + "\n", + "iterator = iter([1,2,3])\n", + "\n", + "for i in iterator:\n", + " print(i)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In the cell below, write a function that takes an iterator and returns the first element in the iterator and returns the first element in the iterator that is divisible by 2. Assume that all iterators contain only numeric data. If we have not found a single element that is divisible by 2, return zero." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": {}, + "outputs": [], + "source": [ + "def divisible2(iterator):\n", + " # This function takes an iterable and returns the first element that is divisible by 2 and zero otherwise\n", + " # Input: Iterable\n", + " # Output: Integer\n", + " \n", + " # Sample Input: iter([1,2,3])\n", + " # Sample Output: 2\n", + " \n", + " # Your code here:\n", + " " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Generators\n", + "\n", + "It is quite difficult to create your own iterator since you would have to implement a `next` function. Generators are functions that enable us to create iterators. The difference between a function and a generator is that instead of using `return`, we use `yield`. For example, below we have a function that returns an iterator containing the numbers 0 through n:" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": {}, + "outputs": [], + "source": [ + "def firstn(n):\n", + " number = 0\n", + " while number < n:\n", + " yield number\n", + " number = number + 1" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "If we pass 5 to the function, we will see that we have a iterator containing the numbers 0 through 4." + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "0\n", + "1\n", + "2\n", + "3\n", + "4\n" + ] + } + ], + "source": [ + "iterator = firstn(5)\n", + "\n", + "for i in iterator:\n", + " print(i)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In the cell below, create a generator that takes a number and returns an iterator containing all even numbers between 0 and the number you passed to the generator." + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": {}, + "outputs": [], + "source": [ + "def even_iterator(n):\n", + " # This function produces an iterator containing all even numbers between 0 and n\n", + " # Input: integer\n", + " # Output: iterator\n", + " \n", + " # Sample Input: 5\n", + " # Sample Output: iter([0, 2, 4])\n", + " \n", + " # Your code here:\n", + " " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Challenge 2 - Applying Functions to DataFrames\n", + "\n", + "In this challenge, we will look at how to transform cells or entire columns at once.\n", + "\n", + "First, let's load a dataset. We will download the famous Iris classification dataset in the cell below." + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": {}, + "outputs": [], + "source": [ + "columns = ['sepal_length', 'sepal_width', 'petal_length','petal_width','iris_type']\n", + "iris = pd.read_csv(\"https://archive.ics.uci.edu/ml/machine-learning-databases/iris/iris.data\", names=columns)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's look at the dataset using the `head` function." + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": {}, + "outputs": [], + "source": [ + "# Your code here:\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's start off by using built-in functions. Try to apply the numpy mean function and describe what happens in the comments of the code." + ] + }, + { + "cell_type": "code", + "execution_count": 16, + "metadata": {}, + "outputs": [], + "source": [ + "# Your code here:\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Next, we'll apply the standard deviation function in numpy (`np.std`). Describe what happened in the comments." + ] + }, + { + "cell_type": "code", + "execution_count": 17, + "metadata": {}, + "outputs": [], + "source": [ + "# Your code here:\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The measurements are in centimeters. Let's convert them all to inches. First, we will create a dataframe that contains only the numeric columns. Assign this new dataframe to `iris_numeric`." + ] + }, + { + "cell_type": "code", + "execution_count": 19, + "metadata": {}, + "outputs": [], + "source": [ + "# Your code here:\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Next, we will write a function that converts centimeters to inches in the cell below. Recall that 1cm = 0.393701in." + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "metadata": {}, + "outputs": [], + "source": [ + "def cm_to_in(x):\n", + " # This function takes in a numeric value in centimeters and converts it to inches\n", + " # Input: numeric value\n", + " # Output: float\n", + " \n", + " # Sample Input: 1.0\n", + " # Sample Output: 0.393701\n", + " \n", + " # Your code here:\n", + " " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Now convert all columns in `iris_numeric` to inches in the cell below. We like to think of functional transformations as immutable. Therefore, save the transformed data in a dataframe called `iris_inch`." + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "metadata": {}, + "outputs": [], + "source": [ + "# Your code here:\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We have just found that the original measurements were off by a constant. Define the global constant `error` and set it to 2. Write a function that uses the global constant and adds it to each cell in the dataframe. Apply this function to `iris_numeric` and save the result in `iris_constant`." + ] + }, + { + "cell_type": "code", + "execution_count": 23, + "metadata": {}, + "outputs": [], + "source": [ + "# Define constant below:\n", + "\n", + "\n", + "def add_constant(x):\n", + " # This function adds a global constant to our input.\n", + " # Input: numeric value\n", + " # Output: numeric value\n", + " \n", + " # Your code here:\n", + " " + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# Bonus Challenge - Applying Functions to Columns\n", + "\n", + "Read more about applying functions to either rows or columns [here](https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.apply.html) and write a function that computes the maximum value for each row of `iris_numeric`" + ] + }, + { + "cell_type": "code", + "execution_count": 24, + "metadata": {}, + "outputs": [], + "source": [ + "# Your code here:\n", + "\n" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Compute the combined lengths for each row and the combined widths for each row using a function. Assign these values to new columns `total_length` and `total_width`." + ] + }, + { + "cell_type": "code", + "execution_count": 25, + "metadata": {}, + "outputs": [], + "source": [ + "# Your code here:\n", + "\n" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3 (ipykernel)", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.9.12" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/lab-functional-programming/your-code/Q1.ipynb b/lab-functional-programming/your-code/Q1.ipynb index 8b07d3d..f584ebd 100644 --- a/lab-functional-programming/your-code/Q1.ipynb +++ b/lab-functional-programming/your-code/Q1.ipynb @@ -19,25 +19,52 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 191, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "\n", + "\n", + "{'bag_of_words': ['ironhack', 'is', 'cool', 'i', 'love', 'am', 'a', 'student', 'at'], 'term_freq': [[1, 1, 1, 0, 0, 0, 0, 0, 0], [1, 0, 0, 1, 1, 0, 0, 0, 0], [1, 0, 0, 1, 0, 1, 1, 1, 1]]}\n" + ] + } + ], "source": [ "# Import required libraries\n", + "import pandas as pd\n", + "import numpy as np\n", + "import os\n", + "import re\n", "\n", + "docs = [i for i in os.listdir('.') if i.endswith('.txt')]\n", "# Define function\n", "def get_bow_from_docs(docs, stop_words=[]):\n", " \n", " # In the function, first define the variables you will use such as `corpus`, `bag_of_words`, and `term_freq`.\n", " \n", + " corpus = []\n", " \n", + " bag_of_words = []\n", + " term_freq = []\n", + " pattern = r'[,\\.\\n\\s]'\n", " \n", " \"\"\"\n", " Loop `docs` and read the content of each doc into a string in `corpus`.\n", " Remember to convert the doc content to lowercases and remove punctuation.\n", " \"\"\"\n", "\n", - " \n", + " for i in docs:\n", + " with open(i,'r') as f:\n", + " lines = f.read()\n", + " corpus1 = re.split(pattern, lines)\n", + " corpus2 = [i.lower() for i in corpus1 if i!=\"\"]\n", + " corpus.append(corpus2) \n", + " \n", + "\n", " \n", " \"\"\"\n", " Loop `corpus`. Append the terms in each doc into the `bag_of_words` array. The terms in `bag_of_words` \n", @@ -46,7 +73,16 @@ " if it is not a stop word.\n", " \"\"\"\n", "\n", - " \n", + " for i in corpus:\n", + " for x in i:\n", + " if x in stop_words:\n", + " print()\n", + " else:\n", + " if x in bag_of_words:\n", + " print()\n", + " else: \n", + " bag_of_words.append(x)\n", + " \n", " \n", " \n", " \"\"\"\n", @@ -54,14 +90,19 @@ " Create an array for each doc's term frequency and append it to `term_freq`.\n", " \"\"\"\n", "\n", - " \n", + " for i in corpus:\n", + " term_freq1=[]\n", + " for x in bag_of_words:\n", + " term_freq1.append(i.count(x))\n", + " term_freq.append(term_freq1)\n", + " \n", " \n", " # Now return your output as an object\n", " return {\n", " \"bag_of_words\": bag_of_words,\n", " \"term_freq\": term_freq\n", " }\n", - " " + "print(get_bow_from_docs(docs, stop_words=[]))" ] }, { @@ -75,12 +116,23 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 192, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "\n", + "\n", + "{'bag_of_words': ['ironhack', 'is', 'cool', 'i', 'love', 'am', 'a', 'student', 'at'], 'term_freq': [[1, 1, 1, 0, 0, 0, 0, 0, 0], [1, 0, 0, 1, 1, 0, 0, 0, 0], [1, 0, 0, 1, 0, 1, 1, 1, 1]]}\n" + ] + } + ], "source": [ "# Define doc paths array\n", - "docs = []\n", + "docs = [i for i in os.listdir('.') if i.endswith('.txt')]\n", "\n", "# Obtain BoW from your function\n", "bow = get_bow_from_docs(docs)\n", @@ -100,12 +152,20 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 186, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "frozenset({'among', 'hence', 'thru', 'found', 'we', 'could', 'yet', 'also', 'top', 'nowhere', 'ltd', 'three', 'almost', 'into', 'describe', 'only', 'his', 'whose', 'us', 'anyone', 'has', 'side', 'still', 'those', 'less', 'fifty', 'so', 'hereupon', 'mostly', 'across', 'on', 'amount', 'was', 'nor', 'ten', 'whoever', 'twelve', 'one', 'myself', 'give', 'few', 'together', 'became', 'have', 'had', 'toward', 'through', 'own', 'take', 'couldnt', 'fill', 'thereupon', 'your', 'afterwards', 'last', 'becomes', 'who', 'nine', 'thus', 'beside', 'yours', 'under', 'all', 'mine', 'that', 'above', 'by', 'me', 'herself', 'be', 'been', 'whither', 'seeming', 'him', 'etc', 'system', 'others', 'cant', 'her', 'towards', 'whence', 'rather', 'behind', 'now', 'next', 'beyond', 'while', 'around', 'may', 'con', 'off', 'else', 'when', 'cannot', 'though', 'hundred', 'another', 'you', 'to', 'elsewhere', 'inc', 'nevertheless', 'themselves', 'if', 'interest', 'between', 'yourselves', 'move', 'please', 'anywhere', 'the', 'or', 'she', 'call', 'put', 'ie', 'bottom', 'down', 'un', 'do', 'there', 'against', 'although', 'hasnt', 'de', 'much', 'anyway', 'of', 'during', 'thereby', 'except', 'therefore', 'eg', 'namely', 'whether', 'i', 'am', 'each', 'himself', 'which', 'over', 'amongst', 'whenever', 'sometime', 'some', 'anything', 'mill', 'from', 'our', 'its', 'formerly', 'these', 'two', 'were', 'everyone', 'my', 'go', 'find', 'see', 'is', 'until', 'always', 'same', 'somehow', 'where', 'everywhere', 'most', 'an', 'in', 'whom', 'sincere', 'eleven', 'therein', 'amoungst', 'someone', 'becoming', 'due', 'very', 'even', 'ever', 'former', 'at', 'per', 'thereafter', 'six', 'itself', 'made', 'forty', 'as', 'he', 'onto', 'with', 'everything', 'never', 'no', 'such', 'nothing', 'how', 'fifteen', 'keep', 'thin', 'here', 'co', 'show', 'several', 'are', 'hers', 'throughout', 'many', 'four', 'they', 'wherein', 'somewhere', 'before', 'well', 'being', 'back', 'hereafter', 'out', 'whereafter', 'a', 'often', 'then', 'whereupon', 'moreover', 'not', 'once', 'via', 'sixty', 'twenty', 'cry', 'along', 'why', 'more', 'wherever', 'hereby', 'empty', 'both', 'it', 'enough', 'whereas', 'upon', 'whereby', 'latterly', 'third', 'detail', 'however', 'full', 'ours', 'serious', 'might', 'already', 'besides', 'neither', 'because', 'least', 'none', 'sometimes', 're', 'them', 'will', 'front', 'below', 'after', 'noone', 'their', 'but', 'become', 'any', 'perhaps', 'should', 'bill', 'this', 'up', 'indeed', 'within', 'anyhow', 'five', 'without', 'other', 'whole', 'herein', 'either', 'alone', 'would', 'further', 'than', 'name', 'first', 'and', 'seem', 'since', 'fire', 'something', 'must', 'yourself', 'seems', 'ourselves', 'beforehand', 'can', 'again', 'thence', 'meanwhile', 'for', 'otherwise', 'seemed', 'nobody', 'too', 'part', 'about', 'latter', 'get', 'eight', 'thick', 'done', 'every', 'what', 'whatever'})\n" + ] + } + ], "source": [ - "from sklearn.feature_extraction import stop_words\n", - "print(stop_words.ENGLISH_STOP_WORDS)" + "from sklearn.feature_extraction import _stop_words\n", + "print(_stop_words.ENGLISH_STOP_WORDS)" ] }, { @@ -128,11 +188,27 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 194, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "\n", + "{'bag_of_words': ['ironhack', 'cool', 'love', 'student'], 'term_freq': [[1, 1, 0, 0], [1, 0, 1, 0], [1, 0, 0, 1]]}\n" + ] + } + ], "source": [ - "bow = get_bow_from_docs(bow, stop_words.ENGLISH_STOP_WORDS)\n", + "bow = get_bow_from_docs(docs, _stop_words.ENGLISH_STOP_WORDS)\n", "\n", "print(bow)" ] @@ -145,18 +221,11 @@ "\n", "```{'bag_of_words': ['ironhack', 'cool', 'love', 'student'], 'term_freq': [[1, 1, 0, 0], [1, 0, 1, 0], [1, 0, 0, 1]]}```" ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [] } ], "metadata": { "kernelspec": { - "display_name": "Python 3", + "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, @@ -170,7 +239,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.6" + "version": "3.9.12" } }, "nbformat": 4, diff --git a/lab-functional-programming/your-code/Q2.ipynb b/lab-functional-programming/your-code/Q2.ipynb index f50f442..8795827 100644 --- a/lab-functional-programming/your-code/Q2.ipynb +++ b/lab-functional-programming/your-code/Q2.ipynb @@ -15,12 +15,35 @@ }, { "cell_type": "code", - "execution_count": 60, + "execution_count": 4, "metadata": {}, - "outputs": [], + "outputs": [ + { + "ename": "IndentationError", + "evalue": "expected an indented block (695300420.py, line 16)", + "output_type": "error", + "traceback": [ + "\u001b[1;36m Input \u001b[1;32mIn [4]\u001b[1;36m\u001b[0m\n\u001b[1;33m \u001b[0m\n\u001b[1;37m ^\u001b[0m\n\u001b[1;31mIndentationError\u001b[0m\u001b[1;31m:\u001b[0m expected an indented block\n" + ] + } + ], "source": [ "# Define your string handling functions below\n", - "# Minimal 3 functions\n" + "# Minimal 3 functions\n", + "import re\n", + "\n", + "def remove_unicode(text):\n", + " pattern = r'\\s'\n", + " words = re.split(pattern, text)\n", + " return words\n", + "\n", + "def remove_punctuation(text):\n", + " pattern = r'[\\.,:;]'\n", + " words = re-split(pattern, text)\n", + " return words\n", + "\n", + "def to_lower_case(text):\n", + " " ] }, { @@ -32,7 +55,7 @@ }, { "cell_type": "code", - "execution_count": 61, + "execution_count": 9, "metadata": {}, "outputs": [], "source": [ @@ -41,8 +64,30 @@ " corpus = []\n", " bag_of_words = []\n", " term_freq = []\n", - " \n", + " pattern = r'[,\\.\\n\\s]'\n", " # write your codes here\n", + " for i in docs:\n", + " with open(i,'r',encoding='utf-8') as f:\n", + " lines = f.read()\n", + " corpus1 = re.split(pattern, lines)\n", + " corpus2 = [i.lower() for i in corpus1 if i!=\"\"]\n", + " corpus.append(corpus2)\n", + " \n", + " for i in corpus:\n", + " for x in i:\n", + " if x in stop_words:\n", + " print()\n", + " else:\n", + " if x in bag_of_words:\n", + " print()\n", + " else: \n", + " bag_of_words.append(x)\n", + " \n", + " for i in corpus:\n", + " term_freq1=[]\n", + " for x in bag_of_words:\n", + " term_freq1.append(i.count(x))\n", + " term_freq.append(term_freq1)\n", " \n", " return {\n", " \"bag_of_words\": bag_of_words,\n", @@ -60,17 +105,29 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 2, "metadata": {}, - "outputs": [], + "outputs": [ + { + "ename": "NameError", + "evalue": "name 'get_bow_from_docs' is not defined", + "output_type": "error", + "traceback": [ + "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[1;31mNameError\u001b[0m Traceback (most recent call last)", + "Input \u001b[1;32mIn [2]\u001b[0m, in \u001b[0;36m\u001b[1;34m()\u001b[0m\n\u001b[0;32m 1\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01msklearn\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mfeature_extraction\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m _stop_words\n\u001b[1;32m----> 2\u001b[0m bow \u001b[38;5;241m=\u001b[39m \u001b[43mget_bow_from_docs\u001b[49m([\n\u001b[0;32m 3\u001b[0m \u001b[38;5;124m'\u001b[39m\u001b[38;5;124mwww.coursereport.com_ironhack.html\u001b[39m\u001b[38;5;124m'\u001b[39m,\n\u001b[0;32m 4\u001b[0m \u001b[38;5;124m'\u001b[39m\u001b[38;5;124men.wikipedia.org_Data_analysis.html\u001b[39m\u001b[38;5;124m'\u001b[39m,\n\u001b[0;32m 5\u001b[0m \u001b[38;5;124m'\u001b[39m\u001b[38;5;124mwww.lipsum.com.html\u001b[39m\u001b[38;5;124m'\u001b[39m\n\u001b[0;32m 6\u001b[0m ],\n\u001b[0;32m 7\u001b[0m _stop_words\u001b[38;5;241m.\u001b[39mENGLISH_STOP_WORDS\n\u001b[0;32m 8\u001b[0m )\n\u001b[0;32m 10\u001b[0m \u001b[38;5;28mprint\u001b[39m(bow)\n", + "\u001b[1;31mNameError\u001b[0m: name 'get_bow_from_docs' is not defined" + ] + } + ], "source": [ - "from sklearn.feature_extraction import stop_words\n", + "from sklearn.feature_extraction import _stop_words\n", "bow = get_bow_from_docs([\n", " 'www.coursereport.com_ironhack.html',\n", " 'en.wikipedia.org_Data_analysis.html',\n", " 'www.lipsum.com.html'\n", " ],\n", - " stop_words.ENGLISH_STOP_WORDS\n", + " _stop_words.ENGLISH_STOP_WORDS\n", ")\n", "\n", "print(bow)" @@ -97,7 +154,7 @@ ], "metadata": { "kernelspec": { - "display_name": "Python 3", + "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, @@ -111,7 +168,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.6" + "version": "3.9.12" } }, "nbformat": 4, diff --git a/lab-functional-programming/your-code/Q3.ipynb b/lab-functional-programming/your-code/Q3.ipynb index 75055ac..8e37fdd 100644 --- a/lab-functional-programming/your-code/Q3.ipynb +++ b/lab-functional-programming/your-code/Q3.ipynb @@ -85,12 +85,21 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 5, "metadata": {}, - "outputs": [], + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "[1, 4, 5]\n" + ] + } + ], "source": [ "numbers = [1, 4, -1, -100, 0, 5, -99]\n", - "\n", + "product = filter(lambda x: x>0, numbers)\n", + "print(list(product))\n", "# Enter your code below" ] }, @@ -113,15 +122,30 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 24, "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/plain": [ + "'good morning everyone'" + ] + }, + "execution_count": 24, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ + "\n", "import langdetect\n", "from functools import reduce\n", "words = ['good morning', '早上好', 'доброго', 'おはようございます', 'everyone', '大家', 'каждый', 'みんな']\n", "\n", - "# Enter your code below" + "# Enter your code below\n", + "phrase = filter(lambda word: langdetect.detect(word)=='en', words)\n", + "finalphrase = reduce(lambda x, y: x + \" \"+ y, phrase)\n", + "finalphrase" ] }, { @@ -160,11 +184,23 @@ }, { "cell_type": "code", - "execution_count": null, + "execution_count": 7, "metadata": {}, - "outputs": [], + "outputs": [ + { + "ename": "NameError", + "evalue": "name 'get_bow_from_docs' is not defined", + "output_type": "error", + "traceback": [ + "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m", + "\u001b[1;31mNameError\u001b[0m Traceback (most recent call last)", + "Input \u001b[1;32mIn [7]\u001b[0m, in \u001b[0;36m\u001b[1;34m()\u001b[0m\n\u001b[0;32m 1\u001b[0m \u001b[38;5;28;01mfrom\u001b[39;00m \u001b[38;5;21;01msklearn\u001b[39;00m\u001b[38;5;21;01m.\u001b[39;00m\u001b[38;5;21;01mfeature_extraction\u001b[39;00m \u001b[38;5;28;01mimport\u001b[39;00m _stop_words\n\u001b[1;32m----> 2\u001b[0m bow \u001b[38;5;241m=\u001b[39m \u001b[43mget_bow_from_docs\u001b[49m([\n\u001b[0;32m 3\u001b[0m \u001b[38;5;124m'\u001b[39m\u001b[38;5;124m../../lab-bag-of-words/your-code/doc1.txt\u001b[39m\u001b[38;5;124m'\u001b[39m, \n\u001b[0;32m 4\u001b[0m \u001b[38;5;124m'\u001b[39m\u001b[38;5;124m../../lab-bag-of-words/your-code/doc2.txt\u001b[39m\u001b[38;5;124m'\u001b[39m, \n\u001b[0;32m 5\u001b[0m \u001b[38;5;124m'\u001b[39m\u001b[38;5;124m../../lab-bag-of-words/your-code/doc3.txt\u001b[39m\u001b[38;5;124m'\u001b[39m],\n\u001b[0;32m 6\u001b[0m stop_words\u001b[38;5;241m.\u001b[39mENGLISH_STOP_WORDS\n\u001b[0;32m 7\u001b[0m )\n\u001b[0;32m 9\u001b[0m \u001b[38;5;28mprint\u001b[39m(bow)\n", + "\u001b[1;31mNameError\u001b[0m: name 'get_bow_from_docs' is not defined" + ] + } + ], "source": [ - "from sklearn.feature_extraction import stop_words\n", + "from sklearn.feature_extraction import _stop_words\n", "bow = get_bow_from_docs([\n", " '../../lab-bag-of-words/your-code/doc1.txt', \n", " '../../lab-bag-of-words/your-code/doc2.txt', \n", @@ -185,7 +221,7 @@ ], "metadata": { "kernelspec": { - "display_name": "Python 3", + "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, @@ -199,7 +235,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.6" + "version": "3.9.12" } }, "nbformat": 4, diff --git a/lab-functional-programming/your-code/main.ipynb b/lab-functional-programming/your-code/main.ipynb index 8017d6e..33d2c1f 100644 --- a/lab-functional-programming/your-code/main.ipynb +++ b/lab-functional-programming/your-code/main.ipynb @@ -146,15 +146,36 @@ }, { "cell_type": "code", - "execution_count": 8, + "execution_count": 28, "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/plain": [ + "2" + ] + }, + "execution_count": 28, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "def divisible2(iterator):\n", " # This function takes an iterable and returns the first element that is divisible by 2 and zero otherwise\n", " # Input: Iterable\n", " # Output: Integer\n", - " \n", + " lst = []\n", + " for i in iterator:\n", + " if i%2 == 0:\n", + " return i\n", + " \n", + " \n", + " \n", + "\n", + "iterator = iter([1,2,3,4])\n", + "divisible2(iterator) \n", + "\n", " # Sample Input: iter([1,2,3])\n", " # Sample Output: 2\n", " \n", @@ -253,7 +274,7 @@ }, { "cell_type": "code", - "execution_count": 14, + "execution_count": 67, "metadata": {}, "outputs": [], "source": [ @@ -270,12 +291,99 @@ }, { "cell_type": "code", - "execution_count": 15, + "execution_count": 39, "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
sepal_lengthsepal_widthpetal_lengthpetal_widthiris_type
05.13.51.40.2Iris-setosa
14.93.01.40.2Iris-setosa
24.73.21.30.2Iris-setosa
34.63.11.50.2Iris-setosa
45.03.61.40.2Iris-setosa
\n", + "
" + ], + "text/plain": [ + " sepal_length sepal_width petal_length petal_width iris_type\n", + "0 5.1 3.5 1.4 0.2 Iris-setosa\n", + "1 4.9 3.0 1.4 0.2 Iris-setosa\n", + "2 4.7 3.2 1.3 0.2 Iris-setosa\n", + "3 4.6 3.1 1.5 0.2 Iris-setosa\n", + "4 5.0 3.6 1.4 0.2 Iris-setosa" + ] + }, + "execution_count": 39, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "# Your code here:\n", - "\n" + "iris.head()" ] }, { @@ -287,12 +395,27 @@ }, { "cell_type": "code", - "execution_count": 16, + "execution_count": 44, "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/plain": [ + "sepal_length 5.843333\n", + "sepal_width 3.054000\n", + "petal_length 3.758667\n", + "petal_width 1.198667\n", + "dtype: float64" + ] + }, + "execution_count": 44, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "# Your code here:\n", - "\n" + "np.mean(iris)\n" ] }, { @@ -304,12 +427,27 @@ }, { "cell_type": "code", - "execution_count": 17, + "execution_count": 46, "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/plain": [ + "sepal_length 0.825301\n", + "sepal_width 0.432147\n", + "petal_length 1.758529\n", + "petal_width 0.760613\n", + "dtype: float64" + ] + }, + "execution_count": 46, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "# Your code here:\n", - "\n" + "np.std(iris)\n" ] }, { @@ -321,12 +459,145 @@ }, { "cell_type": "code", - "execution_count": 19, + "execution_count": 90, "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
sepal_lengthsepal_widthpetal_lengthpetal_width
05.13.51.40.2
14.93.01.40.2
24.73.21.30.2
34.63.11.50.2
45.03.61.40.2
...............
1456.73.05.22.3
1466.32.55.01.9
1476.53.05.22.0
1486.23.45.42.3
1495.93.05.11.8
\n", + "

150 rows × 4 columns

\n", + "
" + ], + "text/plain": [ + " sepal_length sepal_width petal_length petal_width\n", + "0 5.1 3.5 1.4 0.2\n", + "1 4.9 3.0 1.4 0.2\n", + "2 4.7 3.2 1.3 0.2\n", + "3 4.6 3.1 1.5 0.2\n", + "4 5.0 3.6 1.4 0.2\n", + ".. ... ... ... ...\n", + "145 6.7 3.0 5.2 2.3\n", + "146 6.3 2.5 5.0 1.9\n", + "147 6.5 3.0 5.2 2.0\n", + "148 6.2 3.4 5.4 2.3\n", + "149 5.9 3.0 5.1 1.8\n", + "\n", + "[150 rows x 4 columns]" + ] + }, + "execution_count": 90, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "# Your code here:\n", - "\n" + "iris_numeric = iris.loc[:, iris.columns != 'iris_type']\n", + "iris_numeric\n" ] }, { @@ -338,18 +609,30 @@ }, { "cell_type": "code", - "execution_count": 21, + "execution_count": 80, "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/plain": [ + "0.393701" + ] + }, + "execution_count": 80, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "def cm_to_in(x):\n", " # This function takes in a numeric value in centimeters and converts it to inches\n", " # Input: numeric value\n", " # Output: float\n", - " \n", + " inches = x*0.393701\n", + " return inches\n", " # Sample Input: 1.0\n", " # Sample Output: 0.393701\n", - " \n", + "cm_to_in(1)\n", " # Your code here:\n", " " ] @@ -363,12 +646,146 @@ }, { "cell_type": "code", - "execution_count": 22, + "execution_count": 94, "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
sepal_lengthsepal_widthpetal_lengthpetal_width
02.0078751.3779540.5511810.078740
11.9291351.1811030.5511810.078740
21.8503951.2598430.5118110.078740
31.8110251.2204730.5905520.078740
41.9685051.4173240.5511810.078740
...............
1452.6377971.1811032.0472450.905512
1462.4803160.9842531.9685050.748032
1472.5590571.1811032.0472450.787402
1482.4409461.3385832.1259850.905512
1492.3228361.1811032.0078750.708662
\n", + "

150 rows × 4 columns

\n", + "
" + ], + "text/plain": [ + " sepal_length sepal_width petal_length petal_width\n", + "0 2.007875 1.377954 0.551181 0.078740\n", + "1 1.929135 1.181103 0.551181 0.078740\n", + "2 1.850395 1.259843 0.511811 0.078740\n", + "3 1.811025 1.220473 0.590552 0.078740\n", + "4 1.968505 1.417324 0.551181 0.078740\n", + ".. ... ... ... ...\n", + "145 2.637797 1.181103 2.047245 0.905512\n", + "146 2.480316 0.984253 1.968505 0.748032\n", + "147 2.559057 1.181103 2.047245 0.787402\n", + "148 2.440946 1.338583 2.125985 0.905512\n", + "149 2.322836 1.181103 2.007875 0.708662\n", + "\n", + "[150 rows x 4 columns]" + ] + }, + "execution_count": 94, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "# Your code here:\n", - "\n" + "iris_inch = cm_to_in(iris_numeric)\n", + "iris_inch\n", + " " ] }, { @@ -380,20 +797,158 @@ }, { "cell_type": "code", - "execution_count": 23, + "execution_count": 101, "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
sepal_lengthsepal_widthpetal_lengthpetal_width
07.15.53.42.2
16.95.03.42.2
26.75.23.32.2
36.65.13.52.2
47.05.63.42.2
...............
1458.75.07.24.3
1468.34.57.03.9
1478.55.07.24.0
1488.25.47.44.3
1497.95.07.13.8
\n", + "

150 rows × 4 columns

\n", + "
" + ], + "text/plain": [ + " sepal_length sepal_width petal_length petal_width\n", + "0 7.1 5.5 3.4 2.2\n", + "1 6.9 5.0 3.4 2.2\n", + "2 6.7 5.2 3.3 2.2\n", + "3 6.6 5.1 3.5 2.2\n", + "4 7.0 5.6 3.4 2.2\n", + ".. ... ... ... ...\n", + "145 8.7 5.0 7.2 4.3\n", + "146 8.3 4.5 7.0 3.9\n", + "147 8.5 5.0 7.2 4.0\n", + "148 8.2 5.4 7.4 4.3\n", + "149 7.9 5.0 7.1 3.8\n", + "\n", + "[150 rows x 4 columns]" + ] + }, + "execution_count": 101, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "# Define constant below:\n", "\n", "\n", "def add_constant(x):\n", + " global error \n", + " error = 2\n", + " value = x+2\n", + " return value\n", " # This function adds a global constant to our input.\n", " # Input: numeric value\n", " # Output: numeric value\n", - " \n", + "add_constant(3)\n", " # Your code here:\n", - " " + "\n", + "iris_constant = add_constant(iris_numeric)\n", + "iris_constant" ] }, { @@ -407,12 +962,28 @@ }, { "cell_type": "code", - "execution_count": 24, + "execution_count": 105, "metadata": {}, - "outputs": [], + "outputs": [ + { + "data": { + "text/plain": [ + "sepal_length 7.9\n", + "sepal_width 4.4\n", + "petal_length 6.9\n", + "petal_width 2.5\n", + "dtype: float64" + ] + }, + "execution_count": 105, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ "# Your code here:\n", - "\n" + "iris_numeric.max()\n", + "iris_numeric.apply(np.max)\n" ] }, { @@ -424,12 +995,199 @@ }, { "cell_type": "code", - "execution_count": 25, + "execution_count": 121, "metadata": {}, "outputs": [], "source": [ "# Your code here:\n", - "\n" + "iris_numeric['total length'] = iris_numeric.loc[:, ['sepal_length','petal_length']].sum(axis=1)" + ] + }, + { + "cell_type": "code", + "execution_count": 122, + "metadata": {}, + "outputs": [], + "source": [ + "iris_numeric['total width'] = iris_numeric.loc[:, ['sepal_width','petal_width']].sum(axis=1)" + ] + }, + { + "cell_type": "code", + "execution_count": 123, + "metadata": {}, + "outputs": [ + { + "data": { + "text/html": [ + "
\n", + "\n", + "\n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + " \n", + "
sepal_lengthsepal_widthpetal_lengthpetal_widthtotal lengthtotal width
05.13.51.40.26.53.7
14.93.01.40.26.33.2
24.73.21.30.26.03.4
34.63.11.50.26.13.3
45.03.61.40.26.43.8
.....................
1456.73.05.22.311.95.3
1466.32.55.01.911.34.4
1476.53.05.22.011.75.0
1486.23.45.42.311.65.7
1495.93.05.11.811.04.8
\n", + "

150 rows × 6 columns

\n", + "
" + ], + "text/plain": [ + " sepal_length sepal_width petal_length petal_width total length \\\n", + "0 5.1 3.5 1.4 0.2 6.5 \n", + "1 4.9 3.0 1.4 0.2 6.3 \n", + "2 4.7 3.2 1.3 0.2 6.0 \n", + "3 4.6 3.1 1.5 0.2 6.1 \n", + "4 5.0 3.6 1.4 0.2 6.4 \n", + ".. ... ... ... ... ... \n", + "145 6.7 3.0 5.2 2.3 11.9 \n", + "146 6.3 2.5 5.0 1.9 11.3 \n", + "147 6.5 3.0 5.2 2.0 11.7 \n", + "148 6.2 3.4 5.4 2.3 11.6 \n", + "149 5.9 3.0 5.1 1.8 11.0 \n", + "\n", + " total width \n", + "0 3.7 \n", + "1 3.2 \n", + "2 3.4 \n", + "3 3.3 \n", + "4 3.8 \n", + ".. ... \n", + "145 5.3 \n", + "146 4.4 \n", + "147 5.0 \n", + "148 5.7 \n", + "149 4.8 \n", + "\n", + "[150 rows x 6 columns]" + ] + }, + "execution_count": 123, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "iris_numeric" ] }, { @@ -442,7 +1200,7 @@ ], "metadata": { "kernelspec": { - "display_name": "Python 3", + "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, @@ -456,7 +1214,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.6.6" + "version": "3.9.12" } }, "nbformat": 4,