The University of Chicago
MACS 30122, Winter 2022
Final Project by Group "None"
Group Members:
    Chongyu Fang
    Feihong Lei
    Zhennuo (Juno) Wu
    Jiachen (Coco) Yu

Goal of our project:
    Despite the fact that neighborhood security represents an important factor
    in individuals’ choice of housing, average rental websites do not reflect
    crime rates information, probably due to a conflict of interests with landlords.
    The main purpose of this project is to provide an aggregated rental information
    in Chicago City incorporating both rental information and past neighborhood crime
    rates in 2021 for future tenants. The final output will be an interactive map
    where users can click on geographical points to access both crime rates and
    housing features including bedroom numbers, bathroom numbers, price range,
    amenities and other features. With the aggregated information and interactive
    map, we wish to provide a user-friendly interface to facilitate future tenants’
    decision-making on housing.

Packages Used and Versions:
    requests: 2.27.1
    bs4: 4.10.0
    csv: 0.14.1
    json: 0.1.1
    math: 1.2.1
    numpy: 1.20.1
    pandas: 1.4.1
    folium: 0.12.1.post1

    Conda version: 4.11.0 (Python version: 3.8.12)

How To Run Our Files:

    List of files/directories:

        [0] README.txt: This README file
        [1] crawler_apt.py: Python script
        [2] apartments.csv: dataset, generated by [1]
        [3] crawler_domu.py: Python script
        [4] domu.csv: dataset, generated by [3]
        [5] record_linkage.ipynb: Jupyter Notebook file
        [6] combined_housing.csv: dataset, generated by [5]
        [7] crimes.csv: dataset, downloaded from Chicago Open Data Portal
        [8] heatmap.ipynb: : Jupyter Notebook file
        [9] heatmap.html: HTML file, the final heatmap output, generated by [8]
        [10] /slides: directory to our slides files (TeX and PDF)
        [11] /video: directory to our video (Video link in /video/README.txt)



    crawler_apt.py:
        *** No need to rerun the file ***
        This is the crawler for apartments.com. This file generates
        the data file apartments.csv used for later analysis.
        If you would like to replicate the process, you can open a
        terminal and change to current directory, then type
        "python3 crawler_apt.py"

    crawler_domu.py:
        *** No need to rerun the file ***
        This is the crawler for domu.com. This file generates the
        data file domu.csv used for later analysis.
        If you would like to replicate the process, you can open a
        terminal and change to current directory, then type
        "python3 crawler_domu.py"

    record_linkage.ipynb:
        *** Rerun this file ***
        We use record linkage in this file and processed apartments.csv
        and domu.csv by identifying same properties and combine the two
        files into a aggregated dataset combined_housing.csv.
        Open this Jupyter Notebook and execute the lines. We choose to
        write the record linkage algorithm in a notebook rather then a
        .py file so that you can observe how we cleaned the two datasets.
        After executing this file, the combined_housing.csv dataset is
        generated, which we will use in later visaulization.

    heatmap.ipynb:
        *** Rerun this file ***
        In this file, we first read the crimes.csv data and computed the
        crime weights score. Then we read the combined_housing.csv dataset
        and cleaned it.
        The final step is to visualize the crime in the map, and add all
        housing information on the map. Each dot represents a property/unit.
        By clicking it, an info box pops out and you can see the detailed
        housing info in it.
        Open this Jupyter Notebook and execute the lines. In addition to
        the notebook outputs, an HTML document heatmap.html would be
        generated.