Skip to content
Closed
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 14 additions & 34 deletions script.sh
Original file line number Diff line number Diff line change
Expand Up @@ -20,50 +20,28 @@ echo ""


# 0. Tell me who worked on this together
echo "student 1" # please fill in names here
echo "student 2"
echo "Nicola Foglia" # please fill in names here
echo "Giovanni Mannucci"
echo "Matteo Maria Morgia"

# here is a list of tasks for you.
# whenever a line says "don't touch" then you are not supposed to touch what comes below.
# all lines where you need to take action are numbered.

# 1. Go to your home directory:

cd

# 2. from your home, creating a directory structure: new folder `programming-hw`, and inside that folder create folder `hw1`
mkdir -p programming-hw/hw1

# 3. go into that new directory, i.e. into ~/programming-hw/hw1

# checking the folder exists now
# don't touch
[ -d ~/programming-hw/hw1 ] && echo "directory created successfully" || exit 1

# download with wget if file does not exist yet
# if wget does not work for you, manually download from the below URL and place into `~/programming-hw/hw1` as `movies.dat`
# first let me check whether you have wget installed and install it otherwise for you:
# don't touch
command -v wget || { [[ "$OSTYPE" == "darwin"* ]] && brew install wget || sudo apt-get install -y wget; }
echo ""
# don't touch
if [ ! -f movies.dat ]; then
echo "File not found!"
echo "will download now to current directory now\n"
echo ""
wget https://raw.githubusercontent.com/sidooms/MovieTweetings/44c525d0c766944910686c60697203cda39305d6/snapshots/10K/movies.dat -O./movies.dat
fi

# check file exists now
# don't touch
if [ ! -f movies.dat ]; then
echo "File not found! Error."
exit 1
fi
cd programming-hw/hw1

# 4. look at first 4 rows of downloaded data in `movies.dat`. look at this output and try to understand how it is structured. the file ending is `dat`. however, how could you also denominate such a file?

# three variables separated by ::. the file could also be a txt

# 5. look at first 4 rows of downloaded data in `movies.dat` redirect to a file called `first4.txt`

head -4 movies.dat > first4.txt

# don't touch
if [ ! -f first4.txt ]; then
Expand Down Expand Up @@ -92,32 +70,34 @@ echo ""
# fill in for _filename_ the correct file you want to operate on.
# then remove the # character from the start of the line and look at the result

# awk -F '::' '{print $3}' _filename_
awk -F '::' '{print $3}' movies.dat

# 7. observe that the `{print $3}` part prints the third field. that looks like: genre1|genre2
# that is, there is *another* separator in this column, `|`. Let's separate again. copy your command from above and
# add a pipe as follows. here, the second statement will split at `|` and print into *how many parts* it has split.
# i.e. it will tell us *how many genres* that movie belonged to. No need to understand the `awk` part.
# again, remove the # below, fill in for _filename_ and run

# awk -F '::' '{print $3}' _filename_ | awk '{print split($0, a, "\\|")}'
awk -F '::' '{print $3}' movies.dat | awk '{print split($0, a, "\\|")}'


# 8. finish the pipeline by adding 2 commands, exactly like in class, that will produce a contingency table
# we want to know how many movies belong to 0,1,2,... etc genres.

# awk -F '::' '{print $3}' _filename_ | awk '{print split($0, a, "\\|")}' | sort | uniq -c
awk -F '::' '{print $3}' movies.dat | awk '{print split($0, a, "\\|")}' | sort | uniq -c

# 9. redirect (>) the output of your pipeline to a file `outtable.txt` in the current directory

awk -F '::' '{print $3}' movies.dat | awk '{print split($0, a, "\\|")}' | sort | uniq -c > outtable.txt


# dont touch
echo "" # leave this untouched
echo "here is my table:" # this as well

# 10. Print your table to screen


cat outtable.txt

#### End of your tasks
# please do not modify the below lines
Expand Down
Loading