Traffic duplication removal (CIC-IDS-2017)

[lisa-lthorrold]

This is a more refined version of the previously closed pull request by @mlavin. #1

The main improvement over the previous script is a reduction unintended consequences of applying editcap packet deduplication over the entire pcap. This is done by isolating the impacted mac addresses and ip addresses that have been confirmed to have definite duplication by prior analysis.

[Paippi]

The wait in this code doesn't work as intended. Culprit being the remove_traffic_duplication.sh:47, which spawns a subshell causing the while loop to be performed in another process, thus leading to parent process not waiting for the process to finish.

More info here: https://mywiki.wooledge.org/BashFAQ/024
and here: https://stackoverflow.com/a/63489913/17897224

One way to fix this is to use a for loop instead.

# <---snip--->
    # Find pcap files
    for line in $(find "$1" -iname "*.pcap")
    do
# <---snip--->