Common Crawl Quick Scripts

This repository contains a number of useful scripts for attacking the CommonCrawl dataset and WARC/WET/WAT files.

License

MIT License, as per LICENSE

Name		Name	Last commit message	Last commit date
Latest commit History 41 Commits
ensure_generated		ensure_generated
list_crawl		list_crawl
s3_speedtest		s3_speedtest
s3_to_azure		s3_to_azure
set_acl		set_acl
LICENSE		LICENSE
README.md		README.md
count_responses.sh		count_responses.sh