As an example yt-dlp has an option to record all successful downloads in an 'archive 'file'. When the file is used, matching videos won't be redownloaded.
--download-archive FILE
Download only videos not listed in the archive file. Record the IDs of all downloaded videos in it
Is there any similar functionality in the script? I have a list of URLs (with around 10,000 lines, one URL per line) and there's no immediate benefit in saving them again to Wayback Machine. I do keep actively adding new URLs` — frequently reprocessing this much pages would be detrimental.
I know that the script operations are logged.
As an example yt-dlp has an option to record all successful downloads in an 'archive 'file'. When the file is used, matching videos won't be redownloaded.
Is there any similar functionality in the script? I have a list of URLs (with around 10,000 lines, one URL per line) and there's no immediate benefit in saving them again to Wayback Machine. I do keep actively adding new URLs` — frequently reprocessing this much pages would be detrimental.
I know that the script operations are logged.