This downloader is designed for data hoarders who want to archive DOJ Epstein disclosure datasets. It automatically downloads all files in a dataset and saves them to your output directory. (This could be further automated to keep running for longer periods of time. Would love your contributions)
- Download all files from a specified dataset.
- Start from a specific page if you’ve already downloaded some files.
- Skips already downloaded files and continues where you left off.
- Works even if a previous run was interrupted; just provide the last working page.
- You might have heard that these files are just random pictures and emails. NOPE
- Datasets 10 and beyond have everything that actually matters
- And these datasets are really large.
git clone git@github.com:usmanx03/epstein_doj_downloader_python.git
cd epstein_doj_downloader_python
Make sure Python 3 is installed. Then install required packages:
pip install selenium webdriver-manager
Specify the dataset number and optional starting page:
python main.py --dataset 6
Start from a specific page if some files were already downloaded:
python main.py --dataset 6 --start-page 3
- Files are saved in the
outputfolder by default. - If a download fails due to website behavior, simply rerun the command. The downloader will skip existing files and continue.
- Provide the last page that was working fine if many pages are already downloaded.
- This ensures the downloader resumes smoothly without repeating downloads.
Notice: This downloader is provided for personal archiving and research purposes only. The author is not responsible for any misuse, legal consequences, or website changes that may affect its functionality. Use at your own risk and respect all applicable laws and terms of service of the websites you access.