I recently set up regularly scheduled scraping of Personal Data via Selenium (for scraping) and Docker and Prefect (for running regularly scheduled pipelines in the background).

More details coming soon!