r/learnpython • • Nov 29 '20

Python script to be automatically executed once per day

Hi all,

I have a very simple script that it scrapes data from a website. Ideally, I need to be executed once per day. Do you have any idea how could I achieve that? As it isn't something official, I need something for free.

Thanks!
PS I mean something online. I know about cron tabs etc.

322 Upvotes

120 comments sorted by

View all comments

5

u/marcus-luck Nov 29 '20

I'm surprised it's not mentioned yet but you can use the module schedule by Dan Bader: https://github.com/dbader/schedule

Pros: -you can schedule the python task in whatever manner you like. -easy to use -python module solution Con: -python script need to be running all the time and restarted if it fails.

Schedule combined with docker has been my go-to every time. A simple docker container set to auto restart. then it runs your script on startup and keeps it running "forever".

I found a container on my server a few months back that i forgot. It had been scraping traffic accident data for over a year without me remembering it. so the solution is stable over time.