How to automate a Python script (or anything else) in Windows and Linux

A common task I need to perform both at work and on my personal/home computers is configuring a script to run automatically on a schedule. Some examples from my experience include running a script nightly to update an application database with new data from an external source, or running a weekly script on my laptop to remove temporary and downloaded files older than 30 days (I keep a tidy house). These are simple and boring tasks, but are crucial for my day-to-day operations. It would be a minor, but easily forgettable, nuisance to manually execute the script every day, so this is the perfect scenario for task automation! In this post I will show how to configure the automatic execution of a Python script (or any other task you want to perform) using Task Scheduler in Windows or cron in Linux and other Unix-like operating systems.

I’ll start with Windows Task Scheduler as the first example; jump right to Linux cron jobs if this isn’t applicable to you. I am a user of the ArcGIS Pro and ArcGIS Online software, and a particular integration between these two products results in a lot of files with “.pitem” and “.pitemx” suffixes in the Downloads folder.

I like to delete all those once a day using a small script (view it here). In Windows, find and run Task Scheduler

On the left hand panel, navigate to Task Scheduler (Local) > Task Scheduler Library. There will likely be existing entries here, including built-in scheduled tasks from Microsoft, and tasks from other software vendors like Adobe, Mozilla, Esri (as shown above). These are typically added as part of the installation process for their products. Users can add their own as well – right click “Task Scheduler (Local)” and select “Create Task…”

On the “General” tab, give the task a name and add a short description for others or your future self to remember why it was added 🙂 Also note the “Run only when user is logged on” option here. I am leaving this as default, and as the option implies, the task will only run when I am logged into the computer. It will not run if another user is logged in, or if I am logged out. That scenario is acceptable for my use case, but there may be situations where the task needs to run even when no user is logged in, like a web or database server for example. There are additional security and permission considerations here, which are beyond the scope of this example so be sure to read the docs on using Task Scheduler.

On the “Triggers” tab, click the “New” button to add a new trigger. A trigger is a condition that, when met, will allow the task to be executed. For this example, I will leave the default “On a schedule” option, starting it on the day I created the task, at 9:00 AM, and recurring every day thereafter.

Multiple triggers can be added to a task so that different conditions will cause it to be executed. Triggers do not have to be schedule-based, tasks can be triggered at logon, during computer idle, after an event, etc. For the large majority of cases I’ve needed, a simple scheduled trigger works just fine.

The next step is to add an action – the actual task or tasks that will be executed. Click the “Actions” tab, to add a new action. Since my task is to run a Python script, I selected the “Run a program” option, set the location of the python.exe interpreter to use in the “Program/script” input field, and the location of the .py script in the “Add arguments (optional)” input field. This effectively runs my script by executing “python.exe” with the location of my script as an argument, but of course any script/program/command/etc can be run here.

The options on the next tab, “Conditions,” can be left as-is, but I prefer to uncheck the “Start the task only of the computer is on AC power” since my script runs in just a few seconds. Some long-running tasks may not complete before battery power ran out, and this options would prevent that.

I left all the default options on the “Settings” tab

Click “OK” to complete the new task and that’s it! When the trigger condition is met, the script will run; you may briefly see a command line window when it does. I am able to confirm this by selecting my task in the Task Scheduler, and looking at the “History” tab to see each time the task was triggered and which actions were executed.

Linux cron jobs

Linux and Unix-like operating systems also have a similar feature for automating and scheduling tasks: the cron utility. It is command line based, and much less feature-rich, but it gets the job done just fine. In this example, I will use my folder cleanup script, which looks in each specified folder, and deletes files older than the specified number of days. A typical cron job is added to the config file, crontab, and the entry is formatted like this:

# ┌-------- min (0 - 59)
# | ┌-------- hour (0 - 23)
# | | ┌-------- day of month (1 - 31)
# | | | ┌-------- month (1 - 12)
# | | | | ┌-------- day of week (0 - 6)
# | | | | | 
  * * * * * command to execute

Each asterisk * represents one of the specific time variables noted above it, allowing the user to specify when a command will be executed. For example, 0 * * * * command to execute would run the task every hour (or, as read from the format above, run the task on minute 0, of every hour, on every day of the month, on every month, on every day of the week). To run a job at 1:00 PM every Tuesday, the format would be: 0 13 * * 2 command to execute.

To add a cron job, use the crontab -e command and open with your preferred editor. There are some useful comments at the top, and you can add jobs below, one per line.

I want this job to run only on weekdays at 10:00 AM – times I will likely be logged into the computer and using it. I added this line to the config file: 0 10 * * 1-5 python3 /home/matt/scripts/folder_backup.py

Your local python configuration may vary from mine, and the command to invoke the Python interpreter may be different, or you may need to specify a specific interpreter if you have multiple installed. And of course, like the previous example, any command can be run here to accomplish any kind of task automation. Once the entry has been added to the config file, save it, exit the editor, and that’s it! If this is the first time setting up a cron job, you may see something like this:

Otherwise just “installing new crontab” will be displayed. Use command crontab -l to just display the contents of the config file, without editing it. Useful for quickly seeing what jobs are scheduled on a given machine.

The history of executing cron jobs can be a bit cumbersome to view, as it is all found in /var/log/syslog. To make things easier, a separate “cron.log” file can be enabled. Edit this file: “etc/rsyslog.d/50-default.conf”

Uncomment the line starting with “cron.*” and save the file.

Next, create that log file, which can be done with command sudo touch /var/log/cron.log. And finally, restart the rsyslog service with command sudo systemctl restart rsyslog.

As in the Windows Task Scheduler example above, this is also just a simple usage of cron job, and they can be used in many more ways. Here are some additional links I found useful for exploring more about configuring scheduled tasks in Windows and cron jobs in *nix:

• Windows Task Scheduler – Wikipedia
• About the Task Scheduler – Microsoft
• How to create an automated task using Task Scheduler on Windows 10 – Windows Central
• cron – Arch Linux Wiki
• A Beginners Guide To Cron Jobs – OSTechNix
• How to Set Up a Cron Job in Linux – phoenixNAP Support
• crontab guru – “The quick and simple editor for cron schedule expressions”
• How to check cron logs in Linux – Linuxhint