Logging Virtual Race Results with Selenium, Cron and Python

March 29, 2021

Who else hates doing repetitive tasks that you know could be automated? I know I do 🙋‍♀️. This is part two about how I used cron jobs and python to take care of ‘the boring stuff’ (link to Part 1). In this article I will discuss how I used Strava’s API and cron jobs to automatically post new activities for virtual races on RunSignup.

Check out the 30 second video above to see it in action.

Link to the github repository.

The Boring Task to Be Automated:

Coronavirus caused all of us active outdoorsy folks to get creative about how we could continue our races. The virtual Circumpolar Race Across the World (CRAW for short) was the one of the results. For this race you have to track all of your hikes/bikes/runs/etc and then log them online via RunSignup.

For each CRAW Activity I would:

  1. Use the Strava app to track the distance of my activity.
  2. Enter the activity on RunSignup using the following form:

It’s not a very long or complicated process, but I couldn’t get over the feeling that it was one step too long. If Strava was already keeping track of the activities why couldn’t I just have Strava tell RunSignup what I was doing instead of entering it myself?

Automating My CRAW Activity Tracking

The Tools I Used to Automate this Task:

The Python Script

In this section I’ll break down the various functions that come together to make up the python program that downloads the activities from Strava and uploads them to CRAW. They are all executed with a little bit of python glue when the file crawActivities.py is run.

get_tokens()

This function handles accessing the app’s tokens.json file, which provides access to the Strava API. It checks to see if the token is expired, and calls refresh_token() to get a new token if it is.

I recommend this tutorial for a quick and dirty way to create a Strava API application and get an initial token.

get_page_of_activities()

This function makes a get request to Strava’s ‘activities’ endpoint, which returns a json object consisting of an array of SummaryActivity objects. There is no specific definition for this endpoint in the documentation, but it works similarly to the “List Athlete Activities” endpoint. The function uses the ‘&after=’ parameter to request only activities that were uploaded after the last time cron ran the program (this time is stored in the .env file and updated at the end of the program).

print_to_csv()

The simplest way to upload all of my activities using selenium is to upload a CSV of all the data. This involved two button clicks as opposed to five plus if Selenium had to enter each value for each activity by ‘hand’.

If the json object returned by Strava is not empty and there are new activities to upload, this function handles putting the data in a the CSV. It takes the json object returned by Strava and uses csv.DictWriter to write the date, distance and type of activity out to a CSV (ignoring all the other data points). This function also reformats this data a bit to match the format required by RunSignup.

upload_csv()

Here’s where the magic happens. Selenium is essentially a series of finding an element via various methods (your web browser’s developer tools will be helpful for this) and then doing something with the element. This function opens up a Chrome web browser instance, navigates to the CRAW activity upload page, authenticates, uploads the activity CSV and checks to see if the upload was successful.

I think it’d be helpful to see the actual code, I’ve added lots of comments to help clarify what each bit is doing:

I had to download ChromeDriver in order for this to work.

update_last_upload_date()

This function runs at the end of the program and it updates the date of last upload in .env file to the current date and time. By keeping track of when the upload last ran, I make sure to only upload new activities.

main()

This is the code that ties it all together:

The Cron Job

My cron job executes the above python script everyday at 9:01 am, provided that my laptop is on and active. Setting the cron job up is quite simple. Merely run ‘crontab -e’ in your command prompt and then add the following line of code to the file:

Let’s break down what that line is saying:

The Bash Script

Let’s talk a bit more about that bash script cron runs. It’s a simple script that activates the virtual environment, runs crawActivities.py (the python script), and then deactivates the virtual environment. This script also tells the crontab where to find my ChromeDriver download. This is what the script looks like:

Reflection

Setting up Selenium and making sure it could access my ChromeDriver download was a bit finicky, but all in all this was a fun project that made me feel like a wizard and saves me approximately five minutes every night.