A Google Cloud Run job to periodically download the Colorado voter registration totals and add them to a timeseries in Google BigQuery.
The Colorado Secreatry of State's office publishes a monthly report on the voter registration totals for different districts within the state.
This repository creates a Google Cloud Run job that can be run on a schedule or on demand to retrieve the Secretary of State data and upload it your Google BigQuery instance as an entry in a continuous timeseries.
These instructions will get your copy of the project up and running on your local machine for development and testing purposes.
This repository depends on certain user defined variables that need to be configured in two places: config.py
& .env
config.py
will generally contain the latest values that I am using to run this repository. A careful read and update to your own values should be all that's needed to update this file. The variables contained here are mostly file and project names.
This repository includes a template for .env
as the file .env.template
. If you are attempting to run this repository locally and not as a Google Cloud Run job you will need to modify .env.template
to contain your secret variables such as the strings for your Google Service Account credentials. Once you have done so, save .env.template
as .env
This repository is meant to be run as a Google Cloud Run job. That means you will need your own Google Cloud login and several Google Cloud services enabled.
- Google Cloud Run
- Google Big Query
- Google Cloud Scheduler
This repository uploads this information to our Google BigQuery. If you do not have your own BigQuery instance it won't work.
- Clone this repository to your local machine.
git clone https://github.com/bengen343/co-registration-timeseries.git
-
Update variables within
config.py
to name your outputs and correctly reference file origins/destinations. -
Update
.env.template
with values for each of the variables requested. Note that the variableBQ_ACCOUNT_CREDS
should be the entire output of the json credential file associated with the Google service account you're using for this project. Save this file as.env
when you've added all your values. -
Create a virtual environment for the repository, probably by running
python3 -m venv .venv
in the terminal from the local directory. -
Activate the virtual environment, probably by running
.venv\Scripts\activate.bat
on Windows orsource .venv/bin/activate
on Mac. -
Install the required dependencies by running
pip install -r requirements.txt
This is meant to be deployed as a Google Cloud Run Job. To build the package into Google Container Registry you'll need to run the command: gcloud builds submit --pack image=gcr.io/{PROJECT_ID}/co-registration-timeseries