Skip to content

A Google Cloud Run job to periodically snapshot the Colorado voter registration district totals to a BigQuery timeseries database.

Notifications You must be signed in to change notification settings

bengen343/co-registration-timeseries

Repository files navigation

Project logo

Colorado Voter Registration Timeseries


A Google Cloud Run job to periodically download the Colorado voter registration totals and add them to a timeseries in Google BigQuery.

Table of Contents

About

The Colorado Secreatry of State's office publishes a monthly report on the voter registration totals for different districts within the state.

This repository creates a Google Cloud Run job that can be run on a schedule or on demand to retrieve the Secretary of State data and upload it your Google BigQuery instance as an entry in a continuous timeseries.

Getting Started

These instructions will get your copy of the project up and running on your local machine for development and testing purposes.

This repository depends on certain user defined variables that need to be configured in two places: config.py & .env

config.py will generally contain the latest values that I am using to run this repository. A careful read and update to your own values should be all that's needed to update this file. The variables contained here are mostly file and project names.

This repository includes a template for .env as the file .env.template. If you are attempting to run this repository locally and not as a Google Cloud Run job you will need to modify .env.template to contain your secret variables such as the strings for your Google Service Account credentials. Once you have done so, save .env.template as .env

Prerequisites

This repository is meant to be run as a Google Cloud Run job. That means you will need your own Google Cloud login and several Google Cloud services enabled.

  1. Google Cloud Run
  2. Google Big Query
  3. Google Cloud Scheduler

This repository uploads this information to our Google BigQuery. If you do not have your own BigQuery instance it won't work.

Installing

  1. Clone this repository to your local machine.
git clone https://github.com/bengen343/co-registration-timeseries.git
  1. Update variables within config.py to name your outputs and correctly reference file origins/destinations.

  2. Update .env.template with values for each of the variables requested. Note that the variable BQ_ACCOUNT_CREDS should be the entire output of the json credential file associated with the Google service account you're using for this project. Save this file as .env when you've added all your values.

  3. Create a virtual environment for the repository, probably by running python3 -m venv .venv in the terminal from the local directory.

  4. Activate the virtual environment, probably by running .venv\Scripts\activate.bat on Windows or source .venv/bin/activate on Mac.

  5. Install the required dependencies by running pip install -r requirements.txt

Deployment <a name = "deployment>

This is meant to be deployed as a Google Cloud Run Job. To build the package into Google Container Registry you'll need to run the command: gcloud builds submit --pack image=gcr.io/{PROJECT_ID}/co-registration-timeseries

Built Using

Authors

About

A Google Cloud Run job to periodically snapshot the Colorado voter registration district totals to a BigQuery timeseries database.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published