Skip to content

The scraper runs once every few minutes to collect title changes for all media that are tracked by OpenTitles.

License

Notifications You must be signed in to change notification settings

opentitles/scraper

Repository files navigation

See https://opentitles.info/ for download links and more information.

OpenTitles Scraper

This component iterates all the RSS feeds defined in the shared OpenTitles definition and saves them to a MongoDB instance.

Environment

Below are all the environment variables that the scraper picks up with the corresponding defaults.

# Connection string for the RabbitMQ exchange
RABBITMQ_URL=amqp://127.0.0.1
# Connection string for the MongoDB host
MONGO_URL=mongodb://127.0.0.1:27017
# Database name for the production database
MONGO_DB_PROD=opentitles
# Database name for the test/local database
MONGO_DB_TEST=opentitlestest
# Port to listen on
PORT=8083
# Notifier type to use, one of 'null', ' pubsub' or 'webhook'
NOTIFIER=pubsub
# JSON-encoded array of webhook listeners.
WEBHOOK_LISTENERS="[]"

About

The scraper runs once every few minutes to collect title changes for all media that are tracked by OpenTitles.

Resources

License

Stars

Watchers

Forks

Sponsor this project