From c1af83cfe061011dcec8bb55d0f69b1b963576e1 Mon Sep 17 00:00:00 2001 From: Rob Browning Date: Mon, 3 Aug 2020 11:01:23 -0500 Subject: [PATCH] (PDB-4763) Add initial command broadcast docs --- documentation/configure.markdown | 84 +++++++++++++++++++++---- src/puppetlabs/puppetdb/pdb_routing.clj | 1 + 2 files changed, 73 insertions(+), 12 deletions(-) diff --git a/documentation/configure.markdown b/documentation/configure.markdown index c3c9d7413d..e71485ccff 100644 --- a/documentation/configure.markdown +++ b/documentation/configure.markdown @@ -318,6 +318,18 @@ Replace `` with the DB server's hostname. Replace `` with the port on which PostgreSQL is listening. Replace `` with the name of the database you've created for use with PuppetDB. +With this arrangemnt, PuppetDB will use the specified database for all +operations (storage and queries), but PuppetDB supports other +arrangments. It's possible to +[specify a different read-only database](#[read-database]--settings) +to among other things, handle queries, and provide responses for +PuppetDB's [peer-to-peer synchronization process](#sync--settings). + +It's also possible to specify [multiple storage databases](#broadcasting-commands) +to which PuppetDB will attempt delivery of incoming information on a +best-effort basis. Note that any given PuppetDB instance can have +sync enabled or command broadcast, but not both. + #### Using SSL With PostgreSQL It's possible to use SSL to protect connections to the database. There @@ -394,6 +406,51 @@ Then specify `puppetdb_migrator` as the See the [migration coordination documentation](migration_coordination) for a more detailed explanation of the process. +#### Broadcasting commands to multiple databases + +PuppetDB can be configured to attempt to send each command to multiple +databases on a best-effort basis. There are no guarantees that any +given command will reach all of the databases, so the process is +inherently "lossy". At the moment PuppetDB will consider a command to +be successfully processed as soon as it manages to send the command to +at least one of the databases. Note that currently, a server cannot +be configured to broadcast and [sync](#sync--settings). + +To enable broadcast, create one `[database-NAME]` subsection for each +database. For example, to attempt to send commands to a primary and +secondary database, you could add something like this to your +configuration: + + [database] + # When subsections are defined, unset values in any subsection will + # default to the values set here. + subname = //host1:5432/puppetdb + username = + password = + # Currently required + migrate = false + + [database-primary] + # Settings intentionally omitted. + # Unset values default to those in [database] + + [database-secondary] + subname = //host2:5432/puppetdb + username = + password = + +When subsections are defined, the `[database]` section provides +defaults that will apply whenever a value is not specified in a +subsection. These defaults will also be used for the read database +whenever an explict [`[read-database]`]](#[read-database]--settings) +isn't provided. + +PuppetDB currently migrates multiple write databases one at a time. +If you'd prefer more control over the process you can migrate +databases individually by running `puppetdb upgrade -c +just-one-db.ini` for each database and making sure that each command +exits successfully (with non-zero status). + ### `gc-interval` This controls how often, in minutes, to compact the database. The @@ -579,24 +636,27 @@ suggesting appropriate action. If set to zero, this check is disabled. ----- The `[read-database]` section configures PuppetDB's _read-database_ -settings, useful when running a PostgreSQL [Hot -Standby](http://wiki.postgresql.org/wiki/Hot_Standby) cluster. -Currently, only configuring a PostgreSQL read-database is supported. See -the PostgreSQL documentation [here](http://wiki.postgresql.org/wiki/Hot_Standby) -for details on configuring the cluster. The `[read-database]` portion -of the configuration is in addition to the `[database]` settings. If -`[read-database]` is specified, `[database]` must also be specified. +settings, which can be useful if you have +[set up a PostgreSQL Hot Standby](http://wiki.postgresql.org/wiki/Hot_Standby) +cluster. + +Whenever a `[read-database]` is specified, a `[database]` must also be +specified to provide a write database. Whenever a `[read-database]` +is not specified, the relevant read operations will be directed to the +database specified in the `[database]` section, which must have all +the required settings (i.e. any database subsections are irrelevant). -To configure PuppetDB to use a read-only database from the cluster, -add the following to the `[read-database]` section: +To configure PuppetDB to use a read-only database from a hot standby +cluster, add the following to the configuration: + [read-database] subname = //:/ username = password = -Replace `` with the DB server's hostname. Replace `` with -the port on which PostgreSQL is listening. Replace `` with -the name of the database you've created for use with PuppetDB. +Replace `` with the DB server's hostname, `` with the port +on which PostgreSQL is listening, and `` with the name of +the database you've created for PuppetDB. #### Using SSL With PostgreSQL diff --git a/src/puppetlabs/puppetdb/pdb_routing.clj b/src/puppetlabs/puppetdb/pdb_routing.clj index 38c9542832..033135a0db 100644 --- a/src/puppetlabs/puppetdb/pdb_routing.clj +++ b/src/puppetlabs/puppetdb/pdb_routing.clj @@ -52,6 +52,7 @@ (conf/reject-large-commands? defaulted-config) (conf/max-command-size defaulted-config)) "/query" (server/build-app get-shared-globals) + ;; FIXME: /admin/cmd read-db? "/admin" (admin/build-app enqueue-command-fn query-fn db-cfg