Skip to content

Commit

Permalink
PMM-13129 Encryption. (#3002)
Browse files Browse the repository at this point in the history 
* PMM-13129 Encrypt/decrypt basics.

* PMM-13129 DB connection, part of migration.

* PMM-13129 Tidy.

* PMM-13129 Migration basics.

* PMM-13129 Format.

* PMM-13129 Encrypt, EncryptDB, Decrypt, DecryptDB, refactor.

* PMM-13129 Encryption test workflow.

* PMM-13129 Remove install.

* PMM-13129 Encrypt/Decrypt agents.

* PMM-13129 Changes.

* PMM-13145 Fix for tests.

* PMM-13129 Fix Mongo test.

* PMM-13129 Fix.

* PMM-13129 Encrypt fixture.

* PMM-13129 Encryption test.

* PMM-13129 File mode test.

* PMM-13129 Fix credentials for test env.

* PMM-13129 Clean.

* PMM-13129 Correct DB for encryption test.

* PMM-13129 Moved to utils folder.

* PMM-13129 Empty password fix.

* PMM-13129 Debug logs to warning level.

* PMM-13129 Format.

* PMM-13129 Small change in generated query.

* PMM-13129 Password set check.

* PMM-13129 Fix wrong field.

* PMM-13129 Init in migration.

* PMM-13129 Precheck if already encrypted, moved into managed utils.

* PMM-13129 Migration.

* PMM-13129 Fix for EncryptDB. Encrypt/Decrypt username.

* PMM-13129 Formatting of encryption error, createAgent username fix.

* PMM-13129 Remove unused method for now.

* PMM-13129 Correct mode for cert file.

* PMM-13129 Remove DB test, small refactor.

* PMM-13129 Encryption for external exporter.

* PMM-13129 Fix tests after external exporter encryption.

* PMM-13129 Fix mongo tests.

* PMM-13129 Fix another test to expect encrypted username.

* PMM-13129 Another fix for tests to expect encrypted username.

* PMM-13129 Fix for DecryptDB.

* PMM-13129 Err if encryption is not initialized.

* PMM-13129 Delimiter fix.

* PMM-13129 Fix DecryptDB.

* PMM-13129 Small change in agent test.

* PMM-13129 Fix non related test to make it green for now.

* PMM-13129 Add license headers.

* PMM-13129 License.

* PMM-13129 Lint.

* PMM-13129 Another lint.

* PMM-13129 Lint.

* PMM-13129 Default encryption changes.

* PMM-13129 Encrypt, decrypt all other secret, credentials in agents.

* PMM-13129 Changes, some refactors.

* PMM-13129 Another changes.

* PMM-13129 Refactor.

* PMM-13129 Fix.

* PMM-13129 Changes.

* PMM-13129 Changes.

* PMM-13129 Save.

* PMM-13129 Changes.

* PMM-13129 Another changes.

* PMM-13129 Refactor, another changes.

* PMM-13129 Disable migration encryption until it is done.

* PMM-13129 Basics for settings and migration.

* PMM-13129 Original code for isPasswordSet.

* PMM-13129 Fix current settings test.

* PMM-13129 Basic changes to be able pass custom handlers.

* PMM-13129 Handlers, PG handler.

* PMM-13129 Refactor.

* PMM-13129 Changes, refactor.

* PMM-13129 Migrate and encrypt all possible fields.

* PMM-13129 Fix for service info broker.

* PMM-13129 Fix for settings helper test.

* PMM-13129 Refactor.

* PMM-13129 Lint.

* PMM-13129 Lint.

* PMM-13129 Format.

* PMM-13129 Fix settings helpers test.

* PMM-13129 License header.

* PMM-13129 Another lint.

* PMM-13129 Lint.

* PMM-13129 Changes to fix tests. Refactor.

* PMM-13129 Format.

* PMM-13129 Fix.

* PMM-13129 Encrypt items now receive opened DB connection, refactor.

* PMM-13129 Lint (correct ctx).

* PMM-13129 Refactor, lint.

* PMM-13129 Check.

* PMM-13129 Lint.

* PMM-13129 Fix settings test.

* PMM-13129 Fix to prevent double encryption on setup fixtures.

* PMM-13129 Changes.

* PMM-13129 Encrypt only basic fields in tests (migration).

* PMM-13129 Test.

* PMM-13129 Lint.

* PMM-13129 Different encrypted columns for different migration versions.

* PMM-13129 Fix.

* PMM-13129 TODO.

* PMM-13129 TODO.

* PMM-13129 Check for nothing to encrypt.

* PMM-13129 Encrypted fields based on migration version.

* PMM-13129 Better debug.

* PMM-13129 Lint.

* PMM-13129 Fix, better debug.

* PMM-13129 Exit in case of encryption initialization error.

* PMM-13129 Handle nil migration version.

* PMM-13129 Typo.

* PMM-13129 Fix for service broker and connection check.

* PMM-13129 Comments.

* PMM-13129 Remove debug logging.

* PMM-13129 Remove pointer in EncryptAgent, DecryptAgent.

* PMM-13129 Fix.

* PMM-13129 Fix for service_info_broker.

* PMM-13129 Fix service_info_broker options pointer propagation.

* PMM-13129 Fix for custom labels after removed pointer.

* PMM-13129 Hide cipherText in error message.

* PMM-13129 Panic in case of unavailable encryption.

* PMM-13129 Remove CA certificates from encryption/decryption.

* PMM-13129 Required refactor.

* Update api/serverpb/server.proto

Co-authored-by: Alex Demidoff <[email protected]>

* Update managed/models/database.go

Co-authored-by: Alex Demidoff <[email protected]>

* Update managed/utils/encryption/encryption.go

Co-authored-by: Alex Demidoff <[email protected]>

* Update managed/utils/encryption/models.go

Co-authored-by: Alex Demidoff <[email protected]>

* Update managed/utils/encryption/models.go

Co-authored-by: Alex Demidoff <[email protected]>

* Update managed/utils/encryption/helpers.go

Co-authored-by: Alex Demidoff <[email protected]>

* PMM-13129 Gen.

* PMM-13129 Identifiers word.

* PMM-13129 Remove CAs from handlers.

* Update managed/models/settings.go

Co-authored-by: Alex Demidoff <[email protected]>

* Update managed/utils/encryption/encryption.go

Co-authored-by: Alex Demidoff <[email protected]>

* Update managed/utils/encryption/encryption.go

Co-authored-by: Alex Demidoff <[email protected]>

* PMM-13129 Dereference all DB options on encrypt/decrypt.

* PMM-13129 Custom labels.

* Revert "PMM-13129 Custom labels."

This reverts commit 903b4ef.

* Revert "PMM-13129 Dereference all DB options on encrypt/decrypt."

This reverts commit fe3be31.

* Reapply "PMM-13129 Custom labels."

This reverts commit 9fd8982.

* Reapply "PMM-13129 Dereference all DB options on encrypt/decrypt."

This reverts commit f955040.

* PMM-13129 Remove old migrations tests, required refactor.

* Revert "Reapply "PMM-13129 Custom labels.""

This reverts commit 687a2e2.

* Revert "Reapply "PMM-13129 Dereference all DB options on encrypt/decrypt.""

This reverts commit f09bef1.

* PMM-13129 Logic change.

* PMM-13129 Remove username, aws_access_key, aws_secret_key from enc.

* PMM-13129 Env variable for custom encryption key.

* PMM-13129 Custom key for main check.

* PMM-13129 Remove decrypt agent from create agent methods.

* PMM-13129 Change to skip empty values from encryption.

* PMM-13129 Remove unused struct.

* Update managed/models/database.go

Co-authored-by: Nurlan Moldomurov <[email protected]>

* PMM-13129 Renaming of variable.

* PMM-13129 Remove EncryptedItems field from settings proto.

* PMM-13129 Workaround to create FB for now. Will be reverted.

* PMM-13129 Fix connection checker dsn bug.

* PMM-13129 Another dsn bug.

* PMM-13129 Add back decrypt after insert to fix connection checker.

* PMM-13129 Update reduct words.

* PMM-13129 Fix for test after new redact word.

---------

Co-authored-by: Alex Demidoff <[email protected]>
Co-authored-by: Nurlan Moldomurov <[email protected]>
  • Loading branch information
@JiriCtvrtka @ademidoff @BupycHuk
3 people authored Oct 10, 2024
1 parent eb12a93 commit 9f00573
Show file tree
Hide file tree
Showing 22 changed files with 902 additions and 98 deletions.
3 changes: 3 additions & 0 deletions .github/workflows/main.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,9 @@ jobs:
name: Checks
runs-on: ubuntu-22.04

env:
PMM_ENCRYPTION_KEY_PATH: pmm-encryption.key

steps:
- name: Check out code
uses: actions/checkout@v4
Expand Down
2 changes: 1 addition & 1 deletion go.mod
View file
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ require (
github.com/go-sql-driver/mysql v1.7.1
github.com/gogo/status v1.1.1
github.com/golang-migrate/migrate/v4 v4.17.0
github.com/google/tink/go v1.7.0
github.com/google/uuid v1.6.0
github.com/grafana/grafana-api-golang-client v0.27.0
github.com/grpc-ecosystem/go-grpc-middleware v1.4.0
Expand Down Expand Up @@ -101,7 +102,6 @@ require (
github.com/google/btree v1.0.0 // indirect
github.com/hashicorp/go-hclog v1.6.2 // indirect
github.com/hashicorp/go-msgpack/v2 v2.1.1 // indirect
github.com/hashicorp/go-uuid v1.0.2 // indirect
github.com/kr/fs v0.1.0 // indirect
github.com/mattn/go-colorable v0.1.12 // indirect
github.com/miekg/dns v1.1.26 // indirect
Expand Down
2 changes: 2 additions & 0 deletions go.sum
View file
Original file line number Diff line number Diff line change
Expand Up @@ -239,6 +239,8 @@ github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
github.com/google/tink/go v1.7.0 h1:6Eox8zONGebBFcCBqkVmt60LaWZa6xg1cl/DwAh/J1w=
github.com/google/tink/go v1.7.0/go.mod h1:GAUOd+QE3pgj9q8VKIGTCP33c/B7eb4NhxLcgTJZStM=
github.com/google/uuid v1.4.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
Expand Down
1 change: 0 additions & 1 deletion managed/Makefile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,6 @@ clean: ## Remove generated files

release: ## Build pmm-managed release binaries
env CGO_ENABLED=0 go build -v $(PMM_LD_FLAGS) -o $(PMM_RELEASE_PATH)/ ./cmd/...
$(PMM_RELEASE_PATH)/pmm-managed --version

release-starlark:
env CGO_ENABLED=0 go build -v $(PMM_LD_FLAGS) -o $(PMM_RELEASE_PATH)/ ./cmd/pmm-managed-starlark/...
Expand Down
45 changes: 29 additions & 16 deletions managed/models/agent_helpers.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -229,7 +229,8 @@ func FindAgents(q *reform.Querier, filters AgentFilters) ([]*Agent, error) {

agents := make([]*Agent, len(structs))
for i, s := range structs {
agents[i] = s.(*Agent) //nolint:forcetypeassert
decryptedAgent := DecryptAgent(*s.(*Agent)) //nolint:forcetypeassert
agents[i] = &decryptedAgent
}

return agents, nil
Expand All @@ -249,8 +250,9 @@ func FindAgentByID(q *reform.Querier, id string) (*Agent, error) {
}
return nil, errors.WithStack(err)
}
decryptedAgent := DecryptAgent(*agent)

return agent, nil
return &decryptedAgent, nil
}

// FindAgentsByIDs finds Agents by IDs.
Expand All @@ -272,7 +274,8 @@ func FindAgentsByIDs(q *reform.Querier, ids []string) ([]*Agent, error) {

res := make([]*Agent, len(structs))
for i, s := range structs {
res[i] = s.(*Agent) //nolint:forcetypeassert
decryptedAgent := DecryptAgent(*s.(*Agent)) //nolint:forcetypeassert
res[i] = &decryptedAgent
}
return res, nil
}
Expand Down Expand Up @@ -323,7 +326,8 @@ func FindDBConfigForService(q *reform.Querier, serviceID string) (*DBConfig, err

res := make([]*Agent, len(structs))
for i, s := range structs {
res[i] = s.(*Agent) //nolint:forcetypeassert
decryptedAgent := DecryptAgent(*s.(*Agent)) //nolint:forcetypeassert
res[i] = &decryptedAgent
}

if len(res) == 0 {
Expand All @@ -350,8 +354,8 @@ func FindPMMAgentsRunningOnNode(q *reform.Querier, nodeID string) ([]*Agent, err

res := make([]*Agent, 0, len(structs))
for _, str := range structs {
row := str.(*Agent) //nolint:forcetypeassert
res = append(res, row)
decryptedAgent := DecryptAgent(*str.(*Agent)) //nolint:forcetypeassert
res = append(res, &decryptedAgent)
}

return res, nil
Expand Down Expand Up @@ -395,8 +399,8 @@ func FindPMMAgentsForService(q *reform.Querier, serviceID string) ([]*Agent, err
}
res := make([]*Agent, 0, len(pmmAgentRecords))
for _, str := range pmmAgentRecords {
row := str.(*Agent) //nolint:forcetypeassert
res = append(res, row)
decryptedAgent := DecryptAgent(*str.(*Agent)) //nolint:forcetypeassert
res = append(res, &decryptedAgent)
}

return res, nil
Expand Down Expand Up @@ -477,7 +481,8 @@ func FindAgentsForScrapeConfig(q *reform.Querier, pmmAgentID *string, pushMetric

res := make([]*Agent, len(allAgents))
for i, s := range allAgents {
res[i] = s.(*Agent) //nolint:forcetypeassert
decryptedAgent := DecryptAgent(*s.(*Agent)) //nolint:forcetypeassert
res[i] = &decryptedAgent
}
return res, nil
}
Expand Down Expand Up @@ -641,11 +646,14 @@ func CreateNodeExporter(q *reform.Querier,
if err := row.SetCustomLabels(customLabels); err != nil {
return nil, err
}
if err := q.Insert(row); err != nil {

encryptedAgent := EncryptAgent(*row)
if err := q.Insert(&encryptedAgent); err != nil {
return nil, errors.WithStack(err)
}
agent := DecryptAgent(encryptedAgent)

return row, nil
return &agent, nil
}

// CreateExternalExporterParams params for add external exporter.
Expand Down Expand Up @@ -725,11 +733,14 @@ func CreateExternalExporter(q *reform.Querier, params *CreateExternalExporterPar
if err := row.SetCustomLabels(params.CustomLabels); err != nil {
return nil, err
}
if err := q.Insert(row); err != nil {

encryptedAgent := EncryptAgent(*row)
if err := q.Insert(&encryptedAgent); err != nil {
return nil, errors.WithStack(err)
}
agent := DecryptAgent(encryptedAgent)

return row, nil
return &agent, nil
}

// CreateAgentParams params for add common exporter.
Expand Down Expand Up @@ -912,15 +923,17 @@ func CreateAgent(q *reform.Querier, agentType AgentType, params *CreateAgentPara
DisabledCollectors: params.DisableCollectors,
LogLevel: pointer.ToStringOrNil(params.LogLevel),
}

if err := row.SetCustomLabels(params.CustomLabels); err != nil {
return nil, err
}
if err := q.Insert(row); err != nil {

encryptedAgent := EncryptAgent(*row)
if err := q.Insert(&encryptedAgent); err != nil {
return nil, errors.WithStack(err)
}
agent := DecryptAgent(encryptedAgent)

return row, nil
return &agent, nil
}

// ChangeCommonAgentParams contains parameters that can be changed for all Agents.
Expand Down
93 changes: 89 additions & 4 deletions managed/models/database.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ import (
"net"
"net/url"
"os"
"slices"
"strconv"
"strings"

Expand All @@ -36,6 +37,8 @@ import (
"google.golang.org/grpc/status"
"gopkg.in/reform.v1"
"gopkg.in/reform.v1/dialects/postgresql"

"github.com/percona/pmm/managed/utils/encryption"
)

const (
Expand Down Expand Up @@ -1146,12 +1149,87 @@ func SetupDB(ctx context.Context, sqlDB *sql.DB, params SetupDBParams) (*reform.
return nil, errCV
}

if err := migrateDB(db, params); err != nil {
agentColumnsToEncrypt := []encryption.Column{
{Name: "username"},
{Name: "password"},
{Name: "aws_access_key"},
{Name: "aws_secret_key"},
{Name: "mongo_db_tls_options", CustomHandler: EncryptMongoDBOptionsHandler},
{Name: "azure_options", CustomHandler: EncryptAzureOptionsHandler},
{Name: "mysql_options", CustomHandler: EncryptMySQLOptionsHandler},
{Name: "postgresql_options", CustomHandler: EncryptPostgreSQLOptionsHandler},
{Name: "agent_password"},
}

itemsToEncrypt := []encryption.Table{
{
Name: "agents",
Identifiers: []string{"agent_id"},
Columns: agentColumnsToEncrypt,
},
}

if err := migrateDB(db, params, itemsToEncrypt); err != nil {
return nil, err
}

return db, nil
}

// EncryptDB encrypts a set of columns in a specific database and table.
func EncryptDB(tx *reform.TX, params SetupDBParams, itemsToEncrypt []encryption.Table) error {
if len(itemsToEncrypt) == 0 {
return nil
}

settings, err := GetSettings(tx)
if err != nil {
return err
}
alreadyEncrypted := make(map[string]bool)
for _, v := range settings.EncryptedItems {
alreadyEncrypted[v] = true
}

notEncrypted := []encryption.Table{}
newlyEncrypted := []string{}
for _, table := range itemsToEncrypt {
columns := []encryption.Column{}
for _, column := range table.Columns {
dbTableColumn := fmt.Sprintf("%s.%s.%s", params.Name, table.Name, column.Name)
if alreadyEncrypted[dbTableColumn] {
continue
}

columns = append(columns, column)
newlyEncrypted = append(newlyEncrypted, dbTableColumn)
}
if len(columns) == 0 {
continue
}

table.Columns = columns
notEncrypted = append(notEncrypted, table)
}

if len(notEncrypted) == 0 {
return nil
}

err = encryption.EncryptItems(tx, notEncrypted)
if err != nil {
return err
}
_, err = UpdateSettings(tx, &ChangeSettingsParams{
EncryptedItems: slices.Concat(settings.EncryptedItems, newlyEncrypted),
})
if err != nil {
return err
}

return nil
}

// checkVersion checks minimal required PostgreSQL server version.
func checkVersion(ctx context.Context, db reform.DBTXContext) error {
PGVersion, err := GetPostgreSQLVersion(ctx, db)
Expand Down Expand Up @@ -1211,7 +1289,7 @@ func initWithRoot(params SetupDBParams) error {
}

// migrateDB runs PostgreSQL database migrations.
func migrateDB(db *reform.DB, params SetupDBParams) error {
func migrateDB(db *reform.DB, params SetupDBParams, itemsToEncrypt []encryption.Table) error {
var currentVersion int
errDB := db.QueryRow("SELECT id FROM schema_migrations ORDER BY id DESC LIMIT 1").Scan(&currentVersion)
// undefined_table (see https://www.postgresql.org/docs/current/errcodes-appendix.html)
Expand Down Expand Up @@ -1247,6 +1325,11 @@ func migrateDB(db *reform.DB, params SetupDBParams) error {
}
}

err := EncryptDB(tx, params, itemsToEncrypt)
if err != nil {
return err
}

if params.SetupFixtures == SkipFixtures {
return nil
}
Expand All @@ -1260,14 +1343,16 @@ func migrateDB(db *reform.DB, params SetupDBParams) error {
return err
}

if err = setupFixture1(tx.Querier, params); err != nil {
err = setupPMMServerAgents(tx.Querier, params)
if err != nil {
return err
}

return nil
})
}

func setupFixture1(q *reform.Querier, params SetupDBParams) error {
func setupPMMServerAgents(q *reform.Querier, params SetupDBParams) error {
// create PMM Server Node and associated Agents
node, err := createNodeWithID(q, PMMServerNodeID, GenericNodeType, &CreateNodeParams{
NodeName: "pmm-server",
Expand Down
55 changes: 0 additions & 55 deletions managed/models/database_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@ import (
"database/sql"
"fmt"
"testing"
"time"

"github.com/AlekSi/pointer"
"github.com/lib/pq"
Expand Down Expand Up @@ -327,60 +326,6 @@ func TestDatabaseChecks(t *testing.T) {
}

func TestDatabaseMigrations(t *testing.T) {
t.Run("Update metrics resolutions", func(t *testing.T) {
sqlDB := testdb.Open(t, models.SkipFixtures, pointer.ToInt(9))
defer sqlDB.Close() //nolint:errcheck
settings, err := models.GetSettings(sqlDB)
require.NoError(t, err)
metricsResolutions := models.MetricsResolutions{
HR: 5 * time.Second,
MR: 5 * time.Second,
LR: 60 * time.Second,
}
settings.MetricsResolutions = metricsResolutions
err = models.SaveSettings(sqlDB, settings)
require.NoError(t, err)

settings, err = models.GetSettings(sqlDB)
require.NoError(t, err)
require.Equal(t, metricsResolutions, settings.MetricsResolutions)

testdb.SetupDB(t, sqlDB, models.SkipFixtures, pointer.ToInt(10))
settings, err = models.GetSettings(sqlDB)
require.NoError(t, err)
require.Equal(t, models.MetricsResolutions{
HR: 5 * time.Second,
MR: 10 * time.Second,
LR: 60 * time.Second,
}, settings.MetricsResolutions)
})
t.Run("Shouldn' update metrics resolutions if it's already changed", func(t *testing.T) {
sqlDB := testdb.Open(t, models.SkipFixtures, pointer.ToInt(9))
defer sqlDB.Close() //nolint:errcheck
settings, err := models.GetSettings(sqlDB)
require.NoError(t, err)
metricsResolutions := models.MetricsResolutions{
HR: 1 * time.Second,
MR: 5 * time.Second,
LR: 60 * time.Second,
}
settings.MetricsResolutions = metricsResolutions
err = models.SaveSettings(sqlDB, settings)
require.NoError(t, err)

settings, err = models.GetSettings(sqlDB)
require.NoError(t, err)
require.Equal(t, metricsResolutions, settings.MetricsResolutions)

testdb.SetupDB(t, sqlDB, models.SkipFixtures, pointer.ToInt(10))
settings, err = models.GetSettings(sqlDB)
require.NoError(t, err)
require.Equal(t, models.MetricsResolutions{
HR: 1 * time.Second,
MR: 5 * time.Second,
LR: 60 * time.Second,
}, settings.MetricsResolutions)
})
t.Run("stats_collections field migration: string to string array", func(t *testing.T) {
sqlDB := testdb.Open(t, models.SkipFixtures, pointer.ToInt(57))
defer sqlDB.Close() //nolint:errcheck
Expand Down
Loading

0 comments on commit 9f00573

Please sign in to comment.