All notable changes to this project are documented in this file.
Release date: 2022-02-10
This prerelease comes with an update to the sigs.k8s.io/kustomize/api
package,
bringing the controller on par with the Kustomize v4.5.2 release.
Kustomize v4.5.2 contains a regression bug fix for pseudo git HTTP URLs.
Improvements:
- Update controller to kustomize v4.5.2 #567
- Clarify
spec.path
in API docs #566 - Fix typo in API docs #564
Release date: 2022-02-07
This prerelease comes with an update to the sigs.k8s.io/kustomize
packages, bringing
the controller on par with the Kustomize v4.5.1 release.
Improvements:
- Update controller to kustomize v4.5.1 #559
Fixes:
- Transfer ownership of the kubectl managed fields #562
Release date: 2022-02-01
This prerelease comes with security improvements for multi-tenant clusters:
- Platform admins can enforce impersonation across the cluster using the
--default-service-account
flag. When the flag is set, allKustomizations
, which don't havespec.serviceAccountName
specified, use the service account name provided by--default-service-account=<SA Name>
in the namespace of the object. - Platform admins can disable cross-namespace references with the
--no-cross-namespace-refs=true
flag. When this flag is set,Kustomizations
can only refer to sources (GitRepositories
andBuckets
) in the same namespace as theKustomization
object, preventing tenants from accessing another tenant's repositories.
The controller container images are signed with Cosign and GitHub OIDC, and a Software Bill of Materials in SPDX format has been published on the release page.
Starting with this version, the controller deployment conforms to the Kubernetes restricted pod security standard:
- all Linux capabilities were dropped
- the root filesystem was set to read-only
- the seccomp profile was set to the runtime default
- run as non-root was enabled
- the user and group ID was set to 65534
Breaking changes:
- The use of new seccomp API requires Kubernetes 1.19.
- The controller container is now executed under 65534:65534 (userid:groupid). This change may break deployments that hard-coded the user ID of 'controller' in their PodSecurityPolicy.
- When both
spec.kubeConfig
andspec.ServiceAccountName
are specified, the controller will impersonate the service account on the target cluster, previously the controller ignored the service account.
Features:
- Allow setting a default service account for impersonation #550
- Allow disabling cross-namespace references #549
- SOPS: Add support for HashiCorp Vault token-based authentication #538
Improvements:
- Publish SBOM and sign release artifacts #541
- Drop capabilities, enable seccomp and enforce runAsNonRoot #539
- docs: Add var substitution operator escape syntax #537
- Update development documentation #540
- Refactor Fuzz implementation #536
Fixes:
- Revoke kubectl managed fields ownership #527
- Ensure object are finalized under impersonation #552
- Use patch instead of update when adding finalizers #535
- Fix preflight validation #544
- Fix the missing protocol for the first port in manager config #547
Release date: 2022-01-13
This prerelease fixes a regression bug introduced in v0.19.0 that prevented StatefulSets
from being reconciled on Kubernetes <= 1.21.
Fixes:
- Update
fluxcd/pkg/ssa
to exclude the status field from apply #533
Release date: 2022-01-10
This prerelease comes with an update to the Kubernetes and controller-runtime dependencies to align them with the Kubernetes 1.23 release.
In addition, the controller is now built with Go 1.17 and Alpine 3.15.
Improvements:
- Update Go to v1.17 and controller-runtime to v0.11 #478
- Add condition to checkDependencies when SourceRef is the same #521
Fixes:
Release date: 2021-12-09
This prerelease comes with improvements to force applying objects with immutable fields changes.
Improvements:
- Update dependencies (fix CVE-2021-43784) #509
- Update golang.org/x/text to v0.3.7 (fix CVE-2021-38561) #512
- Add test for replacing variables in secrets #505
- Document behaviour when changes are made to fields not stored in git #501
- SOPS: ensure proper wiring to default server #513
Release date: 2021-11-23
This prerelease replaces deprecated dependencies, most notably being the OpenPGP
package where golang.org/x/crypto/openpgp
was replaced with
github.com/ProtonMail/go-crypto/openpgp
.
Improvements:
- Update
source-controller/api
to v0.19.0 #499 - Replace deprecated dependencies #498
- Update
opencontainers/{image-spec, runc}
#497
Release date: 2021-11-12
This prerelease comes with artifact integrity verification.
During the acquisition of an artifact, kustomize-controller computes its checksum using SHA-2
and verifies that it matches the checksum advertised in the Status
of the Source.
The controller dependencies has been updated to match kustomize v4.4.1.
Improvements:
Release date: 2021-11-09
This prerelease comes with support for ignoring changes made to in-cluster resources by annotating them with:
kustomize.toolkit.fluxcd.io/reconcile: disabled
When the kustomize.toolkit.fluxcd.io/reconcile
annotation is set to disabled
,
the controller will no longer apply changes from source, nor will it prune the resource.
To resume reconciliation, set the annotation to enabled
or remove it.
Features:
- Allow disabling the reconciliation of in-cluster resources #484
Fixes:
- Set delete propagation policy to background #482
- Warn when secrets are not decrypted before apply #483
- Remove gopass dependency #480
- Remove deprecated io/ioutil dependency #479
Release date: 2021-10-19
This prerelease comes with support for SOPS encrypted .env
files used in kustomize secret generator.
Improvements:
- SOPS: Decrypt dotenv files used in kustomize secret generator #463
- SOPS: Document dotenv secret generator #469
Fixes:
- Fix cluster scope detection of applied objects #465
Release date: 2021-10-13
This prerelease comes with improvements to drift detection of Kubernetes custom resources.
Improvements:
- Improve drift detection #459
Release date: 2021-10-12
This prerelease comes with fixes to HPA and Service objects validation.
Fixes:
- Fix Service and HPA v2beta1 validation #455
Release date: 2021-10-11
This prerelease comes with fixes for drift detection in Secrets and ConfigMaps.
Fixes:
- Fix drift detection in Secrets and ConfigMaps #451
Release date: 2021-10-10
This prerelease comes with fixes for server-side apply upstream bugs affecting Kubernetes < 1.22.
Fixes:
- Fix SSA upstream bugs for Kubernetes < 1.22 #448
Release date: 2021-10-08
This prerelease comes with fixes to backwards compatibility with Flux CLI 0.17 and older.
Fixes:
- Fix inventory panic for v1beta1 objects #445
Release date: 2021-10-08
This prerelease comes with a new reconciler
based on Kubernetes server-side apply and graduates the API to v1beta2
.
The controller dependencies has been updated to match kustomize v4.4.0 which restores the usage of YAML anchors.
Breaking changes
-
Namespaced objects must contain
metadata.namespace
, defaulting to thedefault
namespace is no longer supported. Setting a namespace for all objects reconciled by a Kustomization can be done withspec.targetNamespace
. -
The logs, events and alerts that report Kubernetes namespaced object changes are now using the
Kind/Namespace/Name
format instead ofKind/Name
. -
The minimum required version of Kubernetes has changed to:
Kubernetes version Minimum required v1.16
>= 1.16.11
v1.17
>= 1.17.7
v1.18
>= 1.18.4
v1.19
and later>= 1.19.0
Features and Improvements
- Being able to validate and reconcile sources that contain both CRDs and CRs.
- Being able to wait for all the applied resources to become ready without requiring users to fill-in the health check list.
- Improve performance (CPU, memory, network, FD usage) and reduce the number of calls to Kubernetes API by replacing kubectl execs with a specialized applier written in Go.
- Detect and report drift between the desired state (git, s3, etc) and cluster state reliably.
- Improve the overall observably of the reconciliation process by reporting in real-time the garbage collection and health assessment actions.
- Reconcile empty sources including pruning of all the resources previously applied.
- Mask secrets data in logs, events and alerts.
API changes
The kustomize.toolkit.fluxcd.io/v1beta2
API is backwards compatible with v1beta1
.
Additions, deprecations and removals:
.spec.patchesStrategicMerge
deprecated in favour of.spec.patches
.spec.patchesJson6902
deprecated in favour of.spec.patches
.spec.validation
deprecated and no longer used (server-side validation is implicit).spec.wait
added (when enabled, will wait for all the reconciled resources to become ready).status.snapshot
replaced by.status.inventory
Updating the manifests in Git to v1beta2
can be done at any time after the kustomize-controller upgrade.
All users are encouraged to update the manifests as the deprecated fields
will be removed when the next API version will be released.
Release date: 2021-09-09
This prerelease comes with improvements to logging. When Kubernetes Secrets can't be reconciled due to validation errors, the controller will mask the secret data from logs and events to prevent disclosing sensitive information.
Improvements:
- Mask the Kubernetes Secrets data from dry-run and apply logs #420
Release date: 2021-08-26
This prerelease comes with improvements to garbage collection. When pruning is enabled, the controller will skip the deletion of objects with ownerReference.BlockOwnerDeletion=true, as they are subject to Kubernetes GC.
The controller dependencies has been updated to match kustomize v4.3.0.
Improvements:
- Update controller to kustomize v4.3.0 #416
- Skip garbage collection of objects with owner references #411
- Add tests for various kustomize transformers #408
Release date: 2021-08-05
This prerelease comes with support for SOPS encrypted kubeconfigs.
Improvements:
- Make the kubeconfig secrets compatible with SOPS #400
- Remove old util ObjectKey #397
- Var substitution opt-in docs #389
- Update dependencies #401
Fixes:
- Prevent nil pointer dereference in health checks #394
Release date: 2021-07-05
This prerelease comes with improvements to health assessment error reporting.
The controller dependencies has been updated to match kustomize v4.2.0.
Improvements:
- Make it easier to reason about health check failures #374
- Update Alpine v3.14 and kubectl v1.21.2 #385
- Update controller to kustomize v4.2.0 #383
Fixes:
- Fix typo in dependency ready log #384
Release date: 2021-06-30
This prerelease comes with kubectl v1.21.1.
Improvements:
- Update kubectl to v1.21.1 #381
- e2e: Update Kubernetes to v1.21.1 #380
- Improve test coverage of the
dependsOn
feature #380
Release date: 2021-06-14
This prerelease brings the controller on a par with Kustomize v4. The Kubernetes and controller-runtime dependencies have been updated to match the Kubernetes 1.21 release.
The Kustomization API has been extended with support for generic in-line patches.
Starting with this version, the controller uses an annotation instead of a label to keep track of removed manifests from source. Please consult the garbage collection docs for more details.
Breaking changes:
- Due to the removal of
hashicorp/go-getter
from Kustomize v4, the set of URLs accepted by Kustomize in theresources
filed is reduced to only file system paths or values compatible withgit clone
. This means you can no longer use resources from archives (zip, tgz, etc). - YAML anchors are no longer supported in Kustomize v4, see kustomize/issues/3675 for more details.
- Due to a bug
in Kustomize v4, if you have non-string keys in your manifests,
the controller will fail with
json: unsupported type
error.
Features:
- Add support for in-line generic patches to Flux Kustomization API #364
Improvements:
Release date: 2021-06-02
This prerelease comes with support for decrypting any file format used with
Kustomize secretGenerator
.
Improvements:
- Support decrypting any file format in secret generator #353
Release date: 2021-05-26
This prerelease comes with a fix to the reconciliation timeout handling.
Improvements:
Fixes:
- Fix validation and application timeout handling #346
Release date: 2021-04-29
This prerelease comes with support for decrypting Kubernetes
secrets generated with SOPS and
Kustomize secretGenerator
.
Features:
- SOPS: Decrypt Kubernetes secrets generated by kustomize #329
Improvements:
- Extract validation error from apply server dry run output #333
Release date: 2021-04-22
This prerelease comes with a bug fix where the rate limited events were delaying the reconciliation.
Improvements:
Fixes:
- Avoid retrying rate limited events #326
- Make log level info for 'Dependencies do not meet ready condition' #317
Release date: 2021-04-06
This prerelease extends the Mozilla SOPS integration with support for age encryption format.
This prerelease comes with a breaking change to SOPS integration.
The OpenPGP private keys stored in Kubernetes secrets must have the .asc
file extension.
For age, the private keys file extension must be .agekey
.
Features:
- Support SOPS age encryption #309
Improvements:
Release date: 2021-03-26
This prerelease comes with a breaking change to the leader election ID
from 7593cc5d.fluxcd.io
to kustomize-controller-leader-election
to be more descriptive. This change should not have an impact on most
installations, as the default replica count is 1
. If you are running
a setup with multiple replicas, it is however advised to scale down
before upgrading.
Improvements:
Release date: 2021-03-17
This prerelease comes with updates to the runtime packages.
The controller exposes a gauge metric to track the suspended status
of Kustomization
objects: gotk_suspend_status{kind,name,namespace}
.
Improvements:
Release date: 2021-03-05
This prerelease comes with improvements to the notification system. The controller retries with exponential backoff when fetching artifacts, preventing spamming events when source-controller becomes unavailable for a short period of time.
Improvements:
- Retry with exponential backoff when fetching artifacts #289
- Validate the var names before substitution #291
Release date: 2021-02-25
This prerelease comes with an update to the sigs.k8s.io/cli-utils
dependency, to guard against a potential bug with health assessments
that was discovered in the flux
CLI.
Improvements:
- Update sigs.k8s.io/cli-utils to v0.22.2 #287
Release date: 2021-02-24
This is the ninth MINOR prerelease.
This prerelease comes with support for recreating Kubernetes objects (e.g. Jobs) when immutable fields are changed in Git.
Features:
- Add support for recreating objects when immutable fields are updated #271
Improvements:
Fixes:
- Avoid prompts on SOPS key import by adding batch flag to gpg #281
Release date: 2021-02-18
This prerelease adds an array field called substituteFrom
to the post build
API. SubstituteFrom
holds references to ConfigMaps
and Secrets
containing
the variables (data keys) and their values (data values) to be substituted in
the YAML manifests.
You can disable the variable substitution for certain resources by either labeling or annotating them with:
kustomize.toolkit.fluxcd.io/substitute: disabled
Features:
- Implement var substitution from ConfigMaps and Secrets #275
Release date: 2021-02-12
This is the eight MINOR prerelease.
This prerelease comes with support for bash-style variable substitutions.
The Kustomization API was extended with in-line support for Kustomize Strategic Merge and JSON 6902 patches.
Pruning can be disabled for certain resources by either labeling or annotating them with:
kustomize.toolkit.fluxcd.io/prune: disabled
Golang pprof
endpoints have been enabled on the metrics server,
making it easier to collect runtime information to debug performance issues.
Features:
- Add support for variable substitutions #253
- Support Strategic Merge and JSON 6902 patches #264
- Allow disabling of prune on certain resources #267
Improvements:
Release date: 2021-02-02
This prerelease comes with a change to the Kustomization status so that the controller can keep track of the last health assessment result and avoid issuing redundant health check events.
Improvements:
- Add Healthy status condition #262
Release date: 2021-02-01
This prerelease comes with support for running the Kustomization validation using service account impersonation.
Improvements:
- Support impersonation for validation #260
Release date: 2021-01-26
This prerelease comes with a fix to the service account impersonation when running health checks and garbage collection.
Fixes:
- Clear
config.BearerTokenFile
when settingBearerToken
#258
Release date: 2021-01-25
This prerelease fixes a regression bug introduced in v0.7.0
that caused
failed reconciliations to be immediately retried instead of being scheduled
at the specified interval.
Fixes:
- Fix reconciliation retry scheduler #256
Release date: 2021-01-22
This is the seventh MINOR prerelease.
An optional field was added spec.retryInterval
that allows users to
requeue a failed reconciliation at a different interval than spec.Interval
.
The LocalObjectReference
from the Kubernetes core has been replaced
with our own, making Name
a required field. The impact of this should
be limited to direct API consumers only, as the field was already
required by controller logic.
Improvements:
- Allow failed reconciliations to be scheduled at a different interval #250
- Update fluxcd/pkg/runtime to v0.8.0 #247
Release date: 2021-01-19
This prerelease comes with fixes to the kustomize build procedure by disabling kyaml. The Kubernetes packages were updated to v1.20.2 and kustomize/api to v0.7.2.
Improvements:
Release date: 2021-01-15
This prerelease adds support for kustomization.yml
and Kustomization
files at the root of the configured path.
Improvements:
- Look for all accepted Kustomization filenames #238
Release date: 2021-01-14
This prerelease fixes a regression bug introduced in v0.6.0
that caused
reconciliation request annotations to be ignored in certain scenarios.
Two new argument flags are introduced to support configuring the QPS
(--kube-api-qps
) and burst (--kube-api-burst
) while communicating
with the Kubernetes API server.
Improvements:
Fixes:
- Upgrade runtime package to v0.6.2 for regression bug fix #234
Release date: 2021-01-12
This is the sixth MINOR prerelease, upgrading the controller-runtime
dependencies to v0.7.0
.
The container image for ARMv7 and ARM64 that used to be published
separately as kustomize-controller:*-arm64
has been merged with the
AMD64 image.
Improvements:
- Update kubectl to v1.20.1 #231
- Update kustomize/api to v0.7.1 #229
- Run GC and health checks using service account impersonation #221
Fixes:
- No longer treat dirs as kustomization files #224
Release date: 2020-12-18
This prerelease comes with improvements to health reporting.
Improvements:
- Emit healthcheck event when Kustomization was not ready #219
- Add debug logs to garbage collection #218
- Add SOPS user error to logs #220
Release date: 2020-12-16
This prerelease comes with improvements to garbage collection and adds safe guards for relative paths.
The Path
field was marked as optional, when not specified,
it defaults to the root path of the SourceRef
.
Health checking and garbage collection for HelmReleases
and other custom resources were fixed by downgrading
to controller-runtime
v0.6.3.
Improvements:
- Refactor garbage collection #210
- Make Path field optional and add safe guards for relative paths #211
Release date: 2020-12-14
This prerelease comes with improvements to error reporting.
Improvements:
- Refactor apply error reporting #205
Release date: 2020-12-11
This is the fifth MINOR prerelease. It comes with support for
overriding container images with spec.images
.
Improvements:
Release date: 2020-11-26
This is the fourth MINOR prerelease. Suspended Kustomizations are no longer marked as not ready, the status sub-resource is left untouched.
Improvements:
- Do not mark suspended resource as not ready #183
- Set field manager to
kustomize-controller
instead ofkubectl
#184
Fixes:
- Requeue after interval on source not found errors #182
Release date: 2020-11-20
This is the third MINOR prerelease. It introduces two breaking changes:
- the
ServiceAccount
field has been removed and replaced byServiceAccountName
; it is no longer possible to specify a namespace for a service account, the namespace is inferred from the Kustomization namespace - the status condition type has changed to the type introduced in Kubernetes API machinery
v1.19.0
Improvements:
- Use ServiceAccountName for impersonation #180
- Adopt Kubernetes condition type #174
- Add docs for excluding non-k8 YAML files in kustomization generation #176
- Use DeletionTimestamp for prune and readiness #177
Fixes:
- Add fsGroup to security context (fix for AWS KMS IAM Role bindings) #178
Release date: 2020-11-12
This prerelease comes with improvements to status reporting.
The Kustomization dry-run can now be explicitly disabled
by setting spec.validation
to none
.
Improvements:
Release date: 2020-11-04
This prerelease comes with improvements to garbage collection. The Kubernetes packages have been updated to v1.19.
Improvements:
- Update k8s to 1.19 + kustomize 0.6.4 #161
- Add openssh client to support git+ssh Kustomize resources #159
Fixes:
Release date: 2020-10-29
This is the second MINOR prerelease, it comes with breaking changes:
- the histogram metric
gotk_reconcile_duration
was renamed togotk_reconcile_duration_seconds
- the annotation
fluxcd.io/reconcileAt
was renamed toreconcile.fluxcd.io/requestedAt
Improvements:
- Refactor predicates and enqueuers #156 #155 #153
- Use annotation helpers #152
- Suppress health check events when no changes made #151
- Use controller-runtime utils for finalizer and health checks #150
- Improve remote cluster documentation #148
Release date: 2020-10-16
This prerelease comes with support for targeting remote clusters created with Cluster-API.
Improvements:
- Implement non-caching, per-kustomization GC-client/statusPoller for cross-cluster kubeconfigs #135
Fixes:
- Fix status reporting when the source is not found #141
- Validate manifests when generating kustomization.yaml #143
- Set correct status on failure events #145
Release date: 2020-10-13
This prerelease comes with Prometheus instrumentation for the controller's resources.
For each kind, the controller exposes a gauge metric to track the Ready
condition status,
and a histogram with the reconciliation duration in seconds:
gotk_reconcile_condition{kind, name, namespace, status, type="Ready"}
gotk_reconcile_duration{kind, name, namespace}
Release date: 2020-09-30
This is the first MINOR prerelease, it promotes the
kustomize.toolkit.fluxcd.io
API to v1beta1
and removes support for v1alpha1
.
Going forward, changes to the API will be accompanied by a conversion mechanism. With this release the API becomes more stable, but while in beta phase there are no guarantees about backwards compatibility between beta releases.
Release date: 2020-09-22
This prerelease comes with support for S3 bucket sources and
cross-namespace dependencies.
Container images for ARMv7 and ARMv8 are published to
ghcr.io/fluxcd/kustomize-controller-arm64
.
Release date: 2020-09-14
This prerelease comes with improvements to health assessment. The health checks leverage the kstatus library to support custom resources such as HelmReleases that implement the Ready condition.
Release date: 2020-09-12
This prerelease comes with the option to watch for resources in the runtime namespace of the controller or at cluster level.
Release date: 2020-09-05
This prerelease comes with a bug fix to garbage collection.
Release date: 2020-09-04
This prerelease comes with support for decrypting Kubernetes secrets with Mozilla SOPS. Container images for linux/amd64 and linux/arm64 are published to GHCR.
Release date: 2020-08-18
This prerelease upgrades the github.com/fluxcd/pkg/*
dependencies to
dedicated versioned modules, and makes the api
package available as
a dedicated versioned module.
Release date: 2020-07-31
This prerelease comes with a breaking change, the CRDs group has been
renamed to kustomize.toolkit.fluxcd.io
. The dependency on source-controller
has been updated to v0.0.7
to be able to work with source.toolkit.fluxcd.io
resources.
Release date: 2020-07-25
This prerelease comes with improvements to the dependency management and service account impersonation.
Release date: 2020-07-20
This prerelease drops support for Kubernetes <1.16.
The CRDs have been updated to apiextensions.k8s.io/v1
.
Release date: 2020-07-16
This prerelease comes with improvements to the alerting system and allows sources to be referenced across namespaces. The kustomize/api has been updated to v0.5.1.
Release date: 2020-07-13
This prerelease comes with improvements to logging.
The default logging format is JSON and the timestamp format is ISO8601.
Introduce fluxcd.io/reconcileAt
annotation for on-demand reconciliation
of kustomization objects.
Release date: 2020-07-02
This prerelease comes with improvements to the alerting system. The reconciliation events can be forwarded to notification controller and alerting can be configured for Slack, MS Teams, Discord and Rocket chat using the notification.fluxcd.io API.
Release date: 2020-06-24
This is the first prerelease ready for public testing. To get started testing, see the GitOps Toolkit guide.
Release date: 2020-06-10
This beta release allows configuring the number of concurrent reconciles. Starting with this version, the controller watches for resources only in the namespace where it's deployed.
Release date: 2020-05-29
This is the first beta release of kustomize controller. This release comes with improvements to the reconciliation engine when dealing with CRDs/CRs. The kustomize/api has been updated to v0.4.1.
Release date: 2020-05-11
This alpha release includes a bug fix for the source event handler and sets the current context to the default namespace.
Release date: 2020-05-09
This alpha release comes with improvements to health assessment and dependency management. When a source revision changes, the Kustomizations are executed based on the depends-on graph.
Release date: 2020-05-05
This alpha release comes with improvements to the garbage collector. The new GC doesn't require label selectors to be set in the kustomization and can prune resources safely without hitting Kubernetes API rate limits.
Release date: 2020-05-03
This alpha release comes with role-based access control for restricting the execution of a kustomization apply to a specific service account.
Release date: 2020-04-27
This alpha release introduces an intermediate state to the status ready condition to signal that a reconciliation is underway. This allows waiting for an on-demand sync to complete.
Release date: 2020-04-24
This alpha release introduces a new status field for recording the last applied source revision.
Feature comparison with Flux has been added to docs/spec.
Release date: 2020-04-23
This alpha release introduces the option to tell the controller to
automatically generate
the kustomization.yaml
for repositories that contain plain Kubernetes manifests.
The controller design and motivation can be found at docs/spec.
Release date: 2020-04-21
This alpha release introduces the Profile CRD that allows grouping Kustomization objects and defining a common behavior for them. The v1alpha1 profiles can be used for configuring Slack and Discord alerting.
Release date: 2020-04-20
This is the first alpha release of kustomize controller. The controller is an implementation of the kustomize.fluxcd.io/v1alpha1 API.