Release v0.3.0

Signed-off-by: Jonathan West <[email protected]>
argoproj · Dec 16, 2021 · 849d5f0 · 849d5f0
1 parent 416b213
commit 849d5f0
Show file tree

Hide file tree

Showing 19 changed files with 368 additions and 45 deletions.
diff --git a/.github/workflows/ci-build.yaml b/.github/workflows/ci-build.yaml
@@ -3,6 +3,7 @@ on:
   push:
     branches:
       - 'master'
+      - 'release-*'
   pull_request:
     branches:
       - 'master'
@@ -178,7 +179,7 @@ jobs:
           # This version is verified to match a version consistent with the kustomize install yaml,
           # by 'hack/verify-argo-cd-versions.sh'.
           # BEGIN-ARGO-CD-VERSION
-          ref: v2.1.0
+          ref: v2.2.0
           # END-ARGO-CD-VERSION
       - name: Setup Golang
         uses: actions/setup-go@v2

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,5 +1,236 @@
 # Changelog
 
+# v0.3.0
+
+I am happy to announce the latest release of the Argo CD ApplicationSet controller, v0.3.0. Many new features were contributed as part of this release, including two new generators, improved error reporting and handling, support for webhook-based refresh trigger, plus doc updates, usability improvements, stability fixes, and more. 
+
+You can learn more about this release from the [ApplicationSet documentation](https://argocd-applicationset.readthedocs.io), or check out the project repository [and learn how you can contribute](https://github.com/argoproj-labs/applicationset/).
+
+## Contributors
+
+Thanks to all the folks who have contributed to the ApplicationSet controller since our last release. 
+- Shunya Murata (@shmurata)
+- Michael Crenshaw (@crenshaw-dev)
+- Jonathan West (@jgwest)
+- Ishita Sequeira (@ishitasequeira)
+- Chetan Banavikalmutt (@chetan-rns)
+- Alexander Matyushentsev (@alexmt)
+- Shiv Jha-Mathur (@shivjm)
+- Subhash Chandra (@TMaYaD)
+- William Tam (@wtam2018)
+- Benoit Gaillard (@benoitg31) 
+- Michal Barecki (@mbarecki)
+- Guillaume Dupin (@yogeek)
+- Krzysztof Dąbrowski (@krzysdabro)
+- Olve S. Hansen (@olvesh)
+- Dewan Ishtiaque Ahmed (@dewan-ahmed)
+- Diego Pomares (@DiegoPomares)
+
+Want to join us for our next release? Check out the project repository (https://github.com/argoproj-labs/applicationset) or visit us on #argo-cd-appset on Slack (https://argoproj.github.io/community/join-slack/).
+
+## New in this release
+
+### New generator: Pull Request generator
+
+With ApplicationSet v0.3.0, a new Pull Request generator has been contributed which uses the API of an SCMaaS provider (e.g. GitHub) to automatically discover open pull requests within an repository. This fits well with users that wish to construct a test environment based on an open pull request.
+
+In this example, we will create an Argo CD `Application` resource for each open pull request:
+```yaml
+apiVersion: argoproj.io/v1alpha1
+kind: ApplicationSet
+metadata:
+  name: myapps
+spec:
+  generators:
+  - pullRequest:
+      github:
+        # The GitHub organization or user.
+        owner: myorg
+        # The Github repository
+        repo: myrepository
+        # For GitHub Enterprise (optional)
+        api: https://git.example.com/
+        # Reference to a Secret containing an access token. (optional)
+        tokenRef:
+          secretName: github-token
+          key: token
+        # Labels is used to filter the PRs that you want to target. (optional)
+        labels:
+        - preview
+  template:
+  # (template the Application using PR generator params)...
+```  
+
+To learn more, check out the [Pull Request generator documentation](https://argocd-applicationset.readthedocs.io/en/master/Generators-Pull-Request/) for details. Contributed by [@shmurata](https://github.com/argoproj-labs/applicationset/pull/366/).
+
+### New generator: Merge generator
+
+Also new in this release is the Merge generator, which is useful when you want to selectively override the parameters generated by one generator, with those generated by another.
+
+In this example, we first gather the list of clusters from Argo CD, then we 'patch' only those clusters with label `use-kakfa: false`, and finally we enable redis on a specfic cluster:
+```yaml
+apiVersion: argoproj.io/v1alpha1
+kind: ApplicationSet
+metadata:
+  name: cluster-git
+spec:
+  generators:
+    # merge 'parent' generator
+    - merge:
+        mergeKeys:
+          - server
+        generators:
+          # Generate parameters for all Argo CD clusters
+          - clusters:
+              values:
+                kafka: 'true'
+                redis: 'false'
+          # For clusters with a specific label, enable Kafka.
+          - clusters:
+              selector:
+                matchLabels:
+                  use-kafka: 'false'
+              values:
+                kafka: 'false'
+          # For a specific cluster, enable Redis.
+          - list:
+              elements: 
+                - server: https://2.4.6.8
+                  values.redis: 'true'
+```
+
+See the [Merge generator documentation](https://argocd-applicationset.readthedocs.io/en/master/Generators-Merge/) for a full example, and for details on generator behaviour. Contributed by [@crenshaw-dev](https://github.com/argoproj-labs/applicationset/pull/404).
+
+### Report error conditions/status for ApplicationSet CR
+   
+When the user-provided generator/template produce invalid Argo CD Applications, the `ApplicationSet` resource's status field will now report errors (or the lack thereof). Here is an example of the new status conditions:
+```yaml
+apiVersion: argoproj.io/v1alpha1
+kind: ApplicationSet
+metadata:
+  name: myapps
+spec:
+  generators: # (...)
+  template: # (...)
+status:
+  conditions:
+  - lastTransitionTime: "2021-11-23T05:47:08Z"
+    type: ErrorOccurred
+    status: "False"
+    reason: ApplicationSetUpToDate
+    message: Successfully generated parameters for all Applications
+  - lastTransitionTime: "2021-11-23T05:47:08Z"
+    type: ParametersGenerated
+    message: Successfully generated parameters for all Applications
+    reason: ParametersGenerated
+    status: "True"
+  - lastTransitionTime: "2021-11-23T05:47:08Z"
+    type: ResourcesUpToDate
+    status: "True"
+    reason: ApplicationSetUpToDate
+    message: ApplicationSet up to date
+```
+
+On parameter generation failure or templating failure, those errors will be reported under the appropriate conditions. Contributed by [@ishitasequeira](https://github.com/argoproj-labs/applicationset/pull/370).
+
+
+### Git Generator: Refresh ApplicationSet resource with Git generator using webhook
+
+This feature adds support for refreshing ApplicationSets via a GitHub webhook trigger. It exposes a service which listens for incoming webhook payloads, and once received triggers the ApplicationSet controller to regenerate resources. In contrast, with the previous release, the ApplicationSet controller only supported polling the Git repository used by the Git generator every 3 mins (but this is at least customizable).
+
+See the [webhook documentation](https://argocd-applicationset.readthedocs.io/en/master/Generators-Git/#webhook-configuration) for details. Contributed by [@chetan-rns](https://github.com/argoproj-labs/applicationset/pull/341).
+
+This contribution also adds general support for webhooks, which is used by the Pull Request generator webhook code, below.
+
+### Gracefully handle application validation errors
+
+This feature changes how the ApplicationSet controller handles ApplicationSets that generate invalid `Application`s. Previously, if at least one Application in the ApplicationSet was invalid, the controller would refuse to proceed further and would skip _all_ Application processing (i.e. it would 'fail fast'). Now, the controller will process *valid* Applications, and only skip *invalid* Applications (logging information about them to the console).
+
+Contributed by [@alexmt](https://github.com/argoproj-labs/applicationset/pull/372).
+
+### Pull Request generator: Support for webhooks
+
+When using a Pull Request generator, the ApplicationSet controller polls every `requeueAfterSeconds` interval (defaulting to every 30 minutes) to detect changes. To eliminate this delay from polling, the ApplicationSet webhook server can be configured to receive webhook events, which will refresh the parameters generated by the Pull Request generator, and thus the corresponding `Application` resources.
+
+More information on configuring webhooks with the Pull Request generator is available from the [Pull Request generator documentation](https://argocd-applicationset.readthedocs.io/en/master/Generators-Pull-Request/#webhook-configuration). Contributed by [@shmurata](https://github.com/argoproj-labs/applicationset/pull/417).
+
+### Support `-logformat=json` as parameter to applicationset-controller
+
+This feature adds a new `--logformat=json` parameter to the applicationset-controller, which switches the logging output of the ApplicationSet controller to JSON. Contributed by [@shivjm](https://github.com/argoproj-labs/applicationset/pull/373).
+
+### SCM Generator: Provide SHA for latest commit on a branch in variables (#307)
+
+This feature adds SHA to the list of parameters exposed by the SCM Generator, with the SHA parameter representing the latest commit. Contributed by [@TMaYaD](https://github.com/argoproj-labs/applicationset/pull/307).
+
+### Improve Git files generator performance (#355)
+
+The Git files generator was consuming too much time (and driving up Git requests) due to inadvertently executing 'git fetch/git checkout' for each discovered file within the repository. With ApplicationSet v0.3.0, that has improved such that we will now issue a Git checkout/fetch repo once per refresh. Contributed by [@alexmt](https://github.com/argoproj-labs/applicationset/pull/355).
+
+### Fixes, test fixes, infrastructure improvements, and documentation updates
+
+#### Fixes
+- Fix: new variable for the normalized version of name field ([#390](https://github.com/argoproj-labs/applicationset/pull/390), contributed by @chetan-rns)
+- Fixes GitLab RepoHasPath error handling ([#423](https://github.com/argoproj-labs/applicationset/pull/423), contributed by @benoitg31)
+
+
+#### Test/infrastructure improvements:
+- Investigate Argo CD deletion failure messages when running ApplicationSet E2E tests in on-cluster configuration ([#392](https://github.com/argoproj-labs/applicationset/pull/392), contributed by @jgwest)
+- Update master branch VERSION file and metadata, and pull up release changes from 0.2.0 ([#343](https://github.com/argoproj-labs/applicationset/pull/343), contributed by @jgwest)
+- Skip E2E tests that require GitHub token, if not specified ([#380](https://github.com/argoproj-labs/applicationset/pull/380), contributed by @jgwest)
+- API rate limit error in image publish action ([#368](https://github.com/argoproj-labs/applicationset/pull/368), contributed by @jgwest)
+- Disable SCM Provider Unit tests on PRs ([#337](https://github.com/argoproj-labs/applicationset/pull/337), contributed by @jgwest)
+- Fix lint-docs ([#411](https://github.com/argoproj-labs/applicationset/pull/411), contributed by @crenshaw-dev)
+- Fix indentation in example ([#360](https://github.com/argoproj-labs/applicationset/pull/360), contributed by @DiegoPomares)
+- Adding required 'version' field for Helm Charts ([#332](https://github.com/argoproj-labs/applicationset/pull/332), contributed by @dewan-ahmed)
+- Adopt latest Argo CD dependencies, in preparation for next release ([#410](https://github.com/argoproj-labs/applicationset/pull/410), contributed by @jgwest)
+
+
+#### Doc updates
+- Update release process docs and include release checklist in docs ([#365](https://github.com/argoproj-labs/applicationset/pull/365), contributed by @jgwest)
+- Fix includeSubgroups reference name ([#357](https://github.com/argoproj-labs/applicationset/pull/357), contributed by @yogeek)
+- Add missing brace ([#349](https://github.com/argoproj-labs/applicationset/pull/349), contributed by @krzysdabro)
+- Fix Git Generator Files path example in docs ([#408](https://github.com/argoproj-labs/applicationset/pull/408), contributed by @mbarecki)
+- Corrected wrong info about path and path.basename ([#412](https://github.com/argoproj-labs/applicationset/pull/412), contributed by @olvesh)
+
+
+
+## Upgrade Notes
+
+When moving from ApplicationSet v0.1.0/v0.2.0, to v0.3.0, there are two behaviour changes to be aware of.
+
+#### Cluster generator: `{{name}}` parameter value will no longer be normalized, but existing normalization behaviour is preserved in a new `{{nameNormalized}}` parameter
+
+The Cluster generator `{{name}}` parameter has now reverted to its original behaviour: the cluster name within Argo CD will no longer be [normalized](https://github.com/argoproj-labs/applicationset/blob/11f1fe893b019c9a530865fa83ee78b16af2c090/pkg/generators/cluster.go#L168). The `{{name}}` parameter generated by the Cluster generator within the ApplicationSet will now be passed unmodified to the ApplicationSet template. 
+
+A new parameter, `{{nameNormalized}}` has been introduced which preserves the 0.2.0 behaviour. This allows you to choose which behaviour you wish to use in your ApplicationSet, based on the context in which it is used: either using the parameter as defined, or in a normalized form (which allows it to be used in the `name` field of an `Application` resource.)
+
+If your Argo CD cluster names are already valid, no change is required. Otherwise, to preserve the v0.2.0 behaviour of your ApplicationSet, replace `{{name}}` with `{{nameNormalized}}` within your ApplicationSet template. 
+
+More information about this change is [available from the issue](https://github.com/argoproj-labs/applicationset/pull/390).
+
+
+#### If an ApplicationSet contains an invalid generated Application, the valid generated Applications will still be processed
+
+The responsibility of the ApplicationSet controller is to convert an `ApplicationSet` resource into one or more `Application` resources. However, with the previous releases, if at least one of the generated `Application` resources was invalid (e.g. it failed the internal validation logic), none of the generated Applications would be processed (they would not be neither created nor modified).
+
+With the latest ApplicationSet release, if a generator generates invalid Applications, those invalid generated Applications will still be skipped, **but** the valid generated Applications will now be processed (created/modified).
+
+Thus no `ApplicationSet` resource changes are required by this new behaviour, but it is worth keeping in mind that your ApplicationSets which were previously blocked by a failing Application may no longer be blocked. This change might cause valid Applications to now be created/modified, whereas previously they were prevented from being processed.
+
+More information about this change is [available from the issue](https://github.com/argoproj-labs/applicationset/pull/372).
+
+
+## Installation
+
+The ApplicationSet controller must be installed into the same namespace as the Argo CD it is targeting:
+```
+kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj-labs/applicationset/v0.3.0/manifests/install.yaml
+```
+
+Once installed, the ApplicationSet controller requires no additional setup. You can learn more about ApplicationSet controller installation from the [Getting Started](https://argocd-applicationset.readthedocs.io/en/v0.3.0/Getting-Started/) page.
+
+
+
 # v0.2.0
 
 I am happy to announce the second release of the Argo CD ApplicationSet controller, v0.2.0. Many new features were contributed as part of this release, including support for combining generator parameters, support for building Argo CD Applications based on GitHub/GitLab organizations, and support for using custom resources to select clusters, plus oft requested improvements to existing generators, and of course doc updates, usability improvements, stability fixes, and more. 

diff --git a/docs/Generators-Cluster.md b/docs/Generators-Cluster.md
@@ -7,6 +7,7 @@ For each cluster registered with Argo CD, the Cluster generator produces paramet
 It automatically provides the following parameter values to the Application template for each cluster:
 
 - `name`
+- `nameNormalized` *('name' but normalized to contain only lowercase alphanumeric characters, '-' or '.')*
 - `server`
 - `metadata.labels.<key>` *(for each label in the Secret)*
 - `metadata.annotations.<key>` *(for each annotation in the Secret)*

diff --git a/docs/Generators-Git.md b/docs/Generators-Git.md
@@ -235,7 +235,7 @@ Git webhook notifications from GitHub and GitLab. The following explains how to
 !!! note
     ApplicationSet exposes the webhook server as a service of type ClusterIP. An Ingress resource needs to be created to expose this service to the webhook source.
 
-### 1. Create The WebHook In The Git Provider
+### 1. Create the webhook in the Git provider
 
 In your Git provider, navigate to the settings page where webhooks can be configured. The payload
 URL configured in the Git provider should use the `/api/webhook` endpoint of your ApplicationSet instance
@@ -247,7 +247,7 @@ arbitrary value in the secret. This value will be used when configuring the webh
 !!! note
     When creating the webhook in GitHub, the "Content type" needs to be set to "application/json". The default value "application/x-www-form-urlencoded" is not supported by the library used to handle the hooks
 
-### 2. Configure ApplicationSet With The WebHook Secret (Optional)
+### 2. Configure ApplicationSet with the webhook secret (Optional)
 
 Configuring a webhook shared secret is optional, since ApplicationSet will still refresh applications
 generated by Git generators, even with unauthenticated webhook events. This is safe to do since

diff --git a/docs/Generators-Merge.md b/docs/Generators-Merge.md
@@ -1,10 +1,10 @@
 # Merge Generator
 
-The merge generator combines parameters produced by the base (first) generator with matching parameter sets produced by subsequent generators. A _matching_ parameter set has the same values for the configured _merge keys_. _Non-matching_ parameter sets are discarded. Override precedence is bottom-to-top. The values from a matching parameter set produced by generator 3 will take precedence over the values from the corresponding parameter set produced by generator 2.
+The Merge generator combines parameters produced by the base (first) generator with matching parameter sets produced by subsequent generators. A _matching_ parameter set has the same values for the configured _merge keys_. _Non-matching_ parameter sets are discarded. Override precedence is bottom-to-top: the values from a matching parameter set produced by generator 3 will take precedence over the values from the corresponding parameter set produced by generator 2.
 
-Using a merge generator is appropriate when a subset of parameter sets require overriding.
+Using a Merge generator is appropriate when a subset of parameter sets require overriding.
 
-## Example: base cluster generator + override cluster generator + list generator 
+## Example: Base Cluster generator + override Cluster generator + List generator 
 
 As an example, imagine that we have two clusters:
 
@@ -130,7 +130,7 @@ When merged with the updated base parameters, the `values.redis` value for the p
             - # (...)
           template: { } # Not processed
 ```
-1. Combination-type generators (matrix or merge) can only be nested once. For example, this will not work:
+1. Combination-type generators (Matrix or Merge) can only be nested once. For example, this will not work:
 ```yaml
 - merge:
     generators: