Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨ adding helm chart for csi-driver #398

Closed
wants to merge 1 commit into from

Conversation

batistein
Copy link
Contributor

What this PR does:

  • Adds a helm chart for the hcloud-csi driver.
  • Fixes feat: Publish Helm Chart #369
  • Adds the possibility to define own storageclasses
  • Enable or disable Metrics and the use of a serviceMonitor Object
  • Adds the possibility to add resources requests and limits to each container
  • Adds an minimal production-ready example how to use the helm chart without monitoring
  • Adds documentation how to configure the helm chart

Things to discuss

Currently, the helm chart is also available in the following repository: https://github.com/syself/charts. While it's possible to use the remote variant of the chart from there, we would like to contribute this helm chart to the "upstream" repository. However, this repository do not have a helm chart publishing process in place.

For the time being, I have left the chart as it is in our helm chart repository so that the community can use a hosted version. I would suggest to revisit this in the future once a publishing process for the helm chart in the csi-driver repository is implemented.

@samcday
Copy link
Contributor

samcday commented Apr 13, 2023

I'd prefer that this PR lands in a way similar to how hccm did: by moving the current canonical Kustomize/vanilla manifests into the Helm chart, and then adding a build step somewhere that helm templates it back out into static manifests suitable for a quick kubectl apply -f ... "default installation".

I can add some commits to this PR that do what I just described, if you like.

If you're interested in doing it, then what I imagine is basically just git rm -rf deploy/kubernetes/*, and add a script to do helm template hcloud-csi-driver > deploy/kubernetes/hcloud-csi.yaml. In hccm we took the opportunity to start with some more appropriate defaults in the Helm chart, which required some explicit backwards-compat values to ensure the "legacy" static manifests did not introduce any breaking changes.

Yes, I'm proposing we go back to how csi-driver k8s manifests looked before someone went in and made a mess of things ;)

deploy/charts/hcloud-csi/Chart.yaml Show resolved Hide resolved
deploy/charts/hcloud-csi/values.yaml Show resolved Hide resolved
deploy/charts/hcloud-csi/values.yaml Show resolved Hide resolved
deploy/charts/hcloud-csi/values.yaml Show resolved Hide resolved
deploy/charts/hcloud-csi/values.yaml Show resolved Hide resolved
deploy/charts/hcloud-csi/values.yaml Show resolved Hide resolved
deploy/charts/hcloud-csi/Chart.yaml Show resolved Hide resolved
@apricote
Copy link
Member

apricote commented May 3, 2023

I'd prefer that this PR lands in a way similar to how hccm did: by moving the current canonical Kustomize/vanilla manifests into the Helm chart, and then adding a build step somewhere that helm templates it back out into static manifests suitable for a quick kubectl apply -f ... "default installation".

I can add some commits to this PR that do what I just described, if you like.

If you're interested in doing it, then what I imagine is basically just git rm -rf deploy/kubernetes/*, and add a script to do helm template hcloud-csi-driver > deploy/kubernetes/hcloud-csi.yaml. In hccm we took the opportunity to start with some more appropriate defaults in the Helm chart, which required some explicit backwards-compat values to ensure the "legacy" static manifests did not introduce any breaking changes.

Yes, I'm proposing we go back to how csi-driver k8s manifests looked before someone went in and made a mess of things ;)

I tested this with the current manifests in the PR (and my review suggestions applied) and found that the labels are currently blocking this. The chart hardcodes the standardized labels in deploy/charts/hcloud-csi/tempaltes/_common_labels.tpl, and those conflict with our existing label (which is just app: hcloud-csi(-controller)). If this were configurable, we can easily render out the static templates from the chart.

There were some other changes, but they are negligible IMO.

Full Diff

Your diff will probably be different, I already applied the changes I suggested in a review.

$ dyff between <(kustomize build deploy/) <(helm template deploy/charts/hcloud-csi --set fullnameOverride=hcloud-csi --set pdb.create=false --set metrics.enabled=true --namespace kube-system )
     _        __  __
   _| |_   _ / _|/ _|  between /proc/self/fd/11, nine documents
 / _' | | | | |_| |_       and /proc/self/fd/12, nine documents
| (_| | |_| |  _|  _|
 \__,_|\__, |_| |_|   returned 39 differences
        |___/

(file level)
  ⇆ order changed
    StorageClass/default/hcloud-volumes                 ServiceAccount/kube-system/hcloud-csi-controller
    ServiceAccount/kube-system/hcloud-csi-controller    StorageClass/default/hcloud-volumes
    ClusterRole/default/hcloud-csi-controller           ClusterRole/default/hcloud-csi-controller
    ClusterRoleBinding/default/hcloud-csi-controller    ClusterRoleBinding/default/hcloud-csi-controller
    Service/kube-system/hcloud-csi-controller-metrics   Service/kube-system/hcloud-csi-controller-metrics
    Service/kube-system/hcloud-csi-node-metrics         Service/kube-system/hcloud-csi-node-metrics
    Deployment/kube-system/hcloud-csi-controller        DaemonSet/kube-system/hcloud-csi-node
    DaemonSet/kube-system/hcloud-csi-node               Deployment/kube-system/hcloud-csi-controller
    CSIDriver/default/csi.hetzner.cloud                 CSIDriver/default/csi.hetzner.cloud
(root level)  (StorageClass/default/hcloud-volumes)
+ one map entry added:
  reclaimPolicy: Retain

metadata  (ServiceAccount/kube-system/hcloud-csi-controller)
  + one map entry added:
    labels:
    │ app.kubernetes.io/name: hcloud-csi
    │ helm.sh/chart: hcloud-csi-1.0.0
    │ app.kubernetes.io/instance: release-name
    │ app.kubernetes.io/managed-by: Helm
    │ app.kubernetes.io/component: controller

metadata  (ClusterRole/default/hcloud-csi-controller)
  + one map entry added:
    labels:
    │ app.kubernetes.io/name: hcloud-csi
    │ helm.sh/chart: hcloud-csi-1.0.0
    │ app.kubernetes.io/instance: release-name
    │ app.kubernetes.io/managed-by: Helm
    │ app.kubernetes.io/component: controller

metadata  (ClusterRoleBinding/default/hcloud-csi-controller)
  + one map entry added:
    labels:
    │ app.kubernetes.io/name: hcloud-csi
    │ helm.sh/chart: hcloud-csi-1.0.0
    │ app.kubernetes.io/instance: release-name
    │ app.kubernetes.io/managed-by: Helm
    │ app.kubernetes.io/component: controller

metadata.labels  (Service/kube-system/hcloud-csi-controller-metrics)
  - one map entry removed:         + five map entries added:
    app: hcloud-csi-controller       app.kubernetes.io/name: hcloud-csi
                                     helm.sh/chart: hcloud-csi-1.0.0
                                     app.kubernetes.io/instance: release-name
                                     app.kubernetes.io/managed-by: Helm
                                     app.kubernetes.io/component: controller

spec.ports.metrics  (Service/kube-system/hcloud-csi-controller-metrics)
  - one map entry removed:
    targetPort: metrics

spec.selector  (Service/kube-system/hcloud-csi-controller-metrics)
  - one map entry removed:         + three map entries added:
    app: hcloud-csi-controller       app.kubernetes.io/name: hcloud-csi
                                     app.kubernetes.io/instance: release-name
                                     app.kubernetes.io/component: controller

metadata.labels  (Service/kube-system/hcloud-csi-node-metrics)
  - one map entry removed:     + five map entries added:
    app: hcloud-csi              app.kubernetes.io/name: hcloud-csi
                                 helm.sh/chart: hcloud-csi-1.0.0
                                 app.kubernetes.io/instance: release-name
                                 app.kubernetes.io/managed-by: Helm
                                 app.kubernetes.io/component: node

spec.ports.metrics  (Service/kube-system/hcloud-csi-node-metrics)
  - one map entry removed:
    targetPort: metrics

spec.selector  (Service/kube-system/hcloud-csi-node-metrics)
  - one map entry removed:     + three map entries added:
    app: hcloud-csi              app.kubernetes.io/name: hcloud-csi
                                 app.kubernetes.io/instance: release-name
                                 app.kubernetes.io/component: node

metadata  (Deployment/kube-system/hcloud-csi-controller)
  + one map entry added:
    labels:
    │ app.kubernetes.io/name: hcloud-csi
    │ helm.sh/chart: hcloud-csi-1.0.0
    │ app.kubernetes.io/instance: release-name
    │ app.kubernetes.io/managed-by: Helm
    │ app.kubernetes.io/component: controller

spec  (Deployment/kube-system/hcloud-csi-controller)
  + one map entry added:
    strategy:
    │ type: RollingUpdate

spec.selector.matchLabels  (Deployment/kube-system/hcloud-csi-controller)
  - one map entry removed:         + three map entries added:
    app: hcloud-csi-controller       app.kubernetes.io/name: hcloud-csi
                                     app.kubernetes.io/instance: release-name
                                     app.kubernetes.io/component: controller

spec.template.metadata.labels  (Deployment/kube-system/hcloud-csi-controller)
  - one map entry removed:         + five map entries added:
    app: hcloud-csi-controller       app.kubernetes.io/name: hcloud-csi
                                     helm.sh/chart: hcloud-csi-1.0.0
                                     app.kubernetes.io/instance: release-name
                                     app.kubernetes.io/managed-by: Helm
                                     app.kubernetes.io/component: controller

spec.template.spec  (Deployment/kube-system/hcloud-csi-controller)
  + two map entries added:
    securityContext:
    │ fsGroup: 1001
    initContainers:

spec.template.spec.containers  (Deployment/kube-system/hcloud-csi-controller)
  ⇆ order changed
    - csi-attacher, csi-resizer, csi-provisioner, hcloud-csi-driver, liveness-probe
    + csi-attacher, csi-resizer, csi-provisioner, liveness-probe, hcloud-csi-driver

spec.template.spec.containers.csi-attacher  (Deployment/kube-system/hcloud-csi-controller)
  + two map entries added:
    resources:
    │ limits: {}
    │ requests: {}
    imagePullPolicy: IfNotPresent

spec.template.spec.containers.csi-resizer  (Deployment/kube-system/hcloud-csi-controller)
  + two map entries added:
    resources:
    │ limits: {}
    │ requests: {}
    imagePullPolicy: IfNotPresent

spec.template.spec.containers.csi-provisioner  (Deployment/kube-system/hcloud-csi-controller)
  + two map entries added:
    resources:
    │ limits: {}
    │ requests: {}
    imagePullPolicy: IfNotPresent

spec.template.spec.containers.hcloud-csi-driver  (Deployment/kube-system/hcloud-csi-controller)
  + one map entry added:
    resources:
    │ limits: {}
    │ requests: {}

spec.template.spec.containers.hcloud-csi-driver.image  (Deployment/kube-system/hcloud-csi-controller)
  ± value change
    - hetznercloud/hcloud-csi-driver:latest
    + docker.io/hetznercloud/hcloud-csi-driver:2.2.0

spec.template.spec.containers.hcloud-csi-driver.imagePullPolicy  (Deployment/kube-system/hcloud-csi-controller)
  ± value change
    - Always
    + IfNotPresent

spec.template.spec.containers.hcloud-csi-driver.livenessProbe  (Deployment/kube-system/hcloud-csi-controller)
  + one map entry added:
    successThreshold: 1

spec.template.spec.containers.liveness-probe  (Deployment/kube-system/hcloud-csi-controller)
  + one map entry added:
    resources:
    │ limits: {}
    │ requests: {}

spec.template.spec.containers.liveness-probe.imagePullPolicy  (Deployment/kube-system/hcloud-csi-controller)
  ± value change
    - Always
    + IfNotPresent

metadata.labels  (DaemonSet/kube-system/hcloud-csi-node)
  - one map entry removed:     + five map entries added:
    app: hcloud-csi              app.kubernetes.io/name: hcloud-csi
                                 helm.sh/chart: hcloud-csi-1.0.0
                                 app.kubernetes.io/instance: release-name
                                 app.kubernetes.io/managed-by: Helm
                                 app.kubernetes.io/component: node

spec  (DaemonSet/kube-system/hcloud-csi-node)
  + one map entry added:
    updateStrategy:
    │ type: RollingUpdate

spec.selector.matchLabels  (DaemonSet/kube-system/hcloud-csi-node)
  - one map entry removed:     + three map entries added:
    app: hcloud-csi              app.kubernetes.io/name: hcloud-csi
                                 app.kubernetes.io/instance: release-name
                                 app.kubernetes.io/component: node

spec.template.metadata.labels  (DaemonSet/kube-system/hcloud-csi-node)
  - one map entry removed:     + five map entries added:
    app: hcloud-csi              app.kubernetes.io/name: hcloud-csi
                                 helm.sh/chart: hcloud-csi-1.0.0
                                 app.kubernetes.io/instance: release-name
                                 app.kubernetes.io/managed-by: Helm
                                 app.kubernetes.io/component: node

spec.template.spec  (DaemonSet/kube-system/hcloud-csi-node)
  + two map entries added:
    securityContext:
    │ fsGroup: 1001
    initContainers:

spec.template.spec.containers  (DaemonSet/kube-system/hcloud-csi-node)
  ⇆ order changed
    - csi-node-driver-registrar, hcloud-csi-driver, liveness-probe
    + csi-node-driver-registrar, liveness-probe, hcloud-csi-driver

spec.template.spec.containers.csi-node-driver-registrar  (DaemonSet/kube-system/hcloud-csi-node)
  + two map entries added:
    resources:
    │ limits: {}
    │ requests: {}
    imagePullPolicy: IfNotPresent

spec.template.spec.containers.hcloud-csi-driver  (DaemonSet/kube-system/hcloud-csi-node)
  + one map entry added:
    resources:
    │ limits: {}
    │ requests: {}

spec.template.spec.containers.hcloud-csi-driver.image  (DaemonSet/kube-system/hcloud-csi-node)
  ± value change
    - hetznercloud/hcloud-csi-driver:latest
    + docker.io/hetznercloud/hcloud-csi-driver:2.2.0

spec.template.spec.containers.hcloud-csi-driver.imagePullPolicy  (DaemonSet/kube-system/hcloud-csi-node)
  ± value change
    - Always
    + IfNotPresent

spec.template.spec.containers.hcloud-csi-driver.livenessProbe  (DaemonSet/kube-system/hcloud-csi-node)
  + one map entry added:
    successThreshold: 1

spec.template.spec.containers.liveness-probe  (DaemonSet/kube-system/hcloud-csi-node)
  + one map entry added:
    resources:
    │ limits: {}
    │ requests: {}

spec.template.spec.containers.liveness-probe.imagePullPolicy  (DaemonSet/kube-system/hcloud-csi-node)
  ± value change
    - Always
    + IfNotPresent

@apricote
Copy link
Member

@batistein Would it be okay for you if I take over this PR and make the requested changes myself?

@batistein
Copy link
Contributor Author

Sure I'm a bit lacking time at the moment!

@apricote
Copy link
Member

No worries :) You already spent quite the time getting this up initially. Just want to get this over the finish line and close one item in my backlog 👍

(And maybe clean up my own cluster configuration while i am at it..)

@apricote apricote self-assigned this Sep 12, 2023
@apricote apricote added the enhancement New feature or request label Sep 12, 2023
@apricote
Copy link
Member

This PR is continued in #500. Thanks a lot @batistein for contributing the initial version :)

@apricote apricote closed this Sep 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

feat: Publish Helm Chart
3 participants