[bug] extra newline in config files #475

ishioni · 2023-11-18T22:39:38Z

Describe your Issue

Chart version 4.5.1 replaced indent with nindent for config files. This unfortunately ends up rendering those config files with a newline prepended

Logs and Errors

As you can see in my local flux repo PR, extra config files are now rendered with a newline and a |2- instead of a |-, and an added newline as a bonus. This in turns errors out nextcloud with an error

Config file has leading content, please remove everything before "<?php" in local.config.php
Fatal error: Uncaught Error: Typed static property OC::$server must not be accessed before initialization in /var/www/html/index.php:71 Stack trace: #0 {main} thrown in /var/www/html/index.php on line 71

Describe your Environment

Kubernetes distribution: talos v1.5.5
Helm Version (or App that manages helm): flux v2.1.2
Helm Chart Version: 4.5.1
values.yaml:

deploymentAnnotations:
  secret.reloader.stakater.com/reload: &secret nextcloud-secret
image:
  repository: public.ecr.aws/docker/library/nextcloud
  tag: 27.1.3-fpm-alpine
nextcloud:
  host: &host "files.${SECRET_DOMAIN}"
  extraInitContainers:
    - name: init-db
      image: ghcr.io/onedr0p/postgres-init:14.9
      envFrom:
        - secretRef:
            name: *secret
  extraEnv:
    - name: S3_ACCESS_KEY
      valueFrom:
        secretKeyRef:
          name: *secret
          key: S3_ACCESS_KEY
    - name: S3_SECRET_KEY
      valueFrom:
        secretKeyRef:
          name: *secret
          key: S3_SECRET_KEY
    - name: OIDC_CLIENT_ID
      valueFrom:
        secretKeyRef:
          name: *secret
          key: OIDC_CLIENT_ID
    - name: OIDC_CLIENT_PASSWORD
      valueFrom:
        secretKeyRef:
          name: *secret
          key: OIDC_CLIENT_PASSWORD
  existingSecret:
    enabled: true
    secretName: *secret
    usernameKey: ADMIN_USER
    passwordKey: ADMIN_PASS
    tokenKey: METRICS_TOKEN
    smtpHostKey: SMTP_HOST
    smtpUsernameKey: SMTP_USERNAME
    smtpPasswordKey: SMTP_PASSWORD
  mail:
    enabled: true
    fromAddress: files
    domain: "${SECRET_DOMAIN}"
    smtp:
      secure: ssl
      port: 465
      authtype: LOGIN
  securityContext:
    runAsUser: 568
    runAsGroup: 568
    fsGroup: 568
    fsGroupChangePolicy: "OnRootMismatch"
    runAsNonRoot: true
  configs:
    local.config.php: |-
      <?php
      $CONFIG = array (
        'trusted_proxies' =>
        array (
          0 => '127.0.0.1',
          1 => '172.16.0.0/16',
        ),
        'forwarded_for_headers' =>
        array (
          0 => 'HTTP_X_FORWARDED_FOR',
        ),
        'default_phone_region' => 'PL',
        'trashbin_retention_obligation' => 'auto, 30',
        'auth.bruteforce.protection.enabled' => true,
      );
    sso.config.php: |-
      <?php
      $CONFIG = array (
        'allow_user_to_change_display_name' => false,
        'lost_password_link' => 'disabled',
        'oidc_login_client_id' => getenv('OIDC_CLIENT_ID'),
        'oidc_login_client_secret' => getenv('OIDC_CLIENT_PASSWORD'),
        'oidc_login_provider_url' => 'https://auth.${SECRET_DOMAIN}/application/o/nextcloud/',
        'oidc_login_end_session_redirect' => true,
        'oidc_login_logout_url' => 'https://auth.${SECRET_DOMAIN}/application/o/nextcloud/end-session/',
        'oidc_login_default_quota' => '1000000000',
        'oidc_login_hide_password_form' => true,
        'oidc_login_disable_registration' => false,
        'oidc_login_webdav_enabled' => true,
        'oidc_login_attributes' => array (
              'id' => 'sub',
              'name' => 'name',
              'mail' => 'email',
              'quota' => 'nextcloudQuota',
              'is_admin' => 'nextcloudAdmin',
          ),
        'oidc_login_scope' => 'openid profile email nextcloud',
        'oidc_login_default_group' => 'cloud',
        'oidc_create_groups' => true,
        'oidc_login_code_challenge_method' => 'S256',
        'oidc_login_auto_redirect' => true, //login?noredir=1
      );
    s3.config.php: |-
      <?php
      $CONFIG = array (
        'objectstore' => array(
          'class' => '\\OC\\Files\\ObjectStore\\S3',
          'arguments' => array(
            'hostname'       => 's3.services.${SECRET_DOMAIN}',
            'port'           => 9000,
            'use_path_style' => true,
            'bucket'         => getenv('S3_ACCESS_KEY'),
            'autocreate'     => false,
            'key'            => getenv('S3_ACCESS_KEY'),
            'secret'         => getenv('S3_SECRET_KEY'),
            'use_ssl'        => true,
          ),
        ),
      );
  phpConfigs:
    uploadLimit.ini: |
      upload_max_filesize = 16G
      post_max_size = 16G
      max_input_time = 3600
      max_execution_time = 3600
    www.conf: |
      [www]
      user = www-data
      group = www-data
      listen = 127.0.0.1:9000
      pm = dynamic
      pm.max_children = 57
      pm.start_servers = 14
      pm.min_spare_servers = 14
      pm.max_spare_servers = 42
      pm.max_requests = 500
      pm.process_idle_timeout = 10s
persistence:
  enabled: true
  existingClaim: nextcloud-app
resources:
  requests:
    cpu: 200m
    memory: 300Mi
  limits:
    memory: 2000Mi
startupProbe:
  enabled: true
  initialDelaySeconds: 5
  periodSeconds: 10
  timeoutSeconds: 5
  failureThreshold: 120
  successThreshold: 1
livenessProbe:
  enabled: true
  initialDelaySeconds: 5
  periodSeconds: 10
  timeoutSeconds: 5
  failureThreshold: 3
  successThreshold: 1
readinessProbe:
  enabled: true
  initialDelaySeconds: 5
  periodSeconds: 10
  timeoutSeconds: 5
  failureThreshold: 3
  successThreshold: 1
cronjob:
  enabled: true
ingress:
  enabled: true
  className: public
  annotations:
    nginx.ingress.kubernetes.io/custom-http-errors: "418"
    hajimari.io/enable: "true"
    hajimari.io/appName: Nextcloud
    hajimari.io/icon: simple-icons:nextcloud
    hajimari.io/group: groupware
  tls:
    - secretName: *host
      hosts:
        - *host
nginx:
  enabled: true
  image:
    repository: public.ecr.aws/nginx/nginx
    tag: "1.25.3"
  config:
    default: true
  resources:
    requests:
      cpu: 50m
      memory: 20Mi
    limits:
      memory: 100Mi
internalDatabase:
  enabled: false
externalDatabase:
  enabled: true
  type: postgresql
  database: nextcloud
  existingSecret:
    enabled: true
    secretName: *secret
    hostKey: POSTGRES_HOST
    databaseKey: POSTGRES_DB
    usernameKey: POSTGRES_USER
    passwordKey: POSTGRES_PASSWORD
redis:
  enabled: true
  architecture: standalone
  auth:
    enabled: true
    existingSecret: *secret
    existingSecretPasswordKey: REDIS_PASSWORD
  master:
    persistence:
      enabled: false
    resources:
      requests:
        cpu: 50m
        memory: 50Mi
      limits:
        memory: 100Mi
  commonConfiguration: |-
    # Enable AOF https://redis.io/topics/persistence#append-only-file
    appendonly yes
    # Disable RDB persistence, AOF persistence already enabled.
    save ""
    maxmemory 94371840
    maxmemory-policy allkeys-lru
  metrics:
    enabled: true
    serviceMonitor:
      enabled: true
    resources:
      requests:
        cpu: 10m
        memory: 10Mi
      limits:
        memory: 20Mi
metrics:
  enabled: false
  https: true
  image:
    repository: docker.io/xperimental/nextcloud-exporter
    tag: 0.6.2
  serviceMonitor:
    enabled: false

Additional context, if any

The text was updated successfully, but these errors were encountered:

DanishVaid · 2023-11-18T22:55:58Z

+1 - this break was introduced in this PR here: #465

I also added a comment on that PR to expand upon the issue being seen

nafets227 · 2023-11-19T08:40:10Z

+1 same issue here, rolling back to 4.5.0

jessebot · 2023-11-19T09:38:17Z

looking! Apologies for the inconvenience!

Update: looks like we have a potential fix in #465.

Update again: I marked the current release as a pre-release, meaning not ready for production, and linked to both this bug and the PR that is in the works to fix it 🙏

djbobyd · 2023-11-19T15:14:24Z

Hello, as I think my issue is related I decided not to open another one. I noticed that the extraVolumes were broken in 4.5.1:
I was getting the following error when trying to upgrade:
YAML parse error on nextcloud/templates/deployment.yaml: error converting YAML to JSON: yaml: line 192: did not find expected key
I did a little digging and what I found is that the nindent in the deployment template for the extraVolumes was wrong. It is currently:

{{- with .Values.nextcloud.extraVolumes }}
{{- toYaml . | nindent 6 }}
{{- end }}

when I change it to "8" the template renders without any issues. Hope you can fix this as well with the above PR.
Thanks!

wrenix · 2023-11-19T17:12:34Z

good hint @djbobyd i but it also into #476

(PS: @ishioni Nice bot which makes an helm diff in context of fluxcd - what bot software do you use for it?)

ishioni · 2023-11-19T18:04:44Z

@wrenix https://github.com/ishioni/homelab-ops/blob/master/.github/workflows/flux-diff.yaml

DanishVaid · 2023-11-19T21:33:49Z

This was meant for the PR with the fix

Hmm, I'm trying to test but seeing an odd error during chart rendering (shown below). I don't think this is tied to your change. Seeing if I can figure out what it is

Error: template: test/charts/postgresql/templates/primary/svc.yaml:11:14: executing "test/charts/postgresql/templates/primary/svc.yaml" at <include "common.labels.standard" (dict "customLabels" .Values.commonLabels "context" $)>: error calling include: template: test/charts/mariadb/charts/common/templates/_labels.tpl:11:27: executing "common.labels.standard" at <include "common.names.name" .>: error calling include: template: test/charts/mariadb/charts/common/templates/_names.tpl:11:18: executing "common.names.name" at <.Chart.Name>: nil pointer evaluating interface {}.Name helm.go:84: [debug] template: test/charts/postgresql/templates/primary/svc.yaml:11:14: executing "test/charts/postgresql/templates/primary/svc.yaml" at <include "common.labels.standard" (dict "customLabels" .Values.commonLabels "context" $)>: error calling include: template: test/charts/mariadb/charts/common/templates/_labels.tpl:11:27: executing "common.labels.standard" at <include "common.names.name" .>: error calling include: template: test/charts/mariadb/charts/common/templates/_names.tpl:11:18: executing "common.names.name" at <.Chart.Name>: nil pointer evaluating interface {}.Name

jessebot · 2023-11-20T10:19:20Z

Thank you, @wrenix for tending to this!

A new release, 4.5.2, should be out since this job completed. We tested it in the pipeline and I tested it on my VPS via Argo CD before pushing it up.

@ishioni , @DanishVaid , @djbobyd , @nafets227 can you all please let me know if your errors are resolved after testing the latest release?

Thanks everyone for their patience and kind vibes 💙

_{edit: the release job hadn't finished yet, so I had to link to it 🤦}

ishioni · 2023-11-20T11:31:08Z

Job's finished but the chart doesn't seem to available yet :/

nafets227 · 2023-11-20T11:40:47Z

Job 81 needs to finish before we can see the new chart version in the helm repo (it's pushing the content from gh-pages branch to the GitHub pages).

I´m waiting for it, too, to start testing

jessebot · 2023-11-20T11:42:19Z

@nafets227 great catch! Forgot that it takes a bit because I think the nextcloud org is low on GHA minutes and they're still implementing self hosted runners. Sorry about that, @ishioni 🙏

jessebot · 2023-11-20T12:03:53Z

ok, that job has completed and after:

helm repo update
helm show chart nextcloud/nextcloud

I finally get:

apiVersion: v2
appVersion: 27.1.3
dependencies:
- condition: postgresql.enabled
  name: postgresql
  repository: oci://registry-1.docker.io/bitnamicharts
  version: 12.12.*
- condition: mariadb.enabled
  name: mariadb
  repository: oci://registry-1.docker.io/bitnamicharts
  version: 12.2.*
- condition: redis.enabled
  name: redis
  repository: oci://registry-1.docker.io/bitnamicharts
  version: 17.13.*
description: A file sharing server that puts the control and security of your own
  data back into your hands.
home: https://nextcloud.com/
icon: https://cdn.rawgit.com/docker-library/docs/defa5ffc7123177acd60ddef6e16bddf694cc35f/nextcloud/logo.svg
keywords:
- nextcloud
- storage
- http
- web
- php
maintainers:
- email: [email protected]
  name: skjnldsv
- email: [email protected]
  name: chrisingenhaag
- email: [email protected]
  name: billimek
name: nextcloud
sources:
- https://github.com/nextcloud/helm
- https://github.com/nextcloud/docker
version: 4.5.2

nafets227 · 2023-11-20T12:15:16Z

same here:

[root@phys1Ci ~]helm search repo nextcloud
NAME               	CHART VERSION	APP VERSION	DESCRIPTION                                       
nextcloud/nextcloud	4.5.2        	27.1.3     	A file sharing server that puts the control and...

CI Tests are running ...

nafets227 · 2023-11-20T12:37:18Z

works here :)

So finally I can confirm that chart version 4.5.2 solves the issue introduced with 4.5.1 for me.

thanks @jessebot @wrenix for quick fix.
And thanks to you and all the next cloud team for the great product and helm chart!

ishioni · 2023-11-20T15:19:38Z

Confirm on my end too

ishioni changed the title ~~[bug] extra config files~~ [bug] extra newline in config files Nov 18, 2023

jessebot mentioned this issue Nov 19, 2023

fix: after #465 (remove before <?php, indent of extraVolumes and podLabels) #476

Merged

3 tasks

jessebot added the bug Something isn't working label Nov 19, 2023

jessebot assigned jessebot and wrenix Nov 20, 2023

ishioni closed this as completed Nov 20, 2023

zumbrunnen mentioned this issue Dec 1, 2023

Environment Variables Parsed From Secrets Get a Newline Appended #490

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[bug] extra newline in config files #475

[bug] extra newline in config files #475

ishioni commented Nov 18, 2023

DanishVaid commented Nov 18, 2023

nafets227 commented Nov 19, 2023

jessebot commented Nov 19, 2023 •

edited

Loading

djbobyd commented Nov 19, 2023

wrenix commented Nov 19, 2023 •

edited

Loading

ishioni commented Nov 19, 2023

DanishVaid commented Nov 19, 2023 •

edited

Loading

jessebot commented Nov 20, 2023 •

edited

Loading

ishioni commented Nov 20, 2023

nafets227 commented Nov 20, 2023

jessebot commented Nov 20, 2023 •

edited

Loading

jessebot commented Nov 20, 2023

nafets227 commented Nov 20, 2023

nafets227 commented Nov 20, 2023

ishioni commented Nov 20, 2023

[bug] extra newline in config files #475

[bug] extra newline in config files #475

Comments

ishioni commented Nov 18, 2023

Describe your Issue

Logs and Errors

Describe your Environment

Additional context, if any

DanishVaid commented Nov 18, 2023

nafets227 commented Nov 19, 2023

jessebot commented Nov 19, 2023 • edited Loading

djbobyd commented Nov 19, 2023

wrenix commented Nov 19, 2023 • edited Loading

ishioni commented Nov 19, 2023

DanishVaid commented Nov 19, 2023 • edited Loading

jessebot commented Nov 20, 2023 • edited Loading

ishioni commented Nov 20, 2023

nafets227 commented Nov 20, 2023

jessebot commented Nov 20, 2023 • edited Loading

jessebot commented Nov 20, 2023

nafets227 commented Nov 20, 2023

nafets227 commented Nov 20, 2023

ishioni commented Nov 20, 2023

jessebot commented Nov 19, 2023 •

edited

Loading

wrenix commented Nov 19, 2023 •

edited

Loading

DanishVaid commented Nov 19, 2023 •

edited

Loading

jessebot commented Nov 20, 2023 •

edited

Loading

jessebot commented Nov 20, 2023 •

edited

Loading