Helm Values Files Without Copy Paste Using Layered Values

A Helm chart is a template, and its configuration is a values file. The pattern that goes wrong is the one every environment ends up with: three near-identical values files of 300 lines, copied and edited, so that a change to a default means editing all three, and the differences between production and staging are impossible to see. Helm's answer is layering. You keep the defaults in one file and pass only the differences for each environment. The rules for how layers combine are simple and have one trap, which is how lists behave. This post demonstrates the merge with a small chart rendered by Helm, and checks the output. It belongs to the cluster on YAML for infrastructure files developers edit weekly, and it applies the ideas in environment specific YAML config without copy paste.

The chart and its defaults

The chart renders a PostgreSQL StatefulSet. Its defaults:

image:
  repository: postgres
  tag: "18.6"
replicas: 1
resources:
  limits: {cpu: 500m, memory: 512Mi}
extraArgs: ["-c", "max_connections=50"]
env:
  POSTGRES_USER: app
  POSTGRES_DB: shop

and the part of the template that uses them:

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: {{ .Release.Name }}-db
spec:
  serviceName: {{ .Release.Name }}-db
  replicas: {{ .Values.replicas }}
  selector:
    matchLabels: {app: {{ .Release.Name }}-db}
  template:
    metadata:
      labels: {app: {{ .Release.Name }}-db}
    spec:
      containers:
        - name: postgres
          image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"
          args: {{ toJson .Values.extraArgs }}
          env:
            {{- range $k, $v := .Values.env }}
            - name: {{ $k }}
              value: {{ $v | quote }}
            {{- end }}
          resources: {{- toYaml .Values.resources | nindent 12 }}

Note two idioms: toYaml ... | nindent 12 turns a values subtree into YAML at the right indentation, and quote turns each environment value into a string, the same string rule as in ConfigMaps and Secrets in YAML. The image tag is quoted in values.yaml ("18.6") so YAML does not read it as a float.

The environment file contains only what differs:

replicas: 3
image:
  tag: "18.6"
resources:
  limits: {memory: 2Gi}
extraArgs: ["-c", "max_connections=200", "-c", "shared_buffers=512MB"]
env:
  POSTGRES_DB: shop_prod

How layers combine

Render with the environment file on top of the defaults: helm template r db -f prod.yaml. Compare the output of the defaults alone with the layered result.

== base
  replicas: 1
          args: ["-c","max_connections=50"]
            - name: POSTGRES_DB
              value: "shop"
              cpu: 500m
              memory: 512Mi
== with prod.yaml
  replicas: 3
          args: ["-c","max_connections=200","-c","shared_buffers=512MB"]
            - name: POSTGRES_DB
              value: "shop_prod"
              value: "app"
              cpu: 500m
              memory: 2Gi

Two rules show up clearly.

Maps merge key by key. prod.yaml set only resources.limits.memory, and the rendered chart has memory 2Gi and CPU still 500m from the base. The environment map kept POSTGRES_USER: app (from the base) and changed POSTGRES_DB to shop_prod. This is why an environment file can be five lines. You override the leaf you mean, and everything else is inherited.

Lists replace. extraArgs in prod.yaml is a list, and the rendered args hold exactly those four items. The base's max_connections=50 is gone, not merged. Helm has no concept of merging two lists, since there is no key to match items by. If you override a list, you restate every item you still want. This catches teams out most with lists of environment variables written as env: [{name: A, value: x}], where overriding one entry replaces all of them. The map form used here (env: {POSTGRES_DB: shop}) avoids the problem, which is why charts that care about layering prefer maps to lists of name/value pairs.

Flags are the top layer. --set beats files:

== --set
  replicas: 5
          args: ["-c","max_connections=10"]

--set replicas=5 overrode the 3 from prod.yaml, and --set 'extraArgs={-c,max_connections=10}' replaced the list again. Files given later on the command line beat earlier ones, and --set beats all files. That order lets CI inject a value (an image tag for a build) without editing a file. Keep --set to a few values of that kind; a deploy built from twenty flags is as hard to read as three copies of a file.

Rendering does not validate

Helm will happily render nonsense. We changed replicas: 3 to replicas: three in the environment file:

== bad
  replicas: three

helm template printed the StatefulSet with replicas: three and no error, because the template inserts the value into text without knowing its type. The mistake appears only when something validates the output. kubeconform -strict on the rendered file reported the StatefulSet invalid, while the good render was Valid: 1. So the pipeline is: lint the chart, render for each environment, validate the rendered output. helm lint passed our chart with a single informational note (icon is recommended), and it checks the chart's structure, not the values you supply.

The chart can defend itself: a values.schema.json file next to values.yaml makes Helm validate every values file against a JSON Schema at install, template and lint time, with an error that names the field. Adding a schema with replicas as an integer would have failed the bad render at the helm template step. We did not add one in this test, and for charts that other people use, it is worth the half hour. Schema writing is covered in validating YAML with JSON Schema.

Practical conventions

• **Defaults in values.yaml, differences in values-<env>.yaml.** Name the files so a directory listing tells you the environments.

• Prefer maps to lists for anything an environment may partly override.

• Quote strings that look like numbers, such as image tags and version strings.

• Keep secrets out. Reference a Kubernetes Secret created elsewhere, and keep literal passwords out of values files, as in keeping secrets out of YAML config files.

• Render in CI for every environment, and validate the output with the tools from checking Kubernetes YAML before you apply it.

• Diff the rendered output between releases (helm template before and after, then diff). Reviewers can read manifests more easily than a change to a values file.

Reading a layered result

When a rendered value surprises you, look at the merged values and not at the rendered manifest. The simplest way to see them before anything is installed is a throwaway template that prints them, such as templates/debug.yaml containing # {{ toYaml .Values | nindent 2 }}, which made helm template print the merged values (our run showed env.POSTGRES_DB: shop_prod, the four-item extraArgs, replicas: 3 and the rest). Only the first line is a comment, so use it to read and never to apply, and delete it afterward. For a release that is already installed, helm get values <release> --all shows what it received, including chart defaults. Compare that output with each layer in turn to learn which file supplied each value. (helm template --debug in the version we ran printed debug log lines about the chart path and dependencies, not the computed values.) Do this before editing any file. Most "Helm ignored my setting" reports turn out to be a key typed under the wrong parent, where Helm silently accepts a key the template never reads. The same drift is why the schema file is worth having: it turns an ignored key into an error.

When layered Helm values stop being manageable

The files have grown to override half the chart. If production's file restates most of the defaults, the layering is not helping; consider whether the chart has the wrong defaults or too many knobs.

You need conditional logic in values. Values files are data. Logic belongs in templates or in a tool above Helm (Helmfile, Kustomize, a CD system).

Many clusters differ in many ways. A combinatorial set of environment files needs a generator or a hierarchy, such as a base, a region layer and a cluster layer, and tooling that composes them.

The chart is not yours. Upstream charts change values between versions, and overrides for removed keys are ignored silently. Pin the chart version and diff helm show values on every upgrade.

The merge behavior is described in the Helm documentation on values files, and schema validation in the chart best practices. Tidy a values file with the YAML formatter.