Finding Inconsistent Versions Across YAML Configs in a Repo

The same database appears in five kinds of YAML in a typical repository: the Compose file that runs it locally, the CI workflow that tests against it, the Helm values that deploy it, the Kubernetes manifest and the docs. Each pins a version, and each pin was right on the day it was written. A year later they disagree, and the test environment is not the production environment. Finding the disagreement is a text-and-structure problem, and this post solves it in a page of Python. It belongs to the cluster on analyzing configuration and metadata held in YAML, and it applies the table approach of counting services across many Compose files.

Two kinds of version references

An image version appears in YAML in one of two shapes.

A single string: image: postgres:18. Compose, Kubernetes, GitHub Actions service containers and GitLab CI all use this form. A regular expression over the text finds it:

IMG = re.compile(r"^\s*(?:-\s*)?image:\s*['\"]?([a-z0-9/._-]+):([A-Za-z0-9._-]+)['\"]?\s*$", re.M)

The pattern allows an optional list dash (- image: ...), optional quotes and a registry path in the name, and it captures name and tag.

Split keys: Helm values often hold repository and tag separately:

image:
  repository: orders
  tag: "1.4.2"

No regex over single lines sees that pair as one reference, so those files need a structured read: load the YAML and combine the two keys. Helm values can also hold postgres: {image: postgres:17} in a nested value, which the regex finds, as in our files. A scan that handles both forms covers most of a repository; charts with templated tags ({{ .Values.image.tag }}) need the values file read together with Chart.yaml's appVersion.

The scan

Walk every YAML file, collect references into a nested dictionary keyed by image name, then by tag, holding the files:

found = defaultdict(lambda: defaultdict(list))
for p in sorted(Path("repo").rglob("*.y*ml")):
    for name, tag in IMG.findall(p.read_text()):
        found[name][tag].append(str(p))
for p in sorted(Path("repo/helm").glob("*.yaml")):
    v = yaml.safe_load(p.read_text())
    found[v["image"]["repository"]][str(v["image"]["tag"])].append(str(p))

The str() on the Helm tag matters: an unquoted tag: 1.10 loads as the float 1.1, and the version you report would be wrong (YAML version numbers turning into floats). The Helm files here quote their tags. Printing the count of files per tag for three images:

postgres   {'15': 6, '16': 2, '17': 3, '18': 6, '18.6': 3}
redis      {'6-alpine': 5, '7-alpine': 3, '7.4': 8}
app        {'1.4.2': 6, '1.5.0': 4, 'latest': 4}

Read the first line: PostgreSQL is referenced in 20 places under five tags, and the majors 15, 16, 17 and 18 all appear. Redis has three tags and app has two pinned versions plus latest. A list like this is the answer to "are we consistent", and in this repository the answer is no.

The majors, collapsed:

postgres majors: ['15', '16', '17', '18']

Four major versions of one database. That matters because major versions differ in behavior, defaults and file formats (the PostgreSQL 18 image changed its data directory, for example). A CI workflow testing on 16 does not tell you that production on 18 works. The scan makes the offenders listable:

files pinned to 16 or 17 while others use 18: ['billing-values.yaml', 'ci0.yml', 'ci1.yml', 'orders-values.yaml', 'stack09.yaml'] ...

and the unpinned references:

latest tags: {'app': ['stack01.yaml', 'stack05.yaml', 'stack06.yaml'], 'billing': ['workloads.yaml'], 'etl': ['workloads.yaml']}

latest is the worst tag for reproducibility: two environments built a week apart run different code. Treat any latest in a deployed file as a defect.

From a report to a rule

A list of disagreements is a start. A rule turns it into a gate. Write down the expected version per image in one place:

expected:
  postgres: "18"
  redis: "7"
  mysql: "8.4"

and have the script compare each reference's major version with the expectation, listing files that deviate and failing the job if any do. Allow explicit exceptions with a reason (legacy-reporting uses 16 until migration X), so deviations are decisions on the record and not drift. Run the script in CI on every pull request that touches YAML, and on a schedule to catch files that changed elsewhere. Pair it with a dependency bot: tools such as Renovate and Dependabot read these files and open pull requests that bump versions, which reduces drift at the source, and the scan checks that nothing was missed.

Be careful about what "consistent" means. Compose files for local development and the production manifests may legitimately differ in minor version. Compare majors for databases and exact tags for your own application images, and decide per image. A scan that complains about everything is ignored.

Pitfalls in the scan itself

Regex over text matches comments. A commented-out # image: postgres:14 counts. Strip comments or parse the YAML for the cases where it matters.

Variables. image: postgres:${PG_VERSION} captures the tag ${PG_VERSION}; report it separately and resolve the variable from the .env file or the defaults.

Digests. image: postgres@sha256:... has no colon-tag form; extend the pattern if you pin by digest.

Multi-line and anchored values. An image set through an anchor appears once in the text and many times in the effective data. Expand anchors before scanning if reuse is heavy.

Duplicated discovery. Files in vendored directories and node_modules pollute the results. Exclude them.

Deciding what the report should say

A scan can produce a wall of text, and the report is only useful if it ends in a decision. Three shapes work. A summary per image with the number of distinct tags and files (the three lines above) tells you where drift exists. A per-file list of deviations from the expected version tells the owner of each file what to change. And a trend over time, the number of distinct majors per image on each run, tells the team whether the situation is improving. Keep the last two in CI output where reviewers will see them, and keep the first in a dashboard or a weekly message.

Decide who owns the expectation. A file of expected versions nobody maintains is another source of drift, and a version bump should be a pull request that changes the expectation and the files together. The scan is then the reviewer's cross-check: if the pull request upgrades the Compose file and forgets the workflow, the job fails and says which file.

Finally, handle the awkward cases explicitly. A file that should differ (a migration test that runs the old version on purpose) gets an exception line with a reason and an expiry date, so exceptions do not become permanent. A scan that reports 40 exceptions has told you something too: the expectation may be wrong.

When a version scan of YAML files is not enough

The version is in code. Dockerfiles, lock files and package manifests hold versions that YAML does not. Scan those too.

The running system differs. What is deployed matters more than what is declared. Compare with a query to the cluster or registry.

Compatibility is not equality. Two minor versions can be compatible, and two patches of one major can differ in a bug that matters. A version list is not a compatibility analysis.

The files are generated. Fix the generator.

The extraction uses plain Python; the YAML versions caveat is in the YAML 1.2.2 specification, and image reference syntax is described in the Docker documentation. Inspect a file in the YAML formatter.