Preserving Comments and Order With ruamel yaml Round Trips

A script that edits a configuration file has an easy job and a hard constraint: leave everything else alone. The file has comments that explain why values are what they are, a key order that people chose, quotes that matter and a layout that the team agreed on. A library that loads into plain dictionaries and writes them back destroys all of it. ruamel.yaml is the Python library built for this job, and it takes a few settings to get a clean result. Here is the test with the output, and the settings.

The sample

# Service settings
service:
  name: "web"        # quoted on purpose
  ports:
  - 80
  - 443
  tags: [a,   b]
  note: |
    keep this
    block
  retries: 3

The edit: change retries to 5, add a region key with a comment.

What round-trip mode does

from ruamel.yaml import YAML
y = YAML()
y.preserve_quotes = True
y.indent(mapping=2, sequence=4, offset=2)
d = y.load(src)
d["service"]["retries"] = 5
d["service"]["region"] = "eu"
d["service"].yaml_add_eol_comment("added by script", "region")
y.dump(d, sys.stdout)

# Service settings
service:
  name: "web"        # quoted on purpose
  ports:
    - 80
    - 443
  tags: [a, b]
  note: |
    keep this
    block
  retries: 5
  region: eu         # added by script

The header comment, the quoted name and its trailing comment, the flow list (with its odd spacing normalized), the block string and the order of keys are all intact. The changed value shows the new number, and the added key carries its own comment, aligned with the other trailing comment. ports is indented under its key because of indent(mapping=2, sequence=4, offset=2), which I chose to match a layout the team might use.

The loaded object behaves like a dictionary (d["service"]["retries"]) and a list, so most code that reads or edits needs no change. The extra methods are for comments: yaml_add_eol_comment, yaml_set_start_comment and yaml_set_comment_before_after_key. Comments attach to keys and list items, and they survive when you move or modify the values around them, but not when you delete the key they are attached to.

Without the settings

Round-trip mode with the defaults still keeps comments and order, with small differences:

y2 = YAML(); d2 = y2.load(src); d2["service"]["retries"] = 5

# Service settings
service:
  name: web          # quoted on purpose
  ports:
  - 80
  - 443
  tags: [a, b]
  note: |
    keep this
    block
  retries: 5

The quotes around web are gone because preserve_quotes defaults to False, and the trailing comment moved over to column 20 because the value got shorter. The list stays at the key's indentation because that is the default, which preserves the original here. If your file has a style, set preserve_quotes = True and the indentation, and compare the result with the original through a diff before writing it. If the diff has only the intended change, the settings match your file.

What PyYAML does to the same edit

yaml.safe_dump(yaml.safe_load(src), sort_keys=False)

service:
  name: web
  ports:
  - 80
  - 443
  tags:
  - a
  - b
  note: 'keep this

    block

    '
  retries: 3

The header comment and the trailing comment are gone, "web" lost its quotes, the flow list became a block list, and the block string became a single-quoted scalar with blank lines. Nothing here is a bug in PyYAML. It builds plain Python data that cannot hold comments or layout, so it writes the data in its own style. For read-only use that is fine. For editing, use ruamel.yaml or a structure-aware tool like yq, which also keeps comments (with small spacing changes) (comments in YAML and what parsers do with them).

Details that matter

YAML 1.2. ruamel.yaml defaults to the 1.2 rules, so NO is a string and 12:30 is not a number, and its output is written accordingly. A file that depends on 1.1 booleans will read differently. If you need 1.1 behavior, set y.version = (1, 1).

Line width. By default long lines are wrapped at 80 columns. Set y.width = 4096 to stop the library from rewrapping long values that you did not change.

Explicit document start. If the original starts with ---, set y.explicit_start = True to keep it.

Duplicate keys. Round-trip mode raises DuplicateKeyError by default, a useful side effect when editing.

Anchors and merge keys. They are preserved as anchors and aliases and reproduced on dump, and merge keys are kept as << entries. Editing a value inside an anchored mapping changes it for all aliases, as in the file's meaning.

Types. Integers keep their written form where possible (hex, octal with underscores are remembered), and floats keep their formatting, which is why a version 3.10 can be written back as 3.10 in round-trip mode, though the parsed value is still the float 3.1. Do not rely on it for versions; quote them.

Speed. On a 1.4 MB document, round-trip loading took 7.52 seconds, ruamel.yaml safe mode 5.49 and PyYAML's C loader 0.67. This is a tool for files people read, not for large data.

A safe editing pattern

1. Load with YAML() and the settings above.

2. Make the minimal change in the loaded structure.

3. Dump to a string, not straight over the file.

4. Diff the string against the original. If any line other than the intended ones changed, adjust the settings or fall back to an in-place edit with yq.

5. Write the file, and reload it in a test to check the values.

6. Keep a backup or rely on version control, and commit the change with its diff.

Automating step 4 is worth the five lines. A script that edits a hundred files and refuses to write any file whose diff is larger than expected stops a formatting regression from landing in a hundred places (using yq to update many YAML files at once shows a bulk-edit workflow).

Adding structure without losing layout

Beyond changing a value, scripts often need to insert a whole new block, append to a list or remove an entry. The round-trip containers support the operations you expect from dictionaries and lists, with the comment machinery carried along. Appending to a list with d["service"]["ports"].append(8080) keeps the existing comments and adds an item in the same style. Inserting a key at a specific position uses insert(pos, key, value) on the mapping, which matters when the file has a convention for the order of keys, such as name first. Removing a key with del removes its trailing comment as well, and the comment above it stays attached to the next key, which can leave a stray comment that now describes the wrong thing. After a deletion, read the diff and move any orphaned comment.

For a new nested block, build it with the same classes the loader uses so that the dumper formats it like the rest of the file: from ruamel.yaml.comments import CommentedMap, create a CommentedMap, fill it and assign it. A plain dictionary also works, and is written in block style with the document's indentation settings. If you want a particular block to be written in flow style, call .fa.set_flow_style() on the container.

Finally, test the edit script on a copy of a real file before pointing it at a directory. Run it, diff the result and read every changed line. Round-trip libraries are good, and files in the wild contain constructs that surprise them: unusual comment placement, anchors nested in lists, tabs in comments, folded scalars with odd indentation. The diff is how you find out, and the cost is a minute.

Related reading

The cluster hub is reading and writing YAML in Python safely. The simpler tool for read-only loading is loading YAML in Python with PyYAML safe_load, and the writing side with PyYAML in writing YAML from Python without ugly output. Documentation: the ruamel.yaml documentation and its PyPI page. Look at the result in the YAML formatter.

When round-tripping with ruamel is not enough

You need byte-identical output. No library guarantees it. Whitespace inside flow collections, comment alignment and line wrapping can change. Use text-level edits for tiny changes.

You edit many files. Round-trip tools are slow per file and drift between versions. Pin the version and check diffs.

You need other formats too. For TOML or JSON with comments, other tools apply.

The file is generated. If a program regenerates the file on each run, comment preservation is moot. Document the generator and keep the comments there.