Reading and Writing YAML in Python Safely and Cleanly

Python and YAML meet in three situations: reading configuration, writing configuration and editing it in place. Each has a safe, boring way to do it and a tempting, risky way, and the risky ways are in the first page of every tutorial. This guide collects the boring ways, tested, in one place. It covers which loader to call and why, what the library does to your data, how to make loading strict, how to write clean output, how to edit without destroying comments, and how to layer files and fill in secrets. It ends with a small reference module that does the loading side properly and the tests that show it working.

Which library

Two libraries cover nearly every need, and a command-line tool fills a gap.

PyYAML is the standard. It is small, widely installed, fast with its C extension and implements YAML 1.1. It loads into plain Python data and loses comments and layout when it writes.

ruamel.yaml implements YAML 1.2 by default and has a round-trip mode that keeps comments, key order, quote style and layout. It is slower and has a larger API.

yq (the Go tool) edits YAML files in place from the shell and keeps comments with some spacing normalization. Use it from scripts when a few lines of shell do the job.

A rule of thumb: PyYAML to read configuration, ruamel.yaml to edit files that people also edit, yq for batch edits across a repository. The rest of this guide explains why.

Loading safely

The function to call is yaml.safe_load. It builds dictionaries, lists, strings, numbers, booleans, None and date objects, and refuses everything else. The risk it removes is real. YAML tags can name Python callables, and PyYAML's unsafe loader will call them during parsing. A harmless demonstration:

safe_load: ConstructorError - could not determine a constructor for the tag 'tag:yaml.org,2002:python/object/apply:built...
code ran during load
UnsafeLoader returned: None
yaml.load() with no Loader: TypeError load() missing 1 required positional argument: 'Loader'

The document asked for builtins.print to be called with one argument. safe_load refused; the unsafe loader printed the text during the load and returned None. Replace print with any importable callable and the file decides what runs on your machine. PyYAML 6 turned the optional Loader argument into a required one so that the bare yaml.load(text) of old tutorials fails with a TypeError. Use safe_load for any file that did not come from code you control, and for files that did, because it costs nothing (why yaml load is dangerous has the mechanism, the history and an audit checklist).

Safe loading removes code execution and not resource exhaustion. For untrusted input also limit the size of the file, the nesting depth and alias expansion, and think about running the parse under a time and memory limit.

Loading quickly

yaml.safe_load always uses the pure Python SafeLoader. If the libyaml C library was present when PyYAML was built, a much faster CSafeLoader exists:

SafeLoader (pure Python)     4.16s
CSafeLoader (libyaml)        0.67s
ruamel safe                  5.49s
ruamel round-trip            7.52s

On a 1.4 MB document the C loader took 0.67 seconds, six times faster than the default and eleven times faster than ruamel.yaml's round-trip mode. Use it with yaml.load(text, Loader=yaml.CSafeLoader) behind a getattr fallback to SafeLoader. For configuration files the difference is not visible. For data files and programs that load many files it is, and for bulk data you should consider another format entirely (JSON parsed 50,000 orders in 132 ms where the C YAML loader needed 7.2 seconds, see YAML versus JSON for configuration). Note that the two loaders word some errors differently: libyaml says did not find expected key where the Python loader says expected <block end>.

What the loader does to your values

PyYAML resolves unquoted scalars with YAML 1.1 rules. The cost of that is a set of conversions that look like bugs:

a: NO      -> False
b: 010     -> 8
c: 12:30   -> 750
d: 3.10    -> 3.1
e: 2026-01-05 -> datetime.date(2026, 1, 5)

A country code, a file mode, a time of day, a version and a date, all converted. A CI workflow showed the same family: the key on: became True, and a Python version matrix entry 3.10 became 3.1 in both PyYAML and ruamel.yaml, because 3.10 is a decimal number in every version of YAML. The defenses are quoting in the file, a typed model in code, and knowing which version your parser implements (the Norway problem and YAML boolean surprises, why a number turns into something else in YAML, YAML version numbers turning into floats).

The quickest way to see what your file loads as is to print it back as JSON with default=str and read it. People who do this on a file they have owned for a year usually find one value that has been wrong all along.

Making loading strict

The default loader is permissive in ways that hide mistakes. Three tightenings are worth the code.

Reject duplicate keys. PyYAML keeps the last value of a repeated key and says nothing: a: 1, b: 2, a: 3 loads as {'a': 3, 'b': 2}. A subclass of the safe loader with a replacement mapping constructor raises instead, in about eight lines. ruamel.yaml raises by default, yamllint flags it, yq passes both entries through (YAML duplicate keys and the silent overwrite). Duplicates are created by merges: two branches that each add a timeout can merge cleanly into a file with the key twice, and the later one silently wins.

Reject unknown keys. A misspelled key is ignored by any code that reads known keys. A model with extra="forbid" turns it into an error.

Check types. A field declared str rejects the False that NO became, and reports the value it received. With a typed model, one bad file produced seven errors at once, each with a path such as servers.1.port (validating YAML configuration with pydantic).

The first is a property of the loader, the other two of the validation layer. Use both, and load into the typed object at one place in the program so that nothing downstream handles raw dictionaries.

Writing clean output

yaml.safe_dump has defaults that nobody would write by hand: keys sorted alphabetically, non-ASCII text escaped, multiline strings as quoted scalars with blank lines and list items at their key's indentation. The options that fix them:

yaml.safe_dump(cfg, sort_keys=False, allow_unicode=True, default_flow_style=False, indent=2)

A tiny str subclass with a registered representer writes marked strings as pipe blocks. And ruamel.yaml with indent(mapping=2, sequence=4, offset=2) indents lists under their keys. Always build the data first and dump it: strings assembled by hand break on the first value with a colon, and a Python string "NO" dumped by a library is quoted so that it reads back as a string, where an f-string writes it bare (writing YAML from Python without ugly output). After writing, reload and compare: assert yaml.safe_load(out) == data catches representer mistakes.

Editing files without damage

PyYAML cannot edit a file that humans maintain: it drops comments and rewrites the layout. The same edit through safe_load and safe_dump removed the header comment and the trailing comment, turned a flow list into a block list, removed the quotes around a string and rewrote a block scalar as a single-quoted blob. ruamel.yaml round-trip mode kept all of it:

y = YAML(); y.preserve_quotes = True; y.indent(mapping=2, sequence=4, offset=2)
d = y.load(src); d["service"]["retries"] = 5; d["service"]["region"] = "eu"
d["service"].yaml_add_eol_comment("added by script", "region")

The result kept the header, the quoted name with its trailing comment, the block string and the key order, and added the new key with its own comment. Always dump to a string, diff against the original and write only if the diff contains the intended change. For many files at once, yq is quicker to script, though it normalized one trailing comment's spacing in a test (preserving comments and order with ruamel yaml, using yq to update many YAML files at once).

Several documents and front matter

A YAML stream can contain many documents separated by three dashes. safe_load raises ComposerError for more than one; safe_load_all returns a generator that parses lazily, so consuming it after the file is closed fails with ValueError: I/O operation on closed file. A trailing separator yields a final None document (loading multiple documents from one YAML file).

Markdown files carry YAML front matter between two lines of three dashes. A single anchored regular expression splits it from the body, safe_load parses it, and the surprises are the usual ones: a date becomes a date object that json.dumps refuses, draft: no becomes False, a title with a colon fails to parse, a BOM or Windows line endings break a naive splitter (reading YAML front matter from Markdown files).

Layering and secrets

Real configuration comes in layers: defaults, an environment file, local overrides. The wrong merge is {**base, **override}, which replaces whole nested blocks: an override that set only the host also deleted the port and the ssl option. A recursive merge combines mappings and replaces everything else, and it leaves two policy decisions that code cannot make: lists replace or append, and a null in an override either unsets or is ignored. Write them down (merging two YAML files in Python with a deep merge).

Secrets and per-environment values come from the environment. Substitute them after parsing, by walking the loaded data and replacing placeholders in string values. Expanding the text first failed when a password was p@ss: word # 1, with mapping values are not allowed here, and would have let the environment inject keys. Placeholders in flow style ({h: ${X}}) fail because of their own braces, so quote them or use block style (environment variable substitution in YAML config files).

A reference module

Here is a loading module that combines the strict loader, the post-parse expansion, the top-level type check and a layered merge, in about sixty lines:

import copy, os, re
import yaml
from yaml.constructor import ConstructorError

class ConfigError(Exception):
    pass

class _Strict(getattr(yaml, "CSafeLoader", yaml.SafeLoader)):
    pass

def _no_duplicates(loader, node, deep=False):
    seen = set()
    for key_node, _ in node.value:
        key = loader.construct_object(key_node, deep=deep)
        if key in seen:
            raise ConstructorError(None, None, f"duplicate key {key!r}", key_node.start_mark)
        seen.add(key)
    return loader.construct_mapping(node, deep)

_Strict.add_constructor(yaml.resolver.BaseResolver.DEFAULT_MAPPING_TAG, _no_duplicates)
_ENV = re.compile(r"\$\{(\w+)(?::-([^}]*))?\}")

def _expand(node, env):
    if isinstance(node, dict):
        return {k: _expand(v, env) for k, v in node.items()}
    if isinstance(node, list):
        return [_expand(v, env) for v in node]
    if isinstance(node, str):
        def one(m):
            name, default = m.group(1), m.group(2)
            if name in env: return env[name]
            if default is not None: return default
            raise ConfigError(f"environment variable {name} is not set")
        return _ENV.sub(one, node)
    return node

def merge(base, over):
    out = copy.deepcopy(base)
    for k, v in over.items():
        out[k] = merge(out[k], v) if isinstance(v, dict) and isinstance(out.get(k), dict) else copy.deepcopy(v)
    return out

def load(path, env=None):
    text = open(path, encoding="utf-8-sig").read()
    try:
        data = yaml.load(text, Loader=_Strict)
    except yaml.YAMLError as e:
        m = getattr(e, "problem_mark", None)
        where = f" (line {m.line + 1}, column {m.column + 1})" if m else ""
        raise ConfigError(f"{path}: {getattr(e, 'problem', e)}{where}") from e
    if data is None:
        data = {}
    if not isinstance(data, dict):
        raise ConfigError(f"{path}: top level must be a mapping, got {type(data).__name__}")
    return _expand(data, os.environ if env is None else env)

def load_layers(paths, env=None):
    cfg = {}
    for p in paths:
        cfg = merge(cfg, load(p, env))
    return cfg

And what it did on a set of test files, with a password that contains a colon and a hash:

layers : {'env': 'prod', 'server': {'host': 'localhost', 'port': 80}, 'password': 'p@ss: word # 1'}
dup.yaml  : ConfigError dup.yaml: duplicate key 'a' (line 2, column 1)
bad.yaml  : ConfigError bad.yaml: mapping values are not allowed in this context (line 1, column 11)
list.yaml : ConfigError list.yaml: top level must be a mapping, got list
empty.yaml: {}
base.yaml: ConfigError environment variable DB_PASS is not set

Two files were layered (a base and a production file whose host used a default), and the password arrived exactly as set. A duplicate key, a syntax error, a top-level list and a missing variable each became a ConfigError with a message that names the file and, where it can, the position. An empty file became an empty mapping. The module does not validate shapes; pass its result to a typed model. It also leaves a trap visible: the syntax error reads in this context, the libyaml wording, because the module uses the C loader.

Seeing what the parser sees

When a value is not what you expect, stop guessing and look at the three layers PyYAML exposes. yaml.scan(text) yields tokens, the scanner's view. yaml.parse(text) yields events, the parser's view. yaml.compose(text) returns the node tree with each scalar's original text, its resolved tag and its position, before any Python object is built. On a four-line document:

== events
['StreamStartEvent', 'DocumentStartEvent', 'MappingStartEvent', 'ScalarEvent:name', 'ScalarEvent:web', 'ScalarEvent:ports', 'SequenceStartEvent', 'ScalarEvent:80', 'ScalarEvent:443', 'SequenceEndEvent', 'ScalarEvent:no', 'ScalarEvent:x']
== nodes with lines
line 1: key='name' tag=str text='web'
line 2: key='ports' tag=seq text=[...]
line 3: key='no' tag=str text='x'
line 4: key='ver' tag=float text='3.10'

The node tree is the one to reach for. The last line shows the document's ver: 3.10 as a node with the float tag and the original text 3.10, which is exactly the information that is gone once the Python float exists. The same approach powers the suspicious-float scanner from the errors cluster (YAML version numbers turning into floats), and it is how you add line numbers to validation errors: look the path up in the node tree and report the start mark. Events help when the structure itself is in doubt, for instance to see whether a flow list ended where you thought. Tokens are for debugging scanner errors and rarely needed.

For a quick look at types, dump the loaded data as JSON with default=str. For a quick look at a file in a browser, the YAML to JSON tool does it without running code.

Recipes that come up

Compare two files as data, not text. Two YAML files with different layout can be the same configuration. Loading both and comparing gives True for files that differ only in key order and style; a small recursive function prints what actually changed:

equal as data: True
~ /a: 1 -> 2
- /b/d: 3
- /e: [1]
+ /f: 'x'

Each line is a change at a path: ~ changed, - removed, + added. This is a better review aid than a text diff for generated or reformatted files, and a good CI check for "this refactor must not change the effective configuration". Remember that it compares loaded values, so NO and False are equal after the load; compare with the parser your program uses (diffing YAML files without whitespace noise or reordering).

Convert to JSON and back. json.dumps(yaml.safe_load(text), default=str) writes a date as a string, and loading that JSON back and dumping it gives the date as a quoted string, when: '2026-10-27'. The round trip changed the type from date to string, which is usually what you want for an index and is worth knowing when you rely on it. Keys that are not strings (a YAML key 1, or on loaded as True) become strings in JSON, which can merge keys: 1 and "1" collide.

Read many files. Loop over pathlib.Path("config").glob("*.yaml") in sorted order, load each through the same function and collect errors in a list instead of stopping at the first. Print the whole list. One failing file out of forty should not hide the other three that also fail.

Never load from a URL without limits. urllib plus safe_load is three lines, and it parses whatever the server sends, of any size. Set a timeout, cap the number of bytes you read and check the content length before parsing.

Use the type your code needs. If the program needs a list of strings, read it as one: list(map(str, cfg.get("hosts") or [])). Converting at the boundary hides a None for an empty value and a number where a host name was meant, and it should be paired with validation so you notice.

Comparing the libraries

Behavior measured in this cluster, side by side:

The table is a summary of tests, not a ranking. PyYAML wins on speed and installation, ruamel.yaml on fidelity and 1.2, yq on shell convenience. Many projects use two. If you pick one, write down which and why, because a future contributor will import the other and get different behavior.

An application walk-through

Putting the pieces together in one small program shows how little code the safe pattern needs. On start-up the program calls load_layers with the base file, the environment file selected by a variable and an optional local file. The result is a plain dictionary with environment values substituted. The program passes it to Config.model_validate, and on a ValidationError it prints every error with its path to standard error and exits with status 2, before any connection is opened or any work starts. Secrets are SecretStr fields, so printing the configuration for debugging shows **********. The typed object is stored in one module-level variable or passed into the application factory; nothing else reads YAML.

The effect is that every class of mistake in this cluster has a place where it is caught. Syntax errors and duplicate keys are caught by the loader, with the file and position. Missing environment variables are caught by the expansion. Wrong types, bad enumerations, unknown keys and out-of-range values are caught by the model, all at once. A failure costs the person editing the file one run and one read of the message, and it never reaches production, because the same code runs in CI against the files in the repository.

That structure is also what you test. The loader has unit tests for its behaviors, the model has tests for its constraints, and one integration test loads the real files. The tests are the executable form of the rules in this guide, and when YAML behavior changes with a library upgrade, they are what notice.

Python types that do not dump

Writing YAML from Python runs into the edges of what the safe dumper can represent. What each of these produced with safe_dump:

tuple        '- 1\n- 2\n'
set          '!!set\n1: null\n2: null\n'
Decimal      RepresenterError ('cannot represent an object', Decimal('1.5'))
Path         RepresenterError ('cannot represent an object', PosixPath('/tmp'))
bytes        '!!binary |\n  YWI=\n'
tz datetime  '2026-01-01 00:00:00+00:00\n...\n'
int key      '1: a\n'

A tuple becomes a list, which is fine and loses the distinction. A set is written with a !!set tag that only YAML-aware readers understand, so convert sets to sorted lists before dumping if the file is for people or other tools. Decimal and Path raise RepresenterError, because the safe dumper only knows basic types; convert them to str or a number first, or register a representer for the type on SafeDumper. Bytes become a !!binary base64 block, which is correct and not readable. A timezone-aware datetime is written in a form with a trailing ... document end marker when it is the whole document, which looks odd and is valid.

Two behaviors surprise people who expect dump to work like json.dumps. yaml.safe_dump(data, stream) writes to the stream and **returns None**, so out = yaml.safe_dump(data, f) leaves out empty; omit the stream to get the string. And the plain yaml.dump, without the safe prefix, is willing to write arbitrary Python objects with tags that only an unsafe loader can read back, a trap for files meant to be exchanged. Use safe_dump for anything a person or another program will read, and convert your own classes to dictionaries first (writing YAML from Python without ugly output).

The same discipline holds on the loading side. A file written with unsafe tags and then loaded with safe_load fails with a ConstructorError, which is the loader telling you the file was never portable. Fix the writer.

Large files and memory

Loading builds the whole structure in memory, and PyYAML's overhead is several times the file size, as it is for XML trees (how much memory a DOM parser needs). For configuration this is irrelevant. For a multi-megabyte data file you have three options: stream a multi-document YAML file with safe_load_all so that only one document is in memory at a time; convert the data to JSON lines or Parquet once and read that instead; or reconsider YAML for the data. The measured cost of loading a 1.4 MB document, 0.67 seconds with the C loader and four to eleven times that with the Python loader or the round-trip library, grows linearly with size, so a 100 MB file is minutes and gigabytes. Put bulk data in a format built for it, and keep YAML for settings.

If you must process a big YAML file, the C loader and a document-per-record layout are the levers. Splitting a long list into one document per item lets safe_load_all stream it, at the cost of a less natural file. That trade is worth making for a data export you control and not for a hand-edited config.

Testing the YAML layer

Test the module and the configs it reads. For the module: one test per behavior in the output above, plus an awkward value (the password), an unset variable with and without a default, and an empty variable. For the configs: a test that loads every file under config/ through the loader and the typed model, so a bad edit fails the pull request. Add fixtures for each production incident, a file that reproduces the mistake, with a comment saying what it caught. Run yamllint in the same job. If you write YAML, add a round-trip test: dump, load, compare.

Reviewing code that touches YAML

When you review a pull request that reads or writes YAML in Python, a few questions find most problems. Which function loads it, and could the file come from outside? Is a duplicate key or an unknown key possible, and what happens then? Are values that must be strings protected from type guessing, by quotes in the file or by a typed model? If the code writes YAML, does it use a library, and does the output survive being loaded again unchanged? If it edits an existing file, does it keep comments, and is there a diff check? If it expands variables, does it do so after parsing? If it merges files, which list and null policies apply, and are they written down?

These seven questions take a minute per change, and each corresponds to an incident someone has had. A reviewer who asks them consistently finds that the answers become part of the code: a docstring on the merge function that states the list policy, a comment on the loader that says why it is strict, a test named after the duplicate-key incident. That is how a team's knowledge about YAML ends up somewhere other than in a few people's heads.

Security checklist

1. safe_load or a subclass of SafeLoader everywhere; search for yaml.load( without a safe loader, Loader=yaml.Loader, FullLoader and unsafe_load.

2. Size, depth and alias limits for untrusted input, and resource limits on the process.

3. No secrets in the file; placeholders or references resolved after parsing.

4. No logging of expanded configuration.

5. Duplicate keys and unknown keys rejected.

6. A schema or typed model between the dictionary and the program.

7. Custom tags implemented as constructors on your own SafeLoader subclass, never by switching to an unsafe loader.

Related reading

Each topic has its own post: loading with safe_load, why yaml.load is dangerous, writing YAML, round trips with ruamel, multiple documents, validation with pydantic, deep merging, front matter and environment substitution. For the syntax underneath, read YAML syntax you need to stop guessing at, and for errors fixing YAML errors that make no sense.

References: the PyYAML documentation, the ruamel.yaml documentation, the pydantic documentation and the YAML 1.2.2 specification. Convert and inspect a file in the YAML to JSON tool.

When this Python guide does not apply

Another language reads the file. The traps differ by library. Go, Ruby, Java and JavaScript libraries each have their own version rules, defaults and unsafe modes. Test the parser that runs in production.

The YAML is generated by a template engine. Debug the rendered output, and keep placeholders out of flow style.

The data is large. For millions of records use a data format, and keep YAML for settings.

You need a guarantee that no comment is lost. Round-trip libraries are good, and they are not perfect. Diff before writing, and keep the original in version control.

Your versions differ. Behavior described here is for PyYAML 6.0.3, ruamel.yaml 0.19.1 and pydantic 2.14.1. Later releases may change defaults, especially around YAML 1.2 and the round-trip API.