Handling YAML Uploads From Users Without Getting Burned

A feature that lets users upload YAML, a settings import, a pipeline definition or a dataset description, accepts input from the least trusted source there is. The usual failures are well known: a loader that builds objects, an alias bomb, a file big enough to exhaust memory, a structure that crashes the code after parsing. None of them is hard to prevent, and all of them have been shipped. This post builds the guard as a short function, runs it against ten inputs and explains each check in the order it should run. It belongs to the cluster on YAML security pitfalls, and it uses the mechanisms explained in why loading untrusted YAML can run code and Billion Laughs style attacks through YAML aliases.

The guard

The function does its work in a fixed order, cheapest checks first, and raises one exception type with a message that is safe to show:

import yaml

MAX_BYTES = 64 * 1024
MAX_EXPANDED_NODES = 10_000
MAX_DEPTH = 20


class Rejected(ValueError):
    pass


def expanded_size(node, memo, depth=0):
    if depth > MAX_DEPTH:
        raise Rejected(f"nesting deeper than {MAX_DEPTH}")
    key = id(node)
    if key in memo:
        return memo[key]
    size = 1
    if isinstance(node, yaml.SequenceNode):
        size += sum(expanded_size(c, memo, depth + 1) for c in node.value)
    elif isinstance(node, yaml.MappingNode):
        size += sum(expanded_size(k, memo, depth + 1) + expanded_size(v, memo, depth + 1) for k, v in node.value)
    memo[key] = size
    if size > MAX_EXPANDED_NODES:
        raise Rejected(f"expands to more than {MAX_EXPANDED_NODES} nodes")
    return size


def load_upload(data: bytes):
    if len(data) > MAX_BYTES:
        raise Rejected(f"larger than {MAX_BYTES} bytes")
    try:
        text = data.decode("utf-8")
    except UnicodeDecodeError:
        raise Rejected("not valid UTF-8")
    try:
        node = yaml.compose(text, Loader=yaml.SafeLoader)
    except yaml.YAMLError as e:
        raise Rejected("not valid YAML: " + str(e).splitlines()[0])
    if node is None:
        raise Rejected("empty document")
    expanded_size(node, {})
    try:
        doc = yaml.safe_load(text)
    except yaml.YAMLError as e:
        raise Rejected("not loadable: " + str(e).splitlines()[0])
    if not isinstance(doc, dict):
        raise Rejected("top level must be a mapping")
    if not all(isinstance(k, str) for k in doc):
        raise Rejected("keys must be strings")
    return doc

The order is the design. Each step is cheaper than the next and rules out a class of input before the next step can be hurt by it.

1. Byte cap. The input is bytes, and its length is checked before anything else. This stops the giant-file case with no parsing at all.

2. UTF-8 decode. A document that is not valid text is rejected before the parser sees it, with a message that does not echo the bytes.

3. Compose, do not construct. yaml.compose builds the node graph (the structure with anchors and aliases) without creating Python objects from it. It runs no constructors, so it cannot trigger a tag handler, and it is where syntax errors surface.

4. Measure the expanded size. The function walks the nodes and counts what the document would become if every alias were expanded. A memo keyed by node identity means each distinct node is measured once, so the walk is fast on a bomb: the cost is proportional to the number of distinct nodes, while the result it reports is the expanded count. It stops the moment the count passes the budget.

5. Depth. The same walk tracks nesting. Deeply nested input is its own attack and a source of RecursionError.

6. **safe_load.** Only now are Python objects built, with the safe loader.

7. Shape. The top level must be a mapping with string keys. Further structure is a schema's job.

Ten inputs

The test harness runs each input through load_upload and prints the verdict:

good                   accepted: {'name': 'orders', 'ports': [8080, 8443]}
python tag             rejected: not loadable: could not determine a constructor for the tag 'tag:yaml.org,2002:python/object/apply:os.system'
alias bomb (depth 7)   rejected: expands to more than 10000 nodes
small anchors          accepted: {'base': {'a': 1}, 'use': {'a': 1}}
deep nesting           rejected: nesting deeper than 20
oversize               rejected: larger than 65536 bytes
not a mapping          rejected: top level must be a mapping
non-string keys        rejected: keys must be strings
bad utf-8              rejected: not valid UTF-8
unclosed               rejected: not valid YAML: while parsing a flow sequence

Two accepted, eight rejected, each with a reason that names the rule. Take them in turn.

The python tag passed the compose step, because composing does not look up constructors, and was stopped by safe_load with could not determine a constructor. That is the right place: the safe loader is the defense, and the guard around it adds the limits.

The alias bomb is 378 bytes and was rejected by the expanded-size count before safe_load ever ran. The distinction matters, because safe_load itself would have returned in under a millisecond and left the bomb to detonate later in serialization. Small anchors were accepted, since a document may legitimately reuse a definition; the count of two copies was far under the budget.

Deep nesting (200 levels of [) was refused by the depth check with a specific message. Without it, the structure might hit Python's recursion limit inside the loader or in the code that walks the result, which turns into a RecursionError and an HTTP 500.

Oversize was refused on length alone. Bad UTF-8 and an unclosed bracket were refused with messages that describe the category and not the input. Notice that the syntax error message includes only the first line of the parser's error, which carries a position; echoing the whole error can include a snippet of the file, and you may not want to print user content into a response or a log line.

What the guard does not do

It does not decide whether the contents are acceptable. A guard that returns a dictionary has established that the upload is a bounded, plain-data mapping. Everything after that is your application's job.

Validate against a schema. Required keys, types, ranges and additionalProperties: false turn "a mapping" into "a valid pipeline definition". Validating YAML with JSON Schema shows the schema and the messages, and the loader-dependent surprises with no and 010 apply to uploads as much as to files: the schema must be run on the data the guard returned.

Never use values as code or paths. A string that becomes a filename allows ../../etc/passwd. A string that becomes a shell command is a remote command execution bug. A string that becomes a template is a server-side template injection. Allow-list the values, or look them up in a table you control.

Bound the work later too. A valid, small document can still describe a large job. If a field says "repeat 1,000,000,000 times", the guard has nothing to say about it.

Handle secrets. If the upload may contain credentials, do not log it, do not echo it back in errors and consider scanning it. The leak mechanics are in secrets accidentally committed in YAML files.

Operational controls

Put controls around the code as well. Run the parser in a process with a memory limit and a timeout, so one pathological input cannot take the web server down. Rate-limit uploads per user. Store the original bytes, not only the parsed result, if you need an audit trail, and store them where they are never executed or served as HTML. Return a generic error to the user and the specific one to the log. Test the guard with new inputs when you find a bypass, keeping a corpus of the attacks that have been tried. A fuzzer pointed at load_upload is cheap to set up and finds crashes in the guard itself.

When a guard function is not enough for YAML uploads

You do not need YAML. If users upload structured data and the features of YAML are not part of the product, accept JSON. It has no anchors, no tags, one meaning per value and a smaller attack surface.

Files are large by design. A dataset import of hundreds of megabytes needs streaming and a different format. YAML is the wrong container.

The document drives complex behavior. A pipeline definition that can run steps needs a policy layer and sandboxing of its own, beyond parsing.

The parser is not Python's. Other languages have their own limits and flags. The same order of checks applies, and the options differ: see limiting alias expansion in a YAML parser for Go, Java, Ruby and DuckDB.

The functions used are documented in the PyYAML reference, and OWASP's guidance on file upload handling is in the File Upload Cheat Sheet. To view an uploaded file's structure safely in a browser, use the YAML formatter.